2023-11-16T16:16:28.540 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-16T16:16:28.540 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918 --owner scheduled_choffman@teuthology --name choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi --verbose --description fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/misc}} -- /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918/orig.config.yaml 2023-11-16T16:16:28.643 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T16:16:28.745 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918 branch: wip-choffman-62793 description: fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/misc}} email: null first_in_suite: false job_id: '7460918' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-choffman-62793 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-choffman-62793 sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 owner: scheduled_choffman@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4433 sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 sleep_before_teardown: 0 suite: fs:workload suite_branch: wip-choffman-62793 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi093.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi125.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-16_16:09:17 tube: smithi user: choffman verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1224587 2023-11-16T16:16:28.745 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa; will attempt to use it 2023-11-16T16:16:28.747 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks 2023-11-16T16:16:28.747 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-16T16:16:28.750 INFO:teuthology.task.internal:Checking packages... 2023-11-16T16:16:28.772 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '5e4a743ad7fd1422ebd6c4dc950a21a303279207' 2023-11-16T16:16:28.772 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-16T16:16:28.772 INFO:teuthology.packaging:ref: None 2023-11-16T16:16:28.772 INFO:teuthology.packaging:tag: None 2023-11-16T16:16:28.772 INFO:teuthology.packaging:branch: wip-choffman-62793 2023-11-16T16:16:28.773 INFO:teuthology.packaging:sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:16:28.773 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-choffman-62793 2023-11-16T16:16:28.992 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3905.g5e4a743a 2023-11-16T16:16:28.992 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-16T16:16:29.002 INFO:teuthology.task.internal:no buildpackages task found 2023-11-16T16:16:29.002 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-16T16:16:29.038 INFO:teuthology.task.internal:Saving configuration 2023-11-16T16:16:29.057 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-16T16:16:29.085 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-16T16:16:29.107 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 16:10:25.786979', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T16:16:29.128 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 16:10:25.788387', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T16:16:29.153 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 16:10:25.789645', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T16:16:29.153 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-16T16:16:29.158 INFO:teuthology.task.internal:roles: ubuntu@smithi093.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-11-16T16:16:29.159 INFO:teuthology.task.internal:roles: ubuntu@smithi125.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-11-16T16:16:29.159 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-11-16T16:16:29.159 INFO:teuthology.run_tasks:Running task console_log... 2023-11-16T16:16:29.288 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdc5d533e50>, signals=[15]) 2023-11-16T16:16:29.289 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-16T16:16:29.295 INFO:teuthology.task.internal:Opening connections... 2023-11-16T16:16:29.295 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi093.front.sepia.ceph.com 2023-11-16T16:16:29.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:16:29.365 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi125.front.sepia.ceph.com 2023-11-16T16:16:29.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:16:29.433 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2023-11-16T16:16:29.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:16:29.503 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-16T16:16:29.509 DEBUG:teuthology.orchestra.run.smithi093:> uname -m 2023-11-16T16:16:29.531 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64 2023-11-16T16:16:29.531 DEBUG:teuthology.orchestra.run.smithi093:> cat /etc/os-release 2023-11-16T16:16:29.591 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Red Hat Enterprise Linux" 2023-11-16T16:16:29.591 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="8.6 (Ootpa)" 2023-11-16T16:16:29.591 INFO:teuthology.orchestra.run.smithi093.stdout:ID="rhel" 2023-11-16T16:16:29.591 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="fedora" 2023-11-16T16:16:29.591 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="8.6" 2023-11-16T16:16:29.592 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el8" 2023-11-16T16:16:29.592 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-16T16:16:29.592 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31" 2023-11-16T16:16:29.592 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-16T16:16:29.592 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.redhat.com/" 2023-11-16T16:16:29.592 INFO:teuthology.orchestra.run.smithi093.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-16T16:16:29.592 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T16:16:29.592 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:16:29.592 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T16:16:29.592 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-16T16:16:29.592 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-16T16:16:29.592 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-16T16:16:29.593 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server 2023-11-16T16:16:29.615 DEBUG:teuthology.orchestra.run.smithi125:> uname -m 2023-11-16T16:16:29.635 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64 2023-11-16T16:16:29.635 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2023-11-16T16:16:29.695 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="Red Hat Enterprise Linux" 2023-11-16T16:16:29.695 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="8.6 (Ootpa)" 2023-11-16T16:16:29.695 INFO:teuthology.orchestra.run.smithi125.stdout:ID="rhel" 2023-11-16T16:16:29.695 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="fedora" 2023-11-16T16:16:29.695 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="8.6" 2023-11-16T16:16:29.695 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el8" 2023-11-16T16:16:29.695 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-16T16:16:29.695 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31" 2023-11-16T16:16:29.696 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-16T16:16:29.696 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://www.redhat.com/" 2023-11-16T16:16:29.696 INFO:teuthology.orchestra.run.smithi125.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-16T16:16:29.696 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T16:16:29.696 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:16:29.696 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T16:16:29.696 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-16T16:16:29.696 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-16T16:16:29.696 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-16T16:16:29.696 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2023-11-16T16:16:29.715 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2023-11-16T16:16:29.735 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2023-11-16T16:16:29.735 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2023-11-16T16:16:29.798 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="Red Hat Enterprise Linux" 2023-11-16T16:16:29.798 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8.6 (Ootpa)" 2023-11-16T16:16:29.798 INFO:teuthology.orchestra.run.smithi159.stdout:ID="rhel" 2023-11-16T16:16:29.798 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="fedora" 2023-11-16T16:16:29.798 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8.6" 2023-11-16T16:16:29.799 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8" 2023-11-16T16:16:29.799 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-16T16:16:29.799 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2023-11-16T16:16:29.799 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-16T16:16:29.799 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://www.redhat.com/" 2023-11-16T16:16:29.799 INFO:teuthology.orchestra.run.smithi159.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-16T16:16:29.799 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T16:16:29.799 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:16:29.799 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T16:16:29.799 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-16T16:16:29.799 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-16T16:16:29.800 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-16T16:16:29.800 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2023-11-16T16:16:29.823 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-16T16:16:29.833 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-16T16:16:29.859 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-16T16:16:29.859 DEBUG:teuthology.orchestra.run.smithi093:> test '!' -e /home/ubuntu/cephtest 2023-11-16T16:16:29.862 DEBUG:teuthology.orchestra.run.smithi125:> test '!' -e /home/ubuntu/cephtest 2023-11-16T16:16:29.864 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2023-11-16T16:16:29.881 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-16T16:16:29.893 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-16T16:16:29.893 DEBUG:teuthology.orchestra.run.smithi093:> test -z $(ls -A /var/lib/ceph) 2023-11-16T16:16:29.921 DEBUG:teuthology.orchestra.run.smithi125:> test -z $(ls -A /var/lib/ceph) 2023-11-16T16:16:29.924 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2023-11-16T16:16:29.958 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-16T16:16:30.036 INFO:teuthology.run_tasks:Running task kernel... 2023-11-16T16:16:30.051 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-11-16T16:16:30.052 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-11-16T16:16:30.052 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-11-16T16:16:30.052 DEBUG:teuthology.orchestra.run.smithi093:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-16T16:16:30.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:16:30.071 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-11-16T16:16:30.218 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-16T16:16:30.218 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-11-16T16:16:30.219 DEBUG:teuthology.orchestra.run.smithi093:> uname -r 2023-11-16T16:16:30.238 INFO:teuthology.orchestra.run.smithi093.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-16T16:16:30.238 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-16T16:16:30.239 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-16T16:16:30.239 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum upgrade -y linux-firmware 2023-11-16T16:16:31.425 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-11-16T16:16:31.425 INFO:teuthology.orchestra.run.smithi093.stdout:Unable to read consumer identity 2023-11-16T16:16:32.863 INFO:teuthology.orchestra.run.smithi093.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-11-16T16:16:33.538 INFO:teuthology.orchestra.run.smithi093.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2023-11-16T16:16:37.038 INFO:teuthology.orchestra.run.smithi093.stdout:lab-extras 458 kB/s | 24 kB 00:00 2023-11-16T16:16:37.825 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-11-16T16:16:37.826 INFO:teuthology.orchestra.run.smithi093.stdout:Nothing to do. 2023-11-16T16:16:37.826 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-11-16T16:16:37.946 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.0... 2023-11-16T16:16:37.946 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-16T16:16:38.088 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/ 2023-11-16T16:16:38.088 DEBUG:teuthology.orchestra.run.smithi093:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/ --input-file=- 2023-11-16T16:16:38.090 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi093.front.sepia.ceph.com to complete... 2023-11-16T16:17:02.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16 16:17:02 URL:https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [383129572/383129572] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-16T16:17:02.514 INFO:teuthology.orchestra.run.smithi093.stderr:FINISHED --2023-11-16 16:17:02-- 2023-11-16T16:17:02.515 INFO:teuthology.orchestra.run.smithi093.stderr:Total wall clock time: 24s 2023-11-16T16:17:02.515 INFO:teuthology.orchestra.run.smithi093.stderr:Downloaded: 1 files, 365M in 24s (15.2 MB/s) 2023-11-16T16:17:02.516 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.0... 2023-11-16T16:17:02.516 DEBUG:teuthology.orchestra.run.smithi093:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-16T16:17:04.125 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying... ######################################## 2023-11-16T16:17:04.179 INFO:teuthology.orchestra.run.smithi093.stdout:Preparing... ######################################## 2023-11-16T16:17:04.263 INFO:teuthology.orchestra.run.smithi093.stdout:Updating / installing... 2023-11-16T16:17:42.620 INFO:teuthology.orchestra.run.smithi093.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-11-16T16:17:44.090 INFO:teuthology.orchestra.run.smithi093.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-16T16:18:08.466 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi093.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-16T16:18:08.466 DEBUG:teuthology.orchestra.run.smithi093:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-16T16:18:08.563 INFO:teuthology.orchestra.run.smithi093.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-16T16:18:08.563 INFO:teuthology.orchestra.run.smithi093.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-11-16T16:18:08.563 INFO:teuthology.orchestra.run.smithi093.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-11-16T16:18:08.563 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-11-16T16:18:08.563 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-11-16T16:18:08.563 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-11-16T16:18:08.563 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-11-16T16:18:08.563 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-11-16T16:18:08.563 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-16T16:18:08.563 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-16T16:18:08.563 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-16T16:18:08.564 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-16T16:18:08.564 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-16T16:18:08.564 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-16T16:18:08.564 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-16T16:18:08.564 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-16T16:18:08.564 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-16T16:18:08.564 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-16T16:18:08.564 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-16T16:18:08.564 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-16T16:18:08.564 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-16T16:18:08.564 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-16T16:18:08.564 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-16T16:18:08.565 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-11-16T16:18:08.566 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-11-16T16:18:08.566 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-11-16T16:18:08.566 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-11-16T16:18:08.566 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-11-16T16:18:08.566 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-11-16T16:18:08.566 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-11-16T16:18:08.566 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-11-16T16:18:08.566 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-16T16:18:08.566 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-11-16T16:18:08.566 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-11-16T16:18:08.566 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-11-16T16:18:08.567 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-11-16T16:18:08.568 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-11-16T16:18:08.569 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-11-16T16:18:08.569 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-11-16T16:18:08.569 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-11-16T16:18:08.569 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-11-16T16:18:08.569 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-11-16T16:18:08.569 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-11-16T16:18:08.569 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-11-16T16:18:08.569 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-11-16T16:18:08.569 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-11-16T16:18:08.569 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-11-16T16:18:08.569 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-11-16T16:18:08.569 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-11-16T16:18:08.570 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-11-16T16:18:08.571 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-11-16T16:18:08.572 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-11-16T16:18:08.573 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-11-16T16:18:08.574 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-11-16T16:18:08.575 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-11-16T16:18:08.576 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-16T16:18:08.577 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-11-16T16:18:08.578 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-11-16T16:18:08.579 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-11-16T16:18:08.580 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-11-16T16:18:08.581 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-11-16T16:18:08.582 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-11-16T16:18:08.582 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-11-16T16:18:08.582 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-11-16T16:18:08.582 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-11-16T16:18:08.582 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-16T16:18:08.582 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-11-16T16:18:08.582 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-11-16T16:18:08.582 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-16T16:18:08.582 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-11-16T16:18:08.582 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-11-16T16:18:08.582 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-11-16T16:18:08.582 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-11-16T16:18:08.583 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-11-16T16:18:08.584 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-11-16T16:18:08.585 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-11-16T16:18:08.585 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-16T16:18:08.585 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-11-16T16:18:08.585 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-16T16:18:08.585 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-16T16:18:08.585 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-16T16:18:08.585 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-16T16:18:08.585 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-16T16:18:08.585 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-16T16:18:08.585 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-16T16:18:08.585 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-11-16T16:18:08.586 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-11-16T16:18:08.587 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-11-16T16:18:08.588 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-11-16T16:18:08.589 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-11-16T16:18:08.590 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-11-16T16:18:08.591 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-11-16T16:18:08.591 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-11-16T16:18:08.591 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-16T16:18:08.591 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-11-16T16:18:08.591 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-11-16T16:18:08.591 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-11-16T16:18:08.591 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-11-16T16:18:08.591 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-11-16T16:18:08.591 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-11-16T16:18:08.591 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-11-16T16:18:08.591 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-11-16T16:18:08.591 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-11-16T16:18:08.592 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-11-16T16:18:08.593 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-11-16T16:18:08.594 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-11-16T16:18:08.595 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-11-16T16:18:08.596 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-11-16T16:18:08.596 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-16T16:18:08.596 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-11-16T16:18:08.596 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-11-16T16:18:08.596 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-11-16T16:18:08.596 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-11-16T16:18:08.596 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-11-16T16:18:08.596 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-11-16T16:18:08.596 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-11-16T16:18:08.596 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-11-16T16:18:08.597 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-16T16:18:08.598 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-16T16:18:08.599 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-16T16:18:08.599 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-16T16:18:08.599 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-16T16:18:08.599 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-16T16:18:08.599 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-16T16:18:08.599 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-16T16:18:08.599 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-16T16:18:08.599 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-16T16:18:08.599 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-16T16:18:08.599 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-16T16:18:08.599 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-16T16:18:08.599 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-11-16T16:18:08.600 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-11-16T16:18:08.601 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-16T16:18:08.602 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-11-16T16:18:08.602 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-16T16:18:08.602 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-11-16T16:18:08.602 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-11-16T16:18:08.602 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-16T16:18:08.602 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-11-16T16:18:08.602 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-16T16:18:08.602 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-11-16T16:18:08.602 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-16T16:18:08.602 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-11-16T16:18:08.602 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-11-16T16:18:08.603 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-16T16:18:08.604 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-16T16:18:08.605 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-16T16:18:08.606 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-16T16:18:08.606 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-16T16:18:08.606 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-16T16:18:08.606 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-16T16:18:08.606 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-16T16:18:08.606 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-11-16T16:18:08.606 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-11-16T16:18:08.606 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-11-16T16:18:08.606 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-11-16T16:18:08.606 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-11-16T16:18:08.606 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-11-16T16:18:08.606 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-11-16T16:18:08.607 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-11-16T16:18:08.607 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-11-16T16:18:08.607 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-11-16T16:18:08.607 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-11-16T16:18:08.607 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-11-16T16:18:08.607 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-11-16T16:18:08.607 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-11-16T16:18:08.607 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-11-16T16:18:08.607 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-11-16T16:18:08.607 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-11-16T16:18:08.607 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-11-16T16:18:08.607 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-11-16T16:18:08.608 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-11-16T16:18:08.609 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-11-16T16:18:08.610 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-11-16T16:18:08.610 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-16T16:18:08.610 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-11-16T16:18:08.610 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-11-16T16:18:08.610 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-11-16T16:18:08.610 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-11-16T16:18:08.610 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-11-16T16:18:08.610 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-11-16T16:18:08.610 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-11-16T16:18:08.610 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-16T16:18:08.610 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-16T16:18:08.610 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-11-16T16:18:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-11-16T16:18:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-11-16T16:18:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-11-16T16:18:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-11-16T16:18:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-11-16T16:18:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-16T16:18:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-16T16:18:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-11-16T16:18:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-11-16T16:18:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-11-16T16:18:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-11-16T16:18:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-11-16T16:18:08.612 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-11-16T16:18:08.613 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-16T16:18:08.614 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-16T16:18:08.614 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-16T16:18:08.614 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-16T16:18:08.614 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-11-16T16:18:08.614 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-16T16:18:08.614 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-16T16:18:08.614 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-16T16:18:08.614 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-16T16:18:08.614 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-11-16T16:18:08.614 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-16T16:18:08.614 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-11-16T16:18:08.614 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-16T16:18:08.615 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-16T16:18:08.616 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-16T16:18:08.616 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-16T16:18:08.616 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-11-16T16:18:08.616 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-16T16:18:08.616 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-16T16:18:08.616 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-11-16T16:18:08.616 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-16T16:18:08.616 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-16T16:18:08.616 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-16T16:18:08.616 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-16T16:18:08.616 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-16T16:18:08.616 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-11-16T16:18:08.617 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-16T16:18:08.617 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-16T16:18:08.617 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-11-16T16:18:08.617 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-11-16T16:18:08.617 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-16T16:18:08.617 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-11-16T16:18:08.617 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-16T16:18:08.617 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-16T16:18:08.617 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-16T16:18:08.617 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-16T16:18:08.618 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-16T16:18:08.618 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-16T16:18:08.618 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-16T16:18:08.618 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-16T16:18:08.618 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-16T16:18:08.618 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-16T16:18:08.618 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-11-16T16:18:08.618 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-11-16T16:18:08.619 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-16T16:18:08.619 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-11-16T16:18:08.619 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-11-16T16:18:08.619 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-16T16:18:08.619 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-16T16:18:08.620 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-16T16:18:08.620 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-16T16:18:08.620 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-16T16:18:08.620 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-16T16:18:08.620 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-11-16T16:18:08.620 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-16T16:18:08.620 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-16T16:18:08.621 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-11-16T16:18:08.621 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-11-16T16:18:08.621 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-11-16T16:18:08.621 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-16T16:18:08.621 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-11-16T16:18:08.621 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-11-16T16:18:08.621 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-11-16T16:18:08.621 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-16T16:18:08.622 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-11-16T16:18:08.622 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-16T16:18:08.622 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-11-16T16:18:08.622 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-16T16:18:08.622 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-11-16T16:18:08.622 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-16T16:18:08.623 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-11-16T16:18:08.623 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-16T16:18:08.623 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-11-16T16:18:08.623 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-16T16:18:08.623 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-11-16T16:18:08.623 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-16T16:18:08.623 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-11-16T16:18:08.623 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-16T16:18:08.623 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-16T16:18:08.624 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-16T16:18:08.625 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-16T16:18:08.625 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-16T16:18:08.625 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-11-16T16:18:08.625 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-16T16:18:08.625 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-11-16T16:18:08.625 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-16T16:18:08.625 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-16T16:18:08.625 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-11-16T16:18:08.625 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-16T16:18:08.625 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-16T16:18:08.626 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-11-16T16:18:08.626 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-16T16:18:08.626 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-11-16T16:18:08.626 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-16T16:18:08.626 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-11-16T16:18:08.626 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-16T16:18:08.626 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-16T16:18:08.626 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-11-16T16:18:08.626 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-16T16:18:08.626 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-16T16:18:08.626 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-16T16:18:08.626 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-16T16:18:08.627 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-16T16:18:08.627 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-11-16T16:18:08.627 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-16T16:18:08.627 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-11-16T16:18:08.627 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-16T16:18:08.627 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-16T16:18:08.627 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-16T16:18:08.627 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-16T16:18:08.627 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-11-16T16:18:08.627 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-16T16:18:08.627 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-11-16T16:18:08.627 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-16T16:18:08.628 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-11-16T16:18:08.628 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-16T16:18:08.628 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-11-16T16:18:08.628 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-16T16:18:08.628 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-16T16:18:08.628 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-11-16T16:18:08.628 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-16T16:18:08.628 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-16T16:18:08.628 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-16T16:18:08.628 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-16T16:18:08.628 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-11-16T16:18:08.628 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-16T16:18:08.629 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-16T16:18:08.629 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-16T16:18:08.629 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-11-16T16:18:08.629 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-16T16:18:08.629 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-16T16:18:08.629 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-16T16:18:08.629 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-16T16:18:08.629 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-11-16T16:18:08.629 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-16T16:18:08.629 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-11-16T16:18:08.629 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-16T16:18:08.629 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-11-16T16:18:08.630 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-16T16:18:08.630 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-16T16:18:08.630 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-11-16T16:18:08.630 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-16T16:18:08.630 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-16T16:18:08.630 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-11-16T16:18:08.630 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-16T16:18:08.630 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-11-16T16:18:08.630 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-11-16T16:18:08.630 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-16T16:18:08.630 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-11-16T16:18:08.630 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-11-16T16:18:08.631 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-11-16T16:18:08.632 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-11-16T16:18:08.632 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-11-16T16:18:08.632 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-16T16:18:08.632 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-16T16:18:08.632 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-11-16T16:18:08.632 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-11-16T16:18:08.632 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-16T16:18:08.632 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-16T16:18:08.632 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-16T16:18:08.632 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-11-16T16:18:08.632 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-11-16T16:18:08.632 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-11-16T16:18:08.633 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-11-16T16:18:08.633 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-11-16T16:18:08.633 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-11-16T16:18:08.633 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-11-16T16:18:08.633 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-11-16T16:18:08.633 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-11-16T16:18:08.633 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-11-16T16:18:08.633 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-11-16T16:18:08.633 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-11-16T16:18:08.633 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-11-16T16:18:08.633 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-11-16T16:18:08.633 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-11-16T16:18:08.634 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-11-16T16:18:08.634 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-11-16T16:18:08.634 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-11-16T16:18:08.634 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-16T16:18:08.634 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-11-16T16:18:08.634 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-16T16:18:08.634 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-16T16:18:08.634 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-16T16:18:08.634 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-16T16:18:08.634 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-11-16T16:18:08.634 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-11-16T16:18:08.634 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-11-16T16:18:08.635 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-11-16T16:18:08.635 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-11-16T16:18:08.635 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-11-16T16:18:08.635 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-11-16T16:18:08.635 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-11-16T16:18:08.635 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-11-16T16:18:08.635 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-11-16T16:18:08.635 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-16T16:18:08.635 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-16T16:18:08.635 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-16T16:18:08.635 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-11-16T16:18:08.636 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-16T16:18:08.636 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-11-16T16:18:08.636 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-11-16T16:18:08.636 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-11-16T16:18:08.636 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-11-16T16:18:08.636 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-16T16:18:08.636 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-11-16T16:18:08.636 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-11-16T16:18:08.636 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-11-16T16:18:08.636 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-16T16:18:08.636 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-11-16T16:18:08.636 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-11-16T16:18:08.637 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-11-16T16:18:08.637 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-11-16T16:18:08.637 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-11-16T16:18:08.637 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-11-16T16:18:08.637 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-11-16T16:18:08.637 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-11-16T16:18:08.637 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-11-16T16:18:08.637 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-11-16T16:18:08.637 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-11-16T16:18:08.637 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-11-16T16:18:08.637 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-11-16T16:18:08.637 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-11-16T16:18:08.638 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-16T16:18:08.638 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-11-16T16:18:08.638 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-16T16:18:08.638 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-11-16T16:18:08.638 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-11-16T16:18:08.638 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-11-16T16:18:08.638 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-11-16T16:18:08.638 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-16T16:18:08.638 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-11-16T16:18:08.638 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-11-16T16:18:08.638 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-16T16:18:08.639 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-16T16:18:08.640 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-11-16T16:18:08.640 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-11-16T16:18:08.640 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-16T16:18:08.640 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-11-16T16:18:08.640 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-16T16:18:08.640 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-16T16:18:08.640 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-16T16:18:08.640 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-16T16:18:08.640 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-16T16:18:08.640 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-16T16:18:08.640 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-16T16:18:08.640 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-11-16T16:18:08.641 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-11-16T16:18:08.642 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-11-16T16:18:08.642 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-11-16T16:18:08.642 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-16T16:18:08.642 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-16T16:18:08.642 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-16T16:18:08.642 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-16T16:18:08.642 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-16T16:18:08.642 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-16T16:18:08.642 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-16T16:18:08.642 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-16T16:18:08.642 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-16T16:18:08.642 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-16T16:18:08.643 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-16T16:18:08.644 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-16T16:18:08.644 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-16T16:18:08.644 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-16T16:18:08.644 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-16T16:18:08.644 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-16T16:18:08.644 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-16T16:18:08.644 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-16T16:18:08.644 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-11-16T16:18:08.644 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-11-16T16:18:08.644 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-11-16T16:18:08.644 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-11-16T16:18:08.644 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-11-16T16:18:08.645 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-16T16:18:08.646 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-11-16T16:18:08.646 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-11-16T16:18:08.646 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-11-16T16:18:08.646 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-16T16:18:08.646 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-11-16T16:18:08.646 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-16T16:18:08.646 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-11-16T16:18:08.646 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-11-16T16:18:08.646 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-16T16:18:08.646 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-11-16T16:18:08.646 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-16T16:18:08.647 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-16T16:18:08.647 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-16T16:18:08.647 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-16T16:18:08.647 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-16T16:18:08.647 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-11-16T16:18:08.647 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-11-16T16:18:08.647 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-16T16:18:08.647 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-16T16:18:08.647 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-16T16:18:08.647 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-16T16:18:08.647 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-11-16T16:18:08.647 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-11-16T16:18:08.648 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-16T16:18:08.648 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-11-16T16:18:08.648 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-11-16T16:18:08.648 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-16T16:18:08.648 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-16T16:18:08.648 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-11-16T16:18:08.648 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-11-16T16:18:08.648 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-11-16T16:18:08.648 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-11-16T16:18:08.648 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-11-16T16:18:08.648 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-11-16T16:18:08.648 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-11-16T16:18:08.649 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-16T16:18:08.650 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-11-16T16:18:08.650 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-11-16T16:18:08.650 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-16T16:18:08.650 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-16T16:18:08.650 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-16T16:18:08.650 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-11-16T16:18:08.650 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-16T16:18:08.650 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-11-16T16:18:08.650 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-16T16:18:08.650 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-11-16T16:18:08.650 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-16T16:18:08.651 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-16T16:18:08.652 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-16T16:18:08.652 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-16T16:18:08.652 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-11-16T16:18:08.652 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-11-16T16:18:08.652 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-11-16T16:18:08.652 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-11-16T16:18:08.652 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-16T16:18:08.652 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-11-16T16:18:08.652 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-16T16:18:08.652 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-11-16T16:18:08.652 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-11-16T16:18:08.652 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-11-16T16:18:08.653 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-11-16T16:18:08.653 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-11-16T16:18:08.653 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-16T16:18:08.653 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-11-16T16:18:08.653 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-16T16:18:08.653 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-11-16T16:18:08.653 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-11-16T16:18:08.653 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-11-16T16:18:08.653 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-11-16T16:18:08.653 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-11-16T16:18:08.653 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-11-16T16:18:08.653 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-16T16:18:08.654 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-11-16T16:18:08.654 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-11-16T16:18:08.654 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-11-16T16:18:08.654 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-16T16:18:08.654 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-16T16:18:08.654 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-16T16:18:08.654 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-16T16:18:08.654 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-16T16:18:08.654 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-16T16:18:08.654 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-16T16:18:08.654 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-16T16:18:08.654 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-16T16:18:08.655 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-16T16:18:08.655 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-16T16:18:08.655 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-16T16:18:08.655 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-16T16:18:08.655 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-16T16:18:08.655 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-16T16:18:08.655 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-11-16T16:18:08.655 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-11-16T16:18:08.655 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-16T16:18:08.655 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-11-16T16:18:08.655 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-11-16T16:18:08.655 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-11-16T16:18:08.656 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-11-16T16:18:08.656 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-11-16T16:18:08.656 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-16T16:18:08.656 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-11-16T16:18:08.656 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-11-16T16:18:08.656 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-16T16:18:08.656 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-11-16T16:18:08.656 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-11-16T16:18:08.656 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-16T16:18:08.656 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-11-16T16:18:08.656 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-16T16:18:08.656 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-11-16T16:18:08.657 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-16T16:18:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-11-16T16:18:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-11-16T16:18:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-11-16T16:18:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-11-16T16:18:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-11-16T16:18:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-11-16T16:18:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-11-16T16:18:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-11-16T16:18:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-11-16T16:18:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-11-16T16:18:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-11-16T16:18:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-11-16T16:18:08.659 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-11-16T16:18:08.660 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-11-16T16:18:08.661 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-11-16T16:18:08.662 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-11-16T16:18:08.662 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-11-16T16:18:08.662 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-11-16T16:18:08.662 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-11-16T16:18:08.662 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-11-16T16:18:08.662 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-11-16T16:18:08.662 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-11-16T16:18:08.662 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-16T16:18:08.662 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-11-16T16:18:08.662 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-11-16T16:18:08.662 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-16T16:18:08.662 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-16T16:18:08.663 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-16T16:18:08.664 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-16T16:18:08.664 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-11-16T16:18:08.664 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-16T16:18:08.664 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-16T16:18:08.664 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-16T16:18:08.664 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-16T16:18:08.664 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-16T16:18:08.664 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-11-16T16:18:08.664 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-11-16T16:18:08.664 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-16T16:18:08.664 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-16T16:18:08.664 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-16T16:18:08.665 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-11-16T16:18:08.665 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-11-16T16:18:08.665 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-16T16:18:08.665 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-16T16:18:08.665 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-11-16T16:18:08.665 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-16T16:18:08.665 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-11-16T16:18:08.665 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-11-16T16:18:08.665 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-16T16:18:08.665 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-11-16T16:18:08.665 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-11-16T16:18:08.665 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-11-16T16:18:08.666 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-11-16T16:18:08.666 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-11-16T16:18:08.666 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-11-16T16:18:08.666 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-16T16:18:08.666 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-11-16T16:18:08.666 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-11-16T16:18:08.666 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-11-16T16:18:08.666 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-11-16T16:18:08.666 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-11-16T16:18:08.666 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-16T16:18:08.666 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-16T16:18:08.667 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-11-16T16:18:08.667 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-11-16T16:18:08.667 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-11-16T16:18:08.667 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-11-16T16:18:08.667 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-11-16T16:18:08.667 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-11-16T16:18:08.667 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-16T16:18:08.667 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-16T16:18:08.667 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-16T16:18:08.667 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-16T16:18:08.667 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-16T16:18:08.667 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-11-16T16:18:08.668 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-16T16:18:08.669 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-11-16T16:18:08.670 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-16T16:18:08.670 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-11-16T16:18:08.670 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-16T16:18:08.670 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-16T16:18:08.670 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-16T16:18:08.670 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-11-16T16:18:08.670 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-11-16T16:18:08.670 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-11-16T16:18:08.670 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-16T16:18:08.670 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-11-16T16:18:08.670 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-11-16T16:18:08.671 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-11-16T16:18:08.671 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-11-16T16:18:08.671 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-16T16:18:08.671 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-11-16T16:18:08.671 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-11-16T16:18:08.671 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-16T16:18:08.671 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-11-16T16:18:08.671 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-11-16T16:18:08.671 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-11-16T16:18:08.671 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-11-16T16:18:08.671 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-11-16T16:18:08.671 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-11-16T16:18:08.672 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-11-16T16:18:08.672 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-11-16T16:18:08.672 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-11-16T16:18:08.672 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-11-16T16:18:08.672 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-11-16T16:18:08.672 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-11-16T16:18:08.672 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-11-16T16:18:08.672 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-11-16T16:18:08.672 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-11-16T16:18:08.672 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-11-16T16:18:08.672 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-11-16T16:18:08.672 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-11-16T16:18:08.673 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-11-16T16:18:08.674 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-11-16T16:18:08.674 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-11-16T16:18:08.674 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-11-16T16:18:08.674 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-11-16T16:18:08.674 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-11-16T16:18:08.674 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-11-16T16:18:08.674 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-11-16T16:18:08.674 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-11-16T16:18:08.674 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-11-16T16:18:08.674 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-11-16T16:18:08.674 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-11-16T16:18:08.675 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-11-16T16:18:08.675 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-11-16T16:18:08.675 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-11-16T16:18:08.675 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-11-16T16:18:08.675 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-11-16T16:18:08.675 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-11-16T16:18:08.675 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-11-16T16:18:08.675 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-11-16T16:18:08.675 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-16T16:18:08.675 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-11-16T16:18:08.675 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-16T16:18:08.675 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-11-16T16:18:08.676 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-11-16T16:18:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-11-16T16:18:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-11-16T16:18:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-11-16T16:18:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-11-16T16:18:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-11-16T16:18:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-11-16T16:18:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-11-16T16:18:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-11-16T16:18:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-11-16T16:18:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-11-16T16:18:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-11-16T16:18:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-11-16T16:18:08.679 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-11-16T16:18:08.679 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-11-16T16:18:08.679 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-11-16T16:18:08.679 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-11-16T16:18:08.679 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-11-16T16:18:08.679 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-11-16T16:18:08.679 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-11-16T16:18:08.679 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-11-16T16:18:08.679 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-11-16T16:18:08.679 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-11-16T16:18:08.679 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-11-16T16:18:08.679 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-11-16T16:18:08.680 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-11-16T16:18:08.680 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-11-16T16:18:08.680 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-11-16T16:18:08.680 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-11-16T16:18:08.680 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-11-16T16:18:08.680 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-11-16T16:18:08.680 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-11-16T16:18:08.680 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-11-16T16:18:08.680 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-11-16T16:18:08.680 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-11-16T16:18:08.680 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-11-16T16:18:08.680 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-16T16:18:08.681 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-11-16T16:18:08.681 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-11-16T16:18:08.681 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-11-16T16:18:08.681 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-16T16:18:08.681 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-11-16T16:18:08.681 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-16T16:18:08.681 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-11-16T16:18:08.681 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-16T16:18:08.681 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-11-16T16:18:08.681 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-16T16:18:08.681 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-16T16:18:08.681 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-16T16:18:08.682 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-11-16T16:18:08.682 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-11-16T16:18:08.682 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-11-16T16:18:08.682 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-11-16T16:18:08.682 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-11-16T16:18:08.682 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-11-16T16:18:08.682 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-11-16T16:18:08.682 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-11-16T16:18:08.682 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-11-16T16:18:08.682 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-11-16T16:18:08.682 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-11-16T16:18:08.682 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-11-16T16:18:08.683 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-11-16T16:18:08.683 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-11-16T16:18:08.683 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-11-16T16:18:08.683 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-11-16T16:18:08.683 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-11-16T16:18:08.683 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-11-16T16:18:08.683 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-11-16T16:18:08.683 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-11-16T16:18:08.683 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-11-16T16:18:08.683 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-11-16T16:18:08.683 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-11-16T16:18:08.683 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-11-16T16:18:08.684 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-11-16T16:18:08.684 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-11-16T16:18:08.684 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-11-16T16:18:08.684 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-11-16T16:18:08.684 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-11-16T16:18:08.684 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-16T16:18:08.684 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-11-16T16:18:08.684 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-11-16T16:18:08.684 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-11-16T16:18:08.684 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-11-16T16:18:08.684 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-11-16T16:18:08.684 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-11-16T16:18:08.685 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-11-16T16:18:08.685 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-11-16T16:18:08.685 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-11-16T16:18:08.685 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-11-16T16:18:08.685 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-11-16T16:18:08.685 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-11-16T16:18:08.685 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-11-16T16:18:08.685 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-11-16T16:18:08.685 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-11-16T16:18:08.685 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-11-16T16:18:08.685 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-11-16T16:18:08.685 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-16T16:18:08.686 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-11-16T16:18:08.686 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-11-16T16:18:08.686 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-11-16T16:18:08.686 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-11-16T16:18:08.686 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-11-16T16:18:08.686 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-11-16T16:18:08.686 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-11-16T16:18:08.686 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-11-16T16:18:08.686 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-11-16T16:18:08.686 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-11-16T16:18:08.686 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-11-16T16:18:08.687 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-11-16T16:18:08.687 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-11-16T16:18:08.687 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-16T16:18:08.687 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-11-16T16:18:08.687 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-11-16T16:18:08.687 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-11-16T16:18:08.687 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-11-16T16:18:08.687 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-11-16T16:18:08.687 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-11-16T16:18:08.687 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-11-16T16:18:08.687 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-11-16T16:18:08.687 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-11-16T16:18:08.688 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-11-16T16:18:08.688 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-11-16T16:18:08.688 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-11-16T16:18:08.688 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-11-16T16:18:08.688 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-11-16T16:18:08.688 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-11-16T16:18:08.688 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-11-16T16:18:08.688 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-11-16T16:18:08.688 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-11-16T16:18:08.688 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-11-16T16:18:08.688 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-11-16T16:18:08.688 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-11-16T16:18:08.689 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-11-16T16:18:08.689 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-16T16:18:08.689 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-16T16:18:08.689 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-16T16:18:08.689 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-16T16:18:08.689 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-16T16:18:08.689 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-16T16:18:08.689 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-16T16:18:08.689 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-16T16:18:08.689 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-16T16:18:08.689 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-16T16:18:08.689 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-16T16:18:08.690 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-16T16:18:08.690 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-16T16:18:08.690 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-16T16:18:08.690 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-16T16:18:08.690 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-16T16:18:08.690 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-16T16:18:08.690 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-16T16:18:08.690 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-16T16:18:08.690 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-16T16:18:08.690 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-11-16T16:18:08.690 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-16T16:18:08.690 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-11-16T16:18:08.691 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-11-16T16:18:08.692 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-11-16T16:18:08.692 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-11-16T16:18:08.692 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-11-16T16:18:08.692 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-11-16T16:18:08.692 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-11-16T16:18:08.692 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-11-16T16:18:08.692 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-11-16T16:18:08.692 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-11-16T16:18:08.692 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-16T16:18:08.692 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-11-16T16:18:08.692 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-16T16:18:08.693 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-11-16T16:18:08.693 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-11-16T16:18:08.693 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-11-16T16:18:08.693 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-11-16T16:18:08.693 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-11-16T16:18:08.693 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-11-16T16:18:08.693 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-11-16T16:18:08.694 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-11-16T16:18:08.694 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-11-16T16:18:08.694 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-11-16T16:18:08.694 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-11-16T16:18:08.694 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-11-16T16:18:08.694 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-16T16:18:08.695 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-16T16:18:08.695 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-16T16:18:08.695 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-16T16:18:08.695 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-16T16:18:08.695 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-16T16:18:08.695 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-16T16:18:08.696 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-16T16:18:08.696 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-16T16:18:08.696 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-16T16:18:08.696 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-16T16:18:08.696 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-16T16:18:08.696 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-16T16:18:08.696 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-16T16:18:08.697 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-16T16:18:08.697 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-16T16:18:08.697 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-16T16:18:08.697 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-16T16:18:08.697 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-16T16:18:08.697 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-16T16:18:08.698 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-16T16:18:08.698 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-11-16T16:18:08.698 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-11-16T16:18:08.698 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-11-16T16:18:08.698 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-11-16T16:18:08.698 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-11-16T16:18:08.698 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-11-16T16:18:08.698 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-11-16T16:18:08.699 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-11-16T16:18:08.699 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-11-16T16:18:08.699 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-11-16T16:18:08.699 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-11-16T16:18:08.699 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-11-16T16:18:08.699 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-11-16T16:18:08.700 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-11-16T16:18:08.700 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-11-16T16:18:08.700 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-11-16T16:18:08.700 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-16T16:18:08.700 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-11-16T16:18:08.700 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-11-16T16:18:08.701 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-11-16T16:18:08.701 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-11-16T16:18:08.701 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-11-16T16:18:08.701 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-16T16:18:08.701 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-11-16T16:18:08.701 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-11-16T16:18:08.702 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-11-16T16:18:08.702 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-16T16:18:08.702 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-16T16:18:08.702 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-16T16:18:08.702 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-16T16:18:08.702 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-16T16:18:08.702 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-16T16:18:08.702 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-16T16:18:08.702 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-16T16:18:08.702 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-16T16:18:08.702 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-16T16:18:08.703 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-11-16T16:18:08.704 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-16T16:18:08.705 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-16T16:18:08.706 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-16T16:18:08.706 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-16T16:18:08.706 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-16T16:18:08.706 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-16T16:18:08.706 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-16T16:18:08.706 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-16T16:18:08.706 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-16T16:18:08.706 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-16T16:18:08.706 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-16T16:18:08.706 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-16T16:18:08.706 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-16T16:18:08.706 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-16T16:18:08.707 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-16T16:18:08.708 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-11-16T16:18:08.709 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-16T16:18:08.710 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-16T16:18:08.710 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-11-16T16:18:08.710 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-11-16T16:18:08.710 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-16T16:18:08.710 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-11-16T16:18:08.710 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-16T16:18:08.710 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-16T16:18:08.710 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-11-16T16:18:08.710 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-11-16T16:18:08.710 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-11-16T16:18:08.710 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-16T16:18:08.710 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-11-16T16:18:08.711 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-11-16T16:18:08.712 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-11-16T16:18:08.713 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-11-16T16:18:08.714 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-11-16T16:18:08.714 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-11-16T16:18:08.714 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-11-16T16:18:08.714 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-11-16T16:18:08.714 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-11-16T16:18:08.714 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-11-16T16:18:08.714 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-11-16T16:18:08.714 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-11-16T16:18:08.714 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-11-16T16:18:08.714 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-11-16T16:18:08.714 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-11-16T16:18:08.715 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-11-16T16:18:08.715 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-11-16T16:18:08.715 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-11-16T16:18:08.715 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-11-16T16:18:08.715 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-11-16T16:18:08.715 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-11-16T16:18:08.715 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-11-16T16:18:08.715 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-11-16T16:18:08.715 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-11-16T16:18:08.715 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-11-16T16:18:08.715 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-11-16T16:18:08.715 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-11-16T16:18:08.716 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-11-16T16:18:08.717 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-11-16T16:18:08.717 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-11-16T16:18:08.717 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-11-16T16:18:08.717 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-11-16T16:18:08.717 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-11-16T16:18:08.717 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-11-16T16:18:08.717 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-11-16T16:18:08.717 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-11-16T16:18:08.717 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-11-16T16:18:08.717 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-11-16T16:18:08.717 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-11-16T16:18:08.717 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-11-16T16:18:08.718 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-11-16T16:18:08.718 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-11-16T16:18:08.718 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-11-16T16:18:08.718 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-11-16T16:18:08.718 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-11-16T16:18:08.718 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-11-16T16:18:08.718 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-11-16T16:18:08.718 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-11-16T16:18:08.718 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-11-16T16:18:08.718 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-11-16T16:18:08.718 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-11-16T16:18:08.718 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-11-16T16:18:08.719 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-11-16T16:18:08.719 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-11-16T16:18:08.719 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-11-16T16:18:08.719 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-11-16T16:18:08.719 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-11-16T16:18:08.719 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-11-16T16:18:08.719 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-11-16T16:18:08.719 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-11-16T16:18:08.719 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-11-16T16:18:08.719 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-11-16T16:18:08.719 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-11-16T16:18:08.720 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-11-16T16:18:08.720 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-11-16T16:18:08.720 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-11-16T16:18:08.720 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-11-16T16:18:08.720 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-11-16T16:18:08.720 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-11-16T16:18:08.720 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-11-16T16:18:08.720 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-11-16T16:18:08.720 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-11-16T16:18:08.720 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-11-16T16:18:08.721 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-11-16T16:18:08.721 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-11-16T16:18:08.721 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-11-16T16:18:08.721 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-11-16T16:18:08.721 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-11-16T16:18:08.721 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-11-16T16:18:08.721 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-11-16T16:18:08.721 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-11-16T16:18:08.721 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-11-16T16:18:08.721 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-11-16T16:18:08.721 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-11-16T16:18:08.721 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-11-16T16:18:08.722 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-11-16T16:18:08.722 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-11-16T16:18:08.722 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-11-16T16:18:08.722 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-11-16T16:18:08.722 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-11-16T16:18:08.722 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-11-16T16:18:08.722 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-11-16T16:18:08.722 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-11-16T16:18:08.722 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-11-16T16:18:08.722 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-11-16T16:18:08.722 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-11-16T16:18:08.722 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-11-16T16:18:08.723 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-11-16T16:18:08.723 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-11-16T16:18:08.723 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-11-16T16:18:08.723 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-11-16T16:18:08.723 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-11-16T16:18:08.723 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-16T16:18:08.723 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-16T16:18:08.723 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-11-16T16:18:08.723 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-11-16T16:18:08.723 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-16T16:18:08.723 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-16T16:18:08.724 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-11-16T16:18:08.725 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-11-16T16:18:08.726 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-11-16T16:18:08.726 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-11-16T16:18:08.726 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-11-16T16:18:08.726 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-11-16T16:18:08.726 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-11-16T16:18:08.726 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-11-16T16:18:08.726 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-11-16T16:18:08.727 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-11-16T16:18:08.727 DEBUG:teuthology.orchestra.run.smithi093:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-16T16:18:08.755 INFO:teuthology.orchestra.run.smithi093.stdout:postinstall scriptlet (using /bin/sh): 2023-11-16T16:18:08.755 INFO:teuthology.orchestra.run.smithi093.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-11-16T16:18:08.755 INFO:teuthology.orchestra.run.smithi093.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-16T16:18:08.755 INFO:teuthology.orchestra.run.smithi093.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-16T16:18:08.755 INFO:teuthology.orchestra.run.smithi093.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-16T16:18:08.755 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-16T16:18:08.755 INFO:teuthology.orchestra.run.smithi093.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-16T16:18:08.755 INFO:teuthology.orchestra.run.smithi093.stdout:fi 2023-11-16T16:18:08.755 INFO:teuthology.orchestra.run.smithi093.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-16T16:18:08.756 INFO:teuthology.orchestra.run.smithi093.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-16T16:18:08.756 INFO:teuthology.orchestra.run.smithi093.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-11-16T16:18:08.756 INFO:teuthology.orchestra.run.smithi093.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-16T16:18:08.756 INFO:teuthology.orchestra.run.smithi093.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-11-16T16:18:08.756 INFO:teuthology.orchestra.run.smithi093.stdout:fi 2023-11-16T16:18:08.756 INFO:teuthology.orchestra.run.smithi093.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-16T16:18:08.756 INFO:teuthology.orchestra.run.smithi093.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-16T16:18:08.756 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-11-16T16:18:08.756 INFO:teuthology.orchestra.run.smithi093.stdout:fi 2023-11-16T16:18:08.757 DEBUG:teuthology.orchestra.run.smithi093:> sudo rpm -qi grub2-tools 2023-11-16T16:18:08.837 INFO:teuthology.orchestra.run.smithi093.stdout:Name : grub2-tools 2023-11-16T16:18:08.837 INFO:teuthology.orchestra.run.smithi093.stdout:Epoch : 1 2023-11-16T16:18:08.838 INFO:teuthology.orchestra.run.smithi093.stdout:Version : 2.02 2023-11-16T16:18:08.838 INFO:teuthology.orchestra.run.smithi093.stdout:Release : 123.el8 2023-11-16T16:18:08.838 INFO:teuthology.orchestra.run.smithi093.stdout:Architecture: x86_64 2023-11-16T16:18:08.838 INFO:teuthology.orchestra.run.smithi093.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-11-16T16:18:08.838 INFO:teuthology.orchestra.run.smithi093.stdout:Group : System Environment/Base 2023-11-16T16:18:08.838 INFO:teuthology.orchestra.run.smithi093.stdout:Size : 9265450 2023-11-16T16:18:08.838 INFO:teuthology.orchestra.run.smithi093.stdout:License : GPLv3+ 2023-11-16T16:18:08.838 INFO:teuthology.orchestra.run.smithi093.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-11-16T16:18:08.838 INFO:teuthology.orchestra.run.smithi093.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-11-16T16:18:08.838 INFO:teuthology.orchestra.run.smithi093.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-11-16T16:18:08.838 INFO:teuthology.orchestra.run.smithi093.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-11-16T16:18:08.838 INFO:teuthology.orchestra.run.smithi093.stdout:Relocations : (not relocatable) 2023-11-16T16:18:08.839 INFO:teuthology.orchestra.run.smithi093.stdout:Packager : Red Hat, Inc. 2023-11-16T16:18:08.839 INFO:teuthology.orchestra.run.smithi093.stdout:Vendor : Red Hat, Inc. 2023-11-16T16:18:08.839 INFO:teuthology.orchestra.run.smithi093.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-16T16:18:08.839 INFO:teuthology.orchestra.run.smithi093.stdout:Summary : Support tools for GRUB. 2023-11-16T16:18:08.839 INFO:teuthology.orchestra.run.smithi093.stdout:Description : 2023-11-16T16:18:08.839 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:18:08.839 INFO:teuthology.orchestra.run.smithi093.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-16T16:18:08.839 INFO:teuthology.orchestra.run.smithi093.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-16T16:18:08.839 INFO:teuthology.orchestra.run.smithi093.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-16T16:18:08.839 INFO:teuthology.orchestra.run.smithi093.stdout:hardware devices. 2023-11-16T16:18:08.839 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:18:08.839 INFO:teuthology.orchestra.run.smithi093.stdout:This subpackage provides tools for support of all platforms. 2023-11-16T16:18:08.840 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-16T16:18:08.840 INFO:teuthology.task.kernel:Updating grub on smithi093 to boot 6.6.0-rc5-g5b73517e0e2f 2023-11-16T16:18:08.841 DEBUG:teuthology.orchestra.run.smithi093:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-16T16:18:11.399 INFO:teuthology.orchestra.run.smithi093.stderr:Generating grub configuration file ... 2023-11-16T16:18:13.280 INFO:teuthology.orchestra.run.smithi093.stderr:done 2023-11-16T16:18:13.282 DEBUG:teuthology.orchestra.run.smithi093:> mktemp 2023-11-16T16:18:13.309 INFO:teuthology.orchestra.run.smithi093.stdout:/tmp/tmp.72u2UHgOIj 2023-11-16T16:18:13.309 DEBUG:teuthology.orchestra.run.smithi093:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.72u2UHgOIj 2023-11-16T16:18:13.382 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 0666 /tmp/tmp.72u2UHgOIj 2023-11-16T16:18:13.531 DEBUG:teuthology.orchestra.remote:smithi093:/tmp/tmp.72u2UHgOIj is 6KB 2023-11-16T16:18:13.579 DEBUG:teuthology.orchestra.run.smithi093:> rm -fr /tmp/tmp.72u2UHgOIj 2023-11-16T16:18:13.595 DEBUG:teuthology.orchestra.run.smithi093:> sudo /bin/ls /boot/loader/entries || true 2023-11-16T16:18:13.675 INFO:teuthology.orchestra.run.smithi093.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-11-16T16:18:13.675 INFO:teuthology.orchestra.run.smithi093.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-11-16T16:18:13.675 INFO:teuthology.orchestra.run.smithi093.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g5b73517e0e2f.conf 2023-11-16T16:18:13.677 DEBUG:teuthology.orchestra.run.smithi093:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g5b73517e0e2f 2023-11-16T16:18:13.848 DEBUG:teuthology.orchestra.run.smithi093:> sudo shutdown -r now 2023-11-16T16:18:43.851 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-16T16:18:43.852 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-16T16:18:43.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:19:02.394 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.93 2023-11-16T16:19:11.396 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-16T16:19:11.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:19:14.458 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.93 2023-11-16T16:19:26.459 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-16T16:19:26.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:19:58.326 DEBUG:teuthology.orchestra.run.smithi093:> true 2023-11-16T16:19:58.724 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-16T16:19:58.725 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g5b73517e0e2f"... 2023-11-16T16:19:58.725 DEBUG:teuthology.orchestra.run.smithi093:> uname -r 2023-11-16T16:19:58.767 INFO:teuthology.orchestra.run.smithi093.stdout:6.6.0-rc5-g5b73517e0e2f 2023-11-16T16:19:58.768 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-11-16T16:19:58.768 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-16T16:19:58.768 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-11-16T16:19:59.769 DEBUG:teuthology.parallel:result is None 2023-11-16T16:19:59.770 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-16T16:19:59.778 INFO:teuthology.task.internal:Creating test directory... 2023-11-16T16:19:59.778 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-16T16:19:59.783 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-16T16:19:59.786 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-16T16:19:59.822 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-16T16:19:59.829 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-16T16:19:59.874 INFO:teuthology.task.internal:Creating archive directory... 2023-11-16T16:19:59.875 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-16T16:19:59.879 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-16T16:19:59.881 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-16T16:19:59.935 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-16T16:19:59.941 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-16T16:19:59.941 DEBUG:teuthology.orchestra.run.smithi125:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-16T16:19:59.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:19:59.970 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-16T16:19:59.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:19:59.989 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-16T16:19:59.994 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-16T16:20:00.013 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-16T16:20:00.076 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T16:20:00.084 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T16:20:00.095 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T16:20:00.098 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T16:20:00.144 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T16:20:00.174 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T16:20:00.178 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-16T16:20:00.187 INFO:teuthology.task.internal:Configuring sudo... 2023-11-16T16:20:00.188 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-16T16:20:00.194 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-16T16:20:00.196 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-16T16:20:00.258 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-16T16:20:00.268 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-16T16:20:00.268 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-16T16:20:00.273 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-16T16:20:00.276 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-16T16:20:00.312 DEBUG:teuthology.orchestra.run.smithi093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T16:20:00.421 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T16:20:00.494 DEBUG:teuthology.orchestra.run.smithi093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T16:20:00.564 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T16:20:00.633 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:20:00.633 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-16T16:20:00.702 DEBUG:teuthology.orchestra.run.smithi125:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T16:20:00.749 DEBUG:teuthology.orchestra.run.smithi125:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T16:20:00.808 DEBUG:teuthology.orchestra.run.smithi125:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T16:20:00.893 DEBUG:teuthology.orchestra.run.smithi125:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T16:20:00.938 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-11-16T16:20:00.939 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-16T16:20:01.011 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T16:20:01.058 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T16:20:01.121 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T16:20:01.166 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T16:20:01.210 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-11-16T16:20:01.210 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-16T16:20:01.299 DEBUG:teuthology.orchestra.run.smithi093:> sudo service rsyslog restart 2023-11-16T16:20:01.304 DEBUG:teuthology.orchestra.run.smithi125:> sudo service rsyslog restart 2023-11-16T16:20:01.306 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2023-11-16T16:20:01.363 INFO:teuthology.orchestra.run.smithi125.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T16:20:01.414 INFO:teuthology.orchestra.run.smithi093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T16:20:01.415 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T16:20:01.840 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-16T16:20:01.848 INFO:teuthology.task.internal:Starting timer... 2023-11-16T16:20:01.848 INFO:teuthology.run_tasks:Running task pcp... 2023-11-16T16:20:01.903 INFO:teuthology.run_tasks:Running task selinux... 2023-11-16T16:20:01.952 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-16T16:20:01.952 DEBUG:teuthology.orchestra.run.smithi093:> sudo service auditd rotate 2023-11-16T16:20:02.064 INFO:teuthology.orchestra.run.smithi093.stdout:Rotating logs: 2023-11-16T16:20:02.067 DEBUG:teuthology.orchestra.run.smithi125:> sudo service auditd rotate 2023-11-16T16:20:02.154 INFO:teuthology.orchestra.run.smithi125.stdout:Rotating logs: 2023-11-16T16:20:02.156 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2023-11-16T16:20:02.240 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2023-11-16T16:20:02.241 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-16T16:20:02.242 DEBUG:teuthology.orchestra.run.smithi093:> /usr/sbin/getenforce 2023-11-16T16:20:02.290 INFO:teuthology.orchestra.run.smithi093.stdout:Permissive 2023-11-16T16:20:02.290 DEBUG:teuthology.orchestra.run.smithi125:> /usr/sbin/getenforce 2023-11-16T16:20:02.321 INFO:teuthology.orchestra.run.smithi125.stdout:Permissive 2023-11-16T16:20:02.321 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2023-11-16T16:20:02.344 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2023-11-16T16:20:02.344 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi093.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi125.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi159.front.sepia.ceph.com': 'permissive'} 2023-11-16T16:20:02.345 DEBUG:teuthology.orchestra.run.smithi093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T16:20:02.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:20:02.414 DEBUG:teuthology.orchestra.run.smithi125:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T16:20:02.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:20:02.443 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T16:20:02.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:20:02.472 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-16T16:20:02.472 DEBUG:teuthology.orchestra.run.smithi093:> sudo /usr/sbin/setenforce permissive 2023-11-16T16:20:02.541 DEBUG:teuthology.orchestra.run.smithi125:> sudo /usr/sbin/setenforce permissive 2023-11-16T16:20:02.568 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2023-11-16T16:20:02.596 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-16T16:20:02.606 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-16T16:20:02.606 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-16T16:20:02.622 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-16T16:20:02.623 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi093.front.sepia.ceph.com,smithi125.front.sepia.ceph.com,smithi159.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-16T16:33:53.817 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi093.front.sepia.ceph.com'), Remote(name='ubuntu@smithi125.front.sepia.ceph.com'), Remote(name='ubuntu@smithi159.front.sepia.ceph.com')] 2023-11-16T16:33:53.818 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-16T16:33:53.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:33:53.909 DEBUG:teuthology.orchestra.run.smithi093:> true 2023-11-16T16:33:54.016 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-16T16:33:54.017 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-11-16T16:33:54.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:33:54.085 DEBUG:teuthology.orchestra.run.smithi125:> true 2023-11-16T16:33:54.157 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-11-16T16:33:54.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-11-16T16:33:54.158 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:33:54.226 DEBUG:teuthology.orchestra.run.smithi159:> true 2023-11-16T16:33:54.298 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-11-16T16:33:54.298 INFO:teuthology.run_tasks:Running task clock... 2023-11-16T16:33:54.311 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-16T16:33:54.312 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-16T16:33:54.312 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T16:33:54.316 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-16T16:33:54.317 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T16:33:54.319 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-16T16:33:54.319 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T16:33:54.354 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-16T16:33:54.371 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-16T16:33:54.372 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-16T16:33:54.385 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-16T16:33:54.387 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-16T16:33:54.419 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-16T16:33:54.447 INFO:teuthology.orchestra.run.smithi125.stderr:sudo: ntpd: command not found 2023-11-16T16:33:54.454 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2023-11-16T16:33:54.461 INFO:teuthology.orchestra.run.smithi125.stdout:506 Cannot talk to daemon 2023-11-16T16:33:54.466 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2023-11-16T16:33:54.479 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-16T16:33:54.481 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-16T16:33:54.496 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-16T16:33:54.496 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-16T16:33:54.510 INFO:teuthology.orchestra.run.smithi093.stderr:sudo: ntpd: command not found 2023-11-16T16:33:54.540 INFO:teuthology.orchestra.run.smithi093.stdout:506 Cannot talk to daemon 2023-11-16T16:33:54.566 INFO:teuthology.orchestra.run.smithi159.stderr:bash: ntpq: command not found 2023-11-16T16:33:54.568 INFO:teuthology.orchestra.run.smithi125.stderr:bash: ntpq: command not found 2023-11-16T16:33:54.569 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T16:33:54.569 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2023-11-16T16:33:54.569 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:33:54.569 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:33:54.569 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:33:54.570 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:33:54.572 INFO:teuthology.orchestra.run.smithi125.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T16:33:54.572 INFO:teuthology.orchestra.run.smithi125.stdout:=============================================================================== 2023-11-16T16:33:54.572 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:33:54.572 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:33:54.573 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:33:54.573 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:33:54.574 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-16T16:33:54.607 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-16T16:33:54.693 INFO:teuthology.orchestra.run.smithi093.stderr:bash: ntpq: command not found 2023-11-16T16:33:54.700 INFO:teuthology.orchestra.run.smithi093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T16:33:54.700 INFO:teuthology.orchestra.run.smithi093.stdout:=============================================================================== 2023-11-16T16:33:54.700 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:33:54.700 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:33:54.700 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:33:54.701 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:33:54.702 INFO:teuthology.run_tasks:Running task pexec... 2023-11-16T16:33:54.716 INFO:teuthology.task.pexec:Executing custom commands... 2023-11-16T16:33:54.717 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi093.front.sepia.ceph.com 2023-11-16T16:33:54.717 DEBUG:teuthology.orchestra.run.smithi093:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-16T16:33:54.717 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi125.front.sepia.ceph.com 2023-11-16T16:33:54.717 DEBUG:teuthology.orchestra.run.smithi125:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-16T16:33:54.718 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi159.front.sepia.ceph.com 2023-11-16T16:33:54.718 DEBUG:teuthology.orchestra.run.smithi159:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-16T16:33:55.089 INFO:teuthology.orchestra.run.smithi159.stdout:Updating Subscription Management repositories. 2023-11-16T16:33:55.110 INFO:teuthology.orchestra.run.smithi125.stdout:Updating Subscription Management repositories. 2023-11-16T16:33:55.196 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-11-16T16:33:56.651 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:05:06 ago on Thu 16 Nov 2023 04:28:50 PM UTC. 2023-11-16T16:33:56.799 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:02:07 ago on Thu 16 Nov 2023 04:31:49 PM UTC. 2023-11-16T16:33:56.864 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:05:06 ago on Thu 16 Nov 2023 04:28:50 PM UTC. 2023-11-16T16:33:58.372 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-11-16T16:33:58.373 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-11-16T16:33:58.373 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2023-11-16T16:33:58.374 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-11-16T16:33:58.374 INFO:teuthology.orchestra.run.smithi159.stdout:Resetting modules: 2023-11-16T16:33:58.374 INFO:teuthology.orchestra.run.smithi159.stdout: container-tools 2023-11-16T16:33:58.374 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:33:58.374 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-11-16T16:33:58.374 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-11-16T16:33:58.374 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:33:58.586 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-11-16T16:33:58.586 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-11-16T16:33:58.587 INFO:teuthology.orchestra.run.smithi125.stdout: Package Architecture Version Repository Size 2023-11-16T16:33:58.587 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-11-16T16:33:58.587 INFO:teuthology.orchestra.run.smithi125.stdout:Resetting modules: 2023-11-16T16:33:58.587 INFO:teuthology.orchestra.run.smithi125.stdout: container-tools 2023-11-16T16:33:58.587 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:33:58.587 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-11-16T16:33:58.587 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-11-16T16:33:58.587 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:33:58.614 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-11-16T16:33:58.615 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-11-16T16:33:58.615 INFO:teuthology.orchestra.run.smithi093.stdout: Package Architecture Version Repository Size 2023-11-16T16:33:58.615 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-11-16T16:33:58.615 INFO:teuthology.orchestra.run.smithi093.stdout:Resetting modules: 2023-11-16T16:33:58.615 INFO:teuthology.orchestra.run.smithi093.stdout: container-tools 2023-11-16T16:33:58.615 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:33:58.616 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-11-16T16:33:58.616 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-11-16T16:33:58.616 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:33:59.338 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:05:08 ago on Thu 16 Nov 2023 04:28:50 PM UTC. 2023-11-16T16:33:59.538 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:05:09 ago on Thu 16 Nov 2023 04:28:50 PM UTC. 2023-11-16T16:33:59.874 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:02:10 ago on Thu 16 Nov 2023 04:31:49 PM UTC. 2023-11-16T16:34:08.103 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-11-16T16:34:08.594 INFO:teuthology.orchestra.run.smithi125.stdout:Updating Subscription Management repositories. 2023-11-16T16:34:08.709 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-11-16T16:34:08.969 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-11-16T16:34:09.205 INFO:teuthology.orchestra.run.smithi159.stdout:Updating Subscription Management repositories. 2023-11-16T16:34:09.557 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-11-16T16:34:10.067 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:05:19 ago on Thu 16 Nov 2023 04:28:50 PM UTC. 2023-11-16T16:34:10.839 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:05:20 ago on Thu 16 Nov 2023 04:28:50 PM UTC. 2023-11-16T16:34:11.295 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:02:21 ago on Thu 16 Nov 2023 04:31:49 PM UTC. 2023-11-16T16:34:11.821 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-11-16T16:34:11.914 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================================ 2023-11-16T16:34:11.914 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-11-16T16:34:11.914 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================================ 2023-11-16T16:34:11.914 INFO:teuthology.orchestra.run.smithi125.stdout:Installing group/module packages: 2023-11-16T16:34:11.914 INFO:teuthology.orchestra.run.smithi125.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-16T16:34:11.914 INFO:teuthology.orchestra.run.smithi125.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-16T16:34:11.914 INFO:teuthology.orchestra.run.smithi125.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-16T16:34:11.915 INFO:teuthology.orchestra.run.smithi125.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-16T16:34:11.915 INFO:teuthology.orchestra.run.smithi125.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-16T16:34:11.915 INFO:teuthology.orchestra.run.smithi125.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-16T16:34:11.915 INFO:teuthology.orchestra.run.smithi125.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-16T16:34:11.915 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-11-16T16:34:11.915 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-16T16:34:11.915 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-16T16:34:11.915 INFO:teuthology.orchestra.run.smithi125.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-16T16:34:11.915 INFO:teuthology.orchestra.run.smithi125.stdout:Installing module profiles: 2023-11-16T16:34:11.915 INFO:teuthology.orchestra.run.smithi125.stdout: container-tools/common 2023-11-16T16:34:11.915 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:34:11.916 INFO:teuthology.orchestra.run.smithi125.stdout:Enabling module streams: 2023-11-16T16:34:11.916 INFO:teuthology.orchestra.run.smithi125.stdout: container-tools rhel8 2023-11-16T16:34:11.916 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:34:11.916 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-11-16T16:34:11.916 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================================ 2023-11-16T16:34:11.916 INFO:teuthology.orchestra.run.smithi125.stdout:Install 10 Packages 2023-11-16T16:34:11.916 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:34:11.917 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 18 M 2023-11-16T16:34:11.917 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 65 M 2023-11-16T16:34:11.917 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-11-16T16:34:12.095 INFO:teuthology.orchestra.run.smithi125.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 229 kB/s | 41 kB 00:00 2023-11-16T16:34:12.120 INFO:teuthology.orchestra.run.smithi125.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 122 kB/s | 25 kB 00:00 2023-11-16T16:34:12.145 INFO:teuthology.orchestra.run.smithi125.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 411 kB/s | 94 kB 00:00 2023-11-16T16:34:12.229 INFO:teuthology.orchestra.run.smithi125.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 369 kB/s | 49 kB 00:00 2023-11-16T16:34:12.362 INFO:teuthology.orchestra.run.smithi125.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.1 MB/s | 524 kB 00:00 2023-11-16T16:34:12.387 INFO:teuthology.orchestra.run.smithi125.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 942 kB/s | 149 kB 00:00 2023-11-16T16:34:12.551 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-11-16T16:34:12.554 INFO:teuthology.orchestra.run.smithi125.stdout:(7/10): buildah-1.26.2-1.module+el8.6.0+16771+2 19 MB/s | 8.0 MB 00:00 2023-11-16T16:34:12.580 INFO:teuthology.orchestra.run.smithi125.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 980 kB/s | 213 kB 00:00 2023-11-16T16:34:12.644 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================================ 2023-11-16T16:34:12.644 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-11-16T16:34:12.644 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================================ 2023-11-16T16:34:12.645 INFO:teuthology.orchestra.run.smithi159.stdout:Installing group/module packages: 2023-11-16T16:34:12.645 INFO:teuthology.orchestra.run.smithi159.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-16T16:34:12.645 INFO:teuthology.orchestra.run.smithi159.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-16T16:34:12.645 INFO:teuthology.orchestra.run.smithi159.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-16T16:34:12.645 INFO:teuthology.orchestra.run.smithi159.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-16T16:34:12.645 INFO:teuthology.orchestra.run.smithi159.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-16T16:34:12.645 INFO:teuthology.orchestra.run.smithi159.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-16T16:34:12.645 INFO:teuthology.orchestra.run.smithi159.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-16T16:34:12.645 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-11-16T16:34:12.645 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-16T16:34:12.645 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-16T16:34:12.645 INFO:teuthology.orchestra.run.smithi159.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-16T16:34:12.646 INFO:teuthology.orchestra.run.smithi159.stdout:Installing module profiles: 2023-11-16T16:34:12.646 INFO:teuthology.orchestra.run.smithi159.stdout: container-tools/common 2023-11-16T16:34:12.646 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:34:12.646 INFO:teuthology.orchestra.run.smithi159.stdout:Enabling module streams: 2023-11-16T16:34:12.646 INFO:teuthology.orchestra.run.smithi159.stdout: container-tools rhel8 2023-11-16T16:34:12.646 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:34:12.646 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-11-16T16:34:12.646 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================================ 2023-11-16T16:34:12.646 INFO:teuthology.orchestra.run.smithi159.stdout:Install 10 Packages 2023-11-16T16:34:12.646 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:34:12.647 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 18 M 2023-11-16T16:34:12.647 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 65 M 2023-11-16T16:34:12.647 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-11-16T16:34:12.697 INFO:teuthology.orchestra.run.smithi125.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 15 MB/s | 2.2 MB 00:00 2023-11-16T16:34:12.788 INFO:teuthology.orchestra.run.smithi125.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 18 MB/s | 7.0 MB 00:00 2023-11-16T16:34:12.789 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:34:12.789 INFO:teuthology.orchestra.run.smithi125.stdout:Total 21 MB/s | 18 MB 00:00 2023-11-16T16:34:12.795 INFO:teuthology.orchestra.run.smithi159.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 633 kB/s | 94 kB 00:00 2023-11-16T16:34:12.821 INFO:teuthology.orchestra.run.smithi159.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 143 kB/s | 25 kB 00:00 2023-11-16T16:34:12.881 INFO:teuthology.orchestra.run.smithi159.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 174 kB/s | 41 kB 00:00 2023-11-16T16:34:12.930 INFO:teuthology.orchestra.run.smithi159.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 369 kB/s | 49 kB 00:00 2023-11-16T16:34:12.989 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-11-16T16:34:13.005 INFO:teuthology.orchestra.run.smithi159.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.8 MB/s | 524 kB 00:00 2023-11-16T16:34:13.007 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-11-16T16:34:13.008 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-11-16T16:34:13.082 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-11-16T16:34:13.105 INFO:teuthology.orchestra.run.smithi159.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 853 kB/s | 149 kB 00:00 2023-11-16T16:34:13.139 INFO:teuthology.orchestra.run.smithi159.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.6 MB/s | 213 kB 00:00 2023-11-16T16:34:13.181 INFO:teuthology.orchestra.run.smithi093.stdout:============================================================================================================ 2023-11-16T16:34:13.181 INFO:teuthology.orchestra.run.smithi093.stdout: Package Arch Version Repository Size 2023-11-16T16:34:13.181 INFO:teuthology.orchestra.run.smithi093.stdout:============================================================================================================ 2023-11-16T16:34:13.181 INFO:teuthology.orchestra.run.smithi093.stdout:Installing group/module packages: 2023-11-16T16:34:13.181 INFO:teuthology.orchestra.run.smithi093.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-16T16:34:13.181 INFO:teuthology.orchestra.run.smithi093.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-16T16:34:13.181 INFO:teuthology.orchestra.run.smithi093.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-16T16:34:13.182 INFO:teuthology.orchestra.run.smithi093.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-16T16:34:13.182 INFO:teuthology.orchestra.run.smithi093.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-16T16:34:13.182 INFO:teuthology.orchestra.run.smithi093.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-16T16:34:13.182 INFO:teuthology.orchestra.run.smithi093.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-16T16:34:13.182 INFO:teuthology.orchestra.run.smithi093.stdout:Installing dependencies: 2023-11-16T16:34:13.182 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-16T16:34:13.182 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-16T16:34:13.182 INFO:teuthology.orchestra.run.smithi093.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-16T16:34:13.182 INFO:teuthology.orchestra.run.smithi093.stdout:Installing module profiles: 2023-11-16T16:34:13.182 INFO:teuthology.orchestra.run.smithi093.stdout: container-tools/common 2023-11-16T16:34:13.182 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:34:13.182 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling module streams: 2023-11-16T16:34:13.183 INFO:teuthology.orchestra.run.smithi093.stdout: container-tools rhel8 2023-11-16T16:34:13.183 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:34:13.183 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-11-16T16:34:13.183 INFO:teuthology.orchestra.run.smithi093.stdout:============================================================================================================ 2023-11-16T16:34:13.183 INFO:teuthology.orchestra.run.smithi093.stdout:Install 10 Packages 2023-11-16T16:34:13.183 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:34:13.183 INFO:teuthology.orchestra.run.smithi093.stdout:Total download size: 18 M 2023-11-16T16:34:13.184 INFO:teuthology.orchestra.run.smithi093.stdout:Installed size: 65 M 2023-11-16T16:34:13.184 INFO:teuthology.orchestra.run.smithi093.stdout:Downloading Packages: 2023-11-16T16:34:13.189 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-11-16T16:34:13.193 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-11-16T16:34:13.306 INFO:teuthology.orchestra.run.smithi159.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 19 MB/s | 8.0 MB 00:00 2023-11-16T16:34:13.359 INFO:teuthology.orchestra.run.smithi093.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 536 kB/s | 94 kB 00:00 2023-11-16T16:34:13.373 INFO:teuthology.orchestra.run.smithi159.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 9.4 MB/s | 2.2 MB 00:00 2023-11-16T16:34:13.385 INFO:teuthology.orchestra.run.smithi093.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 124 kB/s | 25 kB 00:00 2023-11-16T16:34:13.410 INFO:teuthology.orchestra.run.smithi093.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 181 kB/s | 41 kB 00:00 2023-11-16T16:34:13.452 INFO:teuthology.orchestra.run.smithi093.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 536 kB/s | 49 kB 00:00 2023-11-16T16:34:13.490 INFO:teuthology.orchestra.run.smithi159.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 18 MB/s | 7.0 MB 00:00 2023-11-16T16:34:13.490 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:34:13.491 INFO:teuthology.orchestra.run.smithi159.stdout:Total 22 MB/s | 18 MB 00:00 2023-11-16T16:34:13.527 INFO:teuthology.orchestra.run.smithi093.stdout:(5/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.9 MB/s | 149 kB 00:00 2023-11-16T16:34:13.561 INFO:teuthology.orchestra.run.smithi093.stdout:(6/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.9 MB/s | 524 kB 00:00 2023-11-16T16:34:13.645 INFO:teuthology.orchestra.run.smithi093.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.8 MB/s | 213 kB 00:00 2023-11-16T16:34:13.691 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-11-16T16:34:13.710 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-11-16T16:34:13.710 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-11-16T16:34:13.732 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-11-16T16:34:13.795 INFO:teuthology.orchestra.run.smithi093.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 21 MB/s | 8.0 MB 00:00 2023-11-16T16:34:13.892 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-11-16T16:34:13.895 INFO:teuthology.orchestra.run.smithi093.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 8.8 MB/s | 2.2 MB 00:00 2023-11-16T16:34:13.895 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-11-16T16:34:13.968 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-16T16:34:14.062 INFO:teuthology.orchestra.run.smithi093.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 14 MB/s | 7.0 MB 00:00 2023-11-16T16:34:14.062 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:34:14.063 INFO:teuthology.orchestra.run.smithi093.stdout:Total 21 MB/s | 18 MB 00:00 2023-11-16T16:34:14.190 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T16:34:14.392 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-16T16:34:14.395 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction check 2023-11-16T16:34:14.418 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction check succeeded. 2023-11-16T16:34:14.419 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction test 2023-11-16T16:34:14.431 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-11-16T16:34:14.637 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-16T16:34:14.640 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-16T16:34:14.811 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T16:34:14.917 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction test succeeded. 2023-11-16T16:34:14.921 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction 2023-11-16T16:34:15.030 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-16T16:34:15.043 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-16T16:34:15.245 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-16T16:34:15.512 INFO:teuthology.orchestra.run.smithi093.stdout: Preparing : 1/1 2023-11-16T16:34:15.648 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-16T16:34:15.698 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-16T16:34:15.839 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-16T16:34:15.915 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T16:34:16.124 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-16T16:34:16.377 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-16T16:34:16.460 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-16T16:34:16.762 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-16T16:34:16.775 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-16T16:34:17.010 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-16T16:34:17.214 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-16T16:34:17.384 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-16T16:34:17.399 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T16:34:17.568 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-16T16:34:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-16T16:34:17.666 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T16:34:17.666 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-16T16:34:17.666 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T16:34:17.666 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-16T16:34:17.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-16T16:34:17.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-16T16:34:17.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-16T16:34:17.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-16T16:34:17.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-16T16:34:17.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-16T16:34:17.784 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-16T16:34:18.003 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T16:34:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T16:34:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-16T16:34:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T16:34:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-16T16:34:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-16T16:34:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-16T16:34:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-16T16:34:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-16T16:34:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-16T16:34:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-16T16:34:18.482 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-16T16:34:18.762 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-16T16:34:18.903 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-16T16:34:18.903 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:05:28 ago on Thu 16 Nov 2023 04:28:50 PM UTC. 2023-11-16T16:34:18.954 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-16T16:34:19.204 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T16:34:19.488 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-16T16:34:19.488 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:05:28 ago on Thu 16 Nov 2023 04:28:50 PM UTC. 2023-11-16T16:34:19.599 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T16:34:19.599 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-16T16:34:19.599 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T16:34:19.599 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-16T16:34:19.599 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-16T16:34:19.599 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-16T16:34:19.599 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-16T16:34:19.599 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-16T16:34:19.599 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-16T16:34:19.599 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-16T16:34:21.120 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-16T16:34:21.120 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:02:31 ago on Thu 16 Nov 2023 04:31:49 PM UTC. 2023-11-16T16:34:27.008 INFO:teuthology.orchestra.run.smithi159.stdout:Installed products updated. 2023-11-16T16:34:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:34:27.224 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-11-16T16:34:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:34:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T16:34:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:34:27.224 INFO:teuthology.orchestra.run.smithi159.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-16T16:34:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-16T16:34:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-16T16:34:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:34:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:34:27.225 INFO:teuthology.orchestra.run.smithi159.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T16:34:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-16T16:34:27.226 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:34:27.226 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-11-16T16:34:27.416 DEBUG:teuthology.parallel:result is None 2023-11-16T16:34:28.069 INFO:teuthology.orchestra.run.smithi093.stdout:Installed products updated. 2023-11-16T16:34:28.295 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:34:28.295 INFO:teuthology.orchestra.run.smithi093.stdout:Installed: 2023-11-16T16:34:28.295 INFO:teuthology.orchestra.run.smithi093.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:34:28.295 INFO:teuthology.orchestra.run.smithi093.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T16:34:28.296 INFO:teuthology.orchestra.run.smithi093.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:34:28.296 INFO:teuthology.orchestra.run.smithi093.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-16T16:34:28.296 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-16T16:34:28.296 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-16T16:34:28.296 INFO:teuthology.orchestra.run.smithi093.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:34:28.296 INFO:teuthology.orchestra.run.smithi093.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:34:28.296 INFO:teuthology.orchestra.run.smithi093.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T16:34:28.296 INFO:teuthology.orchestra.run.smithi093.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-16T16:34:28.296 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:34:28.296 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-11-16T16:34:28.528 DEBUG:teuthology.parallel:result is None 2023-11-16T16:34:29.084 INFO:teuthology.orchestra.run.smithi125.stdout:Installed products updated. 2023-11-16T16:34:29.286 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:34:29.286 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-11-16T16:34:29.286 INFO:teuthology.orchestra.run.smithi125.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:34:29.287 INFO:teuthology.orchestra.run.smithi125.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T16:34:29.287 INFO:teuthology.orchestra.run.smithi125.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:34:29.287 INFO:teuthology.orchestra.run.smithi125.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-16T16:34:29.288 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-16T16:34:29.288 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-16T16:34:29.288 INFO:teuthology.orchestra.run.smithi125.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:34:29.288 INFO:teuthology.orchestra.run.smithi125.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:34:29.288 INFO:teuthology.orchestra.run.smithi125.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T16:34:29.288 INFO:teuthology.orchestra.run.smithi125.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-16T16:34:29.288 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:34:29.288 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-11-16T16:34:29.432 DEBUG:teuthology.parallel:result is None 2023-11-16T16:34:29.432 INFO:teuthology.run_tasks:Running task install... 2023-11-16T16:34:29.440 DEBUG:teuthology.task.install:project ceph 2023-11-16T16:34:29.441 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207'}} 2023-11-16T16:34:29.441 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207'} 2023-11-16T16:34:29.441 INFO:teuthology.task.install:Using flavor: default 2023-11-16T16:34:29.446 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-16T16:34:29.446 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-16T16:34:29.447 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207', 'tag': None, 'wait_for_package': False} 2023-11-16T16:34:29.447 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:34:29.448 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207', 'tag': None, 'wait_for_package': False} 2023-11-16T16:34:29.449 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:34:29.450 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207', 'tag': None, 'wait_for_package': False} 2023-11-16T16:34:29.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:34:29.595 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/ 2023-11-16T16:34:29.596 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3905.g5e4a743a 2023-11-16T16:34:29.671 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/ 2023-11-16T16:34:29.671 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3905.g5e4a743a 2023-11-16T16:34:29.675 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/ 2023-11-16T16:34:29.675 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3905.g5e4a743a 2023-11-16T16:34:29.713 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-16T16:34:29.713 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:34:29.713 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-16T16:34:29.778 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-16T16:34:29.778 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-11-16T16:34:29.778 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-16T16:34:29.785 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-16T16:34:29.785 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-11-16T16:34:29.785 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-16T16:34:29.787 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-16T16:34:29.788 DEBUG:teuthology.orchestra.run.smithi093:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5e4a743ad7fd1422ebd6c4dc950a21a303279207/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-16T16:34:29.805 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-16T16:34:29.805 DEBUG:teuthology.orchestra.run.smithi125:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5e4a743ad7fd1422ebd6c4dc950a21a303279207/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-16T16:34:29.811 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-16T16:34:29.812 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5e4a743ad7fd1422ebd6c4dc950a21a303279207/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-16T16:34:29.873 DEBUG:teuthology.orchestra.run.smithi125:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-16T16:34:29.878 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-16T16:34:29.899 DEBUG:teuthology.orchestra.run.smithi093:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-16T16:34:29.950 DEBUG:teuthology.orchestra.run.smithi125:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-16T16:34:29.955 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-16T16:34:29.994 DEBUG:teuthology.orchestra.run.smithi093:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-16T16:34:30.016 INFO:teuthology.orchestra.run.smithi125.stdout:check_obsoletes = 1 2023-11-16T16:34:30.017 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum clean all 2023-11-16T16:34:30.021 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2023-11-16T16:34:30.023 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2023-11-16T16:34:30.061 INFO:teuthology.orchestra.run.smithi093.stdout:check_obsoletes = 1 2023-11-16T16:34:30.064 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum clean all 2023-11-16T16:34:30.409 INFO:teuthology.orchestra.run.smithi125.stdout:Updating Subscription Management repositories. 2023-11-16T16:34:30.434 INFO:teuthology.orchestra.run.smithi159.stdout:Updating Subscription Management repositories. 2023-11-16T16:34:30.517 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-11-16T16:34:31.237 INFO:teuthology.orchestra.run.smithi159.stdout:44 files removed 2023-11-16T16:34:31.287 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-16T16:34:31.341 INFO:teuthology.orchestra.run.smithi125.stdout:44 files removed 2023-11-16T16:34:31.390 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-16T16:34:31.593 INFO:teuthology.orchestra.run.smithi093.stdout:44 files removed 2023-11-16T16:34:31.634 INFO:teuthology.orchestra.run.smithi159.stdout:Updating Subscription Management repositories. 2023-11-16T16:34:31.649 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-16T16:34:31.736 INFO:teuthology.orchestra.run.smithi125.stdout:Updating Subscription Management repositories. 2023-11-16T16:34:32.099 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-11-16T16:34:33.127 INFO:teuthology.orchestra.run.smithi125.stdout:ceph packages for x86_64 205 kB/s | 80 kB 00:00 2023-11-16T16:34:33.164 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 169 kB/s | 80 kB 00:00 2023-11-16T16:34:33.580 INFO:teuthology.orchestra.run.smithi125.stdout:ceph noarch packages 22 kB/s | 9.4 kB 00:00 2023-11-16T16:34:33.600 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 23 kB/s | 9.4 kB 00:00 2023-11-16T16:34:33.714 INFO:teuthology.orchestra.run.smithi093.stdout:ceph packages for x86_64 216 kB/s | 80 kB 00:00 2023-11-16T16:34:33.839 INFO:teuthology.orchestra.run.smithi125.stdout:ceph source packages 6.3 kB/s | 1.5 kB 00:00 2023-11-16T16:34:33.951 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2023-11-16T16:34:34.006 INFO:teuthology.orchestra.run.smithi125.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-11-16T16:34:34.085 INFO:teuthology.orchestra.run.smithi159.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-11-16T16:34:34.089 INFO:teuthology.orchestra.run.smithi093.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2023-11-16T16:34:34.507 INFO:teuthology.orchestra.run.smithi093.stdout:ceph source packages 3.8 kB/s | 1.5 kB 00:00 2023-11-16T16:34:34.641 INFO:teuthology.orchestra.run.smithi093.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-11-16T16:34:35.334 INFO:teuthology.orchestra.run.smithi125.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-11-16T16:34:35.488 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-11-16T16:34:35.819 INFO:teuthology.orchestra.run.smithi093.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-11-16T16:34:38.874 INFO:teuthology.orchestra.run.smithi125.stdout:lab-extras 408 kB/s | 24 kB 00:00 2023-11-16T16:34:39.103 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 465 kB/s | 24 kB 00:00 2023-11-16T16:34:39.983 INFO:teuthology.orchestra.run.smithi093.stdout:lab-extras 402 kB/s | 24 kB 00:00 2023-11-16T16:34:40.018 INFO:teuthology.orchestra.run.smithi125.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-11-16T16:34:40.490 INFO:teuthology.orchestra.run.smithi159.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 MB/s | 47 MB 00:01 2023-11-16T16:34:41.335 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 MB/s | 47 MB 00:01 2023-11-16T16:34:48.185 INFO:teuthology.orchestra.run.smithi125.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 53 MB 00:01 2023-11-16T16:34:48.723 INFO:teuthology.orchestra.run.smithi159.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 53 MB 00:01 2023-11-16T16:34:49.995 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 MB/s | 53 MB 00:01 2023-11-16T16:34:59.392 INFO:teuthology.orchestra.run.smithi125.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T16:34:59.393 INFO:teuthology.orchestra.run.smithi125.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T16:34:59.407 INFO:teuthology.orchestra.run.smithi125.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-16T16:34:59.408 INFO:teuthology.orchestra.run.smithi125.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-16T16:34:59.409 INFO:teuthology.orchestra.run.smithi125.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-16T16:34:59.410 INFO:teuthology.orchestra.run.smithi125.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-16T16:34:59.412 INFO:teuthology.orchestra.run.smithi125.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-16T16:34:59.415 INFO:teuthology.orchestra.run.smithi125.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-16T16:34:59.416 INFO:teuthology.orchestra.run.smithi125.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-16T16:34:59.548 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-11-16T16:34:59.561 INFO:teuthology.orchestra.run.smithi125.stdout:=========================================================================================================================== 2023-11-16T16:34:59.561 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-11-16T16:34:59.561 INFO:teuthology.orchestra.run.smithi125.stdout:=========================================================================================================================== 2023-11-16T16:34:59.561 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-11-16T16:34:59.562 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-16T16:34:59.562 INFO:teuthology.orchestra.run.smithi125.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-16T16:34:59.562 INFO:teuthology.orchestra.run.smithi125.stdout: ceph x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 6.4 k 2023-11-16T16:34:59.562 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-base x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 5.3 M 2023-11-16T16:34:59.562 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-fuse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 847 k 2023-11-16T16:34:59.562 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 165 k 2023-11-16T16:34:59.563 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 1.5 M 2023-11-16T16:34:59.563 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 127 k 2023-11-16T16:34:59.563 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 1.7 M 2023-11-16T16:34:59.563 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 7.4 M 2023-11-16T16:34:59.563 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-rook noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 47 k 2023-11-16T16:34:59.563 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-radosgw x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 13 M 2023-11-16T16:34:59.563 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-test x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 41 M 2023-11-16T16:34:59.564 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-volume noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 265 k 2023-11-16T16:34:59.564 INFO:teuthology.orchestra.run.smithi125.stdout: cephadm noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 83 k 2023-11-16T16:34:59.564 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-mirror x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 258 k 2023-11-16T16:34:59.564 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-top noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 18 k 2023-11-16T16:34:59.564 INFO:teuthology.orchestra.run.smithi125.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-16T16:34:59.564 INFO:teuthology.orchestra.run.smithi125.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-16T16:34:59.564 INFO:teuthology.orchestra.run.smithi125.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-16T16:34:59.564 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs-devel x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 31 k 2023-11-16T16:34:59.565 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 771 k 2023-11-16T16:34:59.565 INFO:teuthology.orchestra.run.smithi125.stdout: librados-devel x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 139 k 2023-11-16T16:34:59.565 INFO:teuthology.orchestra.run.smithi125.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-16T16:34:59.565 INFO:teuthology.orchestra.run.smithi125.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-16T16:34:59.565 INFO:teuthology.orchestra.run.smithi125.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-16T16:34:59.565 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-16T16:34:59.565 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-16T16:34:59.566 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-16T16:34:59.566 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cephfs x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 197 k 2023-11-16T16:34:59.566 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rados x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 371 k 2023-11-16T16:34:59.566 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rbd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 365 k 2023-11-16T16:34:59.566 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rgw x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 107 k 2023-11-16T16:34:59.566 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-fuse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 94 k 2023-11-16T16:34:59.566 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-mirror x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.4 M 2023-11-16T16:34:59.567 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-nbd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 180 k 2023-11-16T16:34:59.567 INFO:teuthology.orchestra.run.smithi125.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-16T16:34:59.567 INFO:teuthology.orchestra.run.smithi125.stdout:Upgrading: 2023-11-16T16:34:59.567 INFO:teuthology.orchestra.run.smithi125.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-16T16:34:59.567 INFO:teuthology.orchestra.run.smithi125.stdout: librados2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.7 M 2023-11-16T16:34:59.567 INFO:teuthology.orchestra.run.smithi125.stdout: librbd1 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.6 M 2023-11-16T16:34:59.567 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-11-16T16:34:59.567 INFO:teuthology.orchestra.run.smithi125.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-16T16:34:59.567 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-common x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 22 M 2023-11-16T16:34:59.568 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 23 k 2023-11-16T16:34:59.568 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mds x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 2.3 M 2023-11-16T16:34:59.568 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 240 k 2023-11-16T16:34:59.568 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mon x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 4.6 M 2023-11-16T16:34:59.568 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-osd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 18 M 2023-11-16T16:34:59.568 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 15 k 2023-11-16T16:34:59.568 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-selinux x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 24 k 2023-11-16T16:34:59.569 INFO:teuthology.orchestra.run.smithi125.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-16T16:34:59.569 INFO:teuthology.orchestra.run.smithi125.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-16T16:34:59.569 INFO:teuthology.orchestra.run.smithi125.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-16T16:34:59.569 INFO:teuthology.orchestra.run.smithi125.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-16T16:34:59.569 INFO:teuthology.orchestra.run.smithi125.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-16T16:34:59.569 INFO:teuthology.orchestra.run.smithi125.stdout: libcephsqlite x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 137 k 2023-11-16T16:34:59.569 INFO:teuthology.orchestra.run.smithi125.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-16T16:34:59.569 INFO:teuthology.orchestra.run.smithi125.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-16T16:34:59.569 INFO:teuthology.orchestra.run.smithi125.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-16T16:34:59.570 INFO:teuthology.orchestra.run.smithi125.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-16T16:34:59.570 INFO:teuthology.orchestra.run.smithi125.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-16T16:34:59.570 INFO:teuthology.orchestra.run.smithi125.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-16T16:34:59.570 INFO:teuthology.orchestra.run.smithi125.stdout: libradosstriper1 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 473 k 2023-11-16T16:34:59.570 INFO:teuthology.orchestra.run.smithi125.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-16T16:34:59.570 INFO:teuthology.orchestra.run.smithi125.stdout: librgw2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 6.5 M 2023-11-16T16:34:59.570 INFO:teuthology.orchestra.run.smithi125.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-16T16:34:59.571 INFO:teuthology.orchestra.run.smithi125.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-16T16:34:59.571 INFO:teuthology.orchestra.run.smithi125.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-16T16:34:59.571 INFO:teuthology.orchestra.run.smithi125.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-16T16:34:59.571 INFO:teuthology.orchestra.run.smithi125.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-16T16:34:59.571 INFO:teuthology.orchestra.run.smithi125.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-16T16:34:59.571 INFO:teuthology.orchestra.run.smithi125.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-16T16:34:59.571 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-16T16:34:59.571 INFO:teuthology.orchestra.run.smithi125.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-16T16:34:59.571 INFO:teuthology.orchestra.run.smithi125.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-16T16:34:59.572 INFO:teuthology.orchestra.run.smithi125.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-16T16:34:59.572 INFO:teuthology.orchestra.run.smithi125.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-16T16:34:59.572 INFO:teuthology.orchestra.run.smithi125.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-16T16:34:59.572 INFO:teuthology.orchestra.run.smithi125.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-16T16:34:59.572 INFO:teuthology.orchestra.run.smithi125.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-16T16:34:59.572 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-16T16:34:59.572 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-argparse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 45 k 2023-11-16T16:34:59.572 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-common x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 119 k 2023-11-16T16:34:59.573 INFO:teuthology.orchestra.run.smithi125.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-16T16:34:59.573 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-16T16:34:59.573 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-16T16:34:59.573 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-16T16:34:59.573 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-16T16:34:59.573 INFO:teuthology.orchestra.run.smithi125.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-16T16:34:59.573 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-16T16:34:59.573 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-16T16:34:59.574 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-16T16:34:59.574 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-16T16:34:59.574 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-16T16:34:59.574 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T16:34:59.574 INFO:teuthology.orchestra.run.smithi125.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-16T16:34:59.574 INFO:teuthology.orchestra.run.smithi125.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-16T16:34:59.574 INFO:teuthology.orchestra.run.smithi125.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-16T16:34:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-16T16:34:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-16T16:34:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-16T16:34:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-16T16:34:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-16T16:34:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-16T16:34:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-16T16:34:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-16T16:34:59.575 INFO:teuthology.orchestra.run.smithi125.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-16T16:34:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-16T16:34:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-16T16:34:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-16T16:34:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-16T16:34:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-16T16:34:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-16T16:34:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T16:34:59.576 INFO:teuthology.orchestra.run.smithi125.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-16T16:34:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-16T16:34:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-16T16:34:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-16T16:34:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-16T16:34:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-16T16:34:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-16T16:34:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-16T16:34:59.577 INFO:teuthology.orchestra.run.smithi125.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-16T16:34:59.578 INFO:teuthology.orchestra.run.smithi125.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-16T16:34:59.578 INFO:teuthology.orchestra.run.smithi125.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-16T16:34:59.578 INFO:teuthology.orchestra.run.smithi125.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-16T16:34:59.578 INFO:teuthology.orchestra.run.smithi125.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-16T16:34:59.578 INFO:teuthology.orchestra.run.smithi125.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-16T16:34:59.578 INFO:teuthology.orchestra.run.smithi125.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-16T16:34:59.578 INFO:teuthology.orchestra.run.smithi125.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-16T16:34:59.579 INFO:teuthology.orchestra.run.smithi125.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-16T16:34:59.579 INFO:teuthology.orchestra.run.smithi125.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-16T16:34:59.579 INFO:teuthology.orchestra.run.smithi125.stdout:Enabling module streams: 2023-11-16T16:34:59.579 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql 10 2023-11-16T16:34:59.579 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:34:59.579 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-11-16T16:34:59.579 INFO:teuthology.orchestra.run.smithi125.stdout:=========================================================================================================================== 2023-11-16T16:34:59.579 INFO:teuthology.orchestra.run.smithi125.stdout:Install 128 Packages 2023-11-16T16:34:59.579 INFO:teuthology.orchestra.run.smithi125.stdout:Upgrade 3 Packages 2023-11-16T16:34:59.580 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:34:59.580 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 197 M 2023-11-16T16:34:59.580 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-11-16T16:34:59.878 INFO:teuthology.orchestra.run.smithi125.stdout:(1/131): ceph-18.0.0-3905.g5e4a743a.el8.x86_64. 54 kB/s | 6.4 kB 00:00 2023-11-16T16:34:59.930 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T16:34:59.931 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T16:34:59.945 INFO:teuthology.orchestra.run.smithi159.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-16T16:34:59.946 INFO:teuthology.orchestra.run.smithi159.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-16T16:34:59.947 INFO:teuthology.orchestra.run.smithi159.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-16T16:34:59.948 INFO:teuthology.orchestra.run.smithi159.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-16T16:34:59.950 INFO:teuthology.orchestra.run.smithi159.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-16T16:34:59.953 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-16T16:34:59.954 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-16T16:35:00.085 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-11-16T16:35:00.097 INFO:teuthology.orchestra.run.smithi159.stdout:=========================================================================================================================== 2023-11-16T16:35:00.097 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-11-16T16:35:00.097 INFO:teuthology.orchestra.run.smithi159.stdout:=========================================================================================================================== 2023-11-16T16:35:00.097 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-11-16T16:35:00.097 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-16T16:35:00.097 INFO:teuthology.orchestra.run.smithi159.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-16T16:35:00.097 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 6.4 k 2023-11-16T16:35:00.097 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 5.3 M 2023-11-16T16:35:00.097 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 847 k 2023-11-16T16:35:00.098 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 165 k 2023-11-16T16:35:00.098 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 1.5 M 2023-11-16T16:35:00.098 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 127 k 2023-11-16T16:35:00.098 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 1.7 M 2023-11-16T16:35:00.098 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 7.4 M 2023-11-16T16:35:00.098 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 47 k 2023-11-16T16:35:00.098 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 13 M 2023-11-16T16:35:00.098 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 41 M 2023-11-16T16:35:00.098 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 265 k 2023-11-16T16:35:00.098 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 83 k 2023-11-16T16:35:00.099 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 258 k 2023-11-16T16:35:00.099 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 18 k 2023-11-16T16:35:00.099 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-16T16:35:00.099 INFO:teuthology.orchestra.run.smithi159.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-16T16:35:00.099 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-16T16:35:00.099 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 31 k 2023-11-16T16:35:00.099 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 771 k 2023-11-16T16:35:00.099 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 139 k 2023-11-16T16:35:00.099 INFO:teuthology.orchestra.run.smithi159.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-16T16:35:00.099 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-16T16:35:00.099 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-16T16:35:00.100 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-16T16:35:00.100 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-16T16:35:00.100 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-16T16:35:00.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 197 k 2023-11-16T16:35:00.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 371 k 2023-11-16T16:35:00.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 365 k 2023-11-16T16:35:00.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 107 k 2023-11-16T16:35:00.100 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 94 k 2023-11-16T16:35:00.100 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.4 M 2023-11-16T16:35:00.100 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 180 k 2023-11-16T16:35:00.100 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-16T16:35:00.100 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.7 M 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.6 M 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 22 M 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 23 k 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 2.3 M 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 240 k 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 4.6 M 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 18 M 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 15 k 2023-11-16T16:35:00.101 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 24 k 2023-11-16T16:35:00.102 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-16T16:35:00.102 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-16T16:35:00.102 INFO:teuthology.orchestra.run.smithi159.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-16T16:35:00.102 INFO:teuthology.orchestra.run.smithi159.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-16T16:35:00.102 INFO:teuthology.orchestra.run.smithi159.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-16T16:35:00.102 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 137 k 2023-11-16T16:35:00.102 INFO:teuthology.orchestra.run.smithi159.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-16T16:35:00.102 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-16T16:35:00.102 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-16T16:35:00.102 INFO:teuthology.orchestra.run.smithi159.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-16T16:35:00.102 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-16T16:35:00.102 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-16T16:35:00.103 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 473 k 2023-11-16T16:35:00.103 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-16T16:35:00.103 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 6.5 M 2023-11-16T16:35:00.103 INFO:teuthology.orchestra.run.smithi159.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-16T16:35:00.103 INFO:teuthology.orchestra.run.smithi159.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-16T16:35:00.103 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-16T16:35:00.103 INFO:teuthology.orchestra.run.smithi159.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-16T16:35:00.103 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-16T16:35:00.103 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-16T16:35:00.103 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-16T16:35:00.103 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-16T16:35:00.103 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 45 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 119 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-16T16:35:00.104 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-16T16:35:00.105 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-16T16:35:00.106 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-16T16:35:00.107 INFO:teuthology.orchestra.run.smithi159.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-16T16:35:00.108 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-16T16:35:00.108 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-16T16:35:00.108 INFO:teuthology.orchestra.run.smithi159.stdout:Enabling module streams: 2023-11-16T16:35:00.108 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql 10 2023-11-16T16:35:00.108 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:35:00.108 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-11-16T16:35:00.108 INFO:teuthology.orchestra.run.smithi159.stdout:=========================================================================================================================== 2023-11-16T16:35:00.108 INFO:teuthology.orchestra.run.smithi159.stdout:Install 128 Packages 2023-11-16T16:35:00.108 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 3 Packages 2023-11-16T16:35:00.108 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:35:00.109 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 197 M 2023-11-16T16:35:00.109 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-11-16T16:35:00.178 INFO:teuthology.orchestra.run.smithi125.stdout:(2/131): ceph-fuse-18.0.0-3905.g5e4a743a.el8.x8 2.8 MB/s | 847 kB 00:00 2023-11-16T16:35:00.229 INFO:teuthology.orchestra.run.smithi125.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 3.2 MB/s | 165 kB 00:00 2023-11-16T16:35:00.412 INFO:teuthology.orchestra.run.smithi125.stdout:(4/131): ceph-mds-18.0.0-3905.g5e4a743a.el8.x86 12 MB/s | 2.3 MB 00:00 2023-11-16T16:35:00.415 INFO:teuthology.orchestra.run.smithi159.stdout:(1/131): ceph-18.0.0-3905.g5e4a743a.el8.x86_64. 45 kB/s | 6.4 kB 00:00 2023-11-16T16:35:00.529 INFO:teuthology.orchestra.run.smithi125.stdout:(5/131): ceph-mgr-18.0.0-3905.g5e4a743a.el8.x86 13 MB/s | 1.5 MB 00:00 2023-11-16T16:35:00.862 INFO:teuthology.orchestra.run.smithi125.stdout:(6/131): ceph-mon-18.0.0-3905.g5e4a743a.el8.x86 14 MB/s | 4.6 MB 00:00 2023-11-16T16:35:00.924 INFO:teuthology.orchestra.run.smithi159.stdout:(2/131): ceph-fuse-18.0.0-3905.g5e4a743a.el8.x8 1.6 MB/s | 847 kB 00:00 2023-11-16T16:35:01.058 INFO:teuthology.orchestra.run.smithi159.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 1.2 MB/s | 165 kB 00:00 2023-11-16T16:35:01.506 INFO:teuthology.orchestra.run.smithi093.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T16:35:01.507 INFO:teuthology.orchestra.run.smithi093.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T16:35:01.522 INFO:teuthology.orchestra.run.smithi093.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-16T16:35:01.523 INFO:teuthology.orchestra.run.smithi093.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-16T16:35:01.525 INFO:teuthology.orchestra.run.smithi093.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-16T16:35:01.526 INFO:teuthology.orchestra.run.smithi093.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-16T16:35:01.528 INFO:teuthology.orchestra.run.smithi093.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-16T16:35:01.532 INFO:teuthology.orchestra.run.smithi093.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-16T16:35:01.533 INFO:teuthology.orchestra.run.smithi093.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-16T16:35:01.627 INFO:teuthology.orchestra.run.smithi125.stdout:(7/131): ceph-base-18.0.0-3905.g5e4a743a.el8.x8 2.9 MB/s | 5.3 MB 00:01 2023-11-16T16:35:01.672 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-11-16T16:35:01.685 INFO:teuthology.orchestra.run.smithi093.stdout:=========================================================================================================================== 2023-11-16T16:35:01.685 INFO:teuthology.orchestra.run.smithi093.stdout: Package Arch Version Repository Size 2023-11-16T16:35:01.686 INFO:teuthology.orchestra.run.smithi093.stdout:=========================================================================================================================== 2023-11-16T16:35:01.686 INFO:teuthology.orchestra.run.smithi093.stdout:Installing: 2023-11-16T16:35:01.686 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-16T16:35:01.686 INFO:teuthology.orchestra.run.smithi093.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-16T16:35:01.686 INFO:teuthology.orchestra.run.smithi093.stdout: ceph x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 6.4 k 2023-11-16T16:35:01.686 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-base x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 5.3 M 2023-11-16T16:35:01.686 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-fuse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 847 k 2023-11-16T16:35:01.686 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 165 k 2023-11-16T16:35:01.687 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 1.5 M 2023-11-16T16:35:01.687 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 127 k 2023-11-16T16:35:01.687 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 1.7 M 2023-11-16T16:35:01.687 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 7.4 M 2023-11-16T16:35:01.687 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-rook noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 47 k 2023-11-16T16:35:01.687 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-radosgw x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 13 M 2023-11-16T16:35:01.687 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-test x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 41 M 2023-11-16T16:35:01.687 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-volume noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 265 k 2023-11-16T16:35:01.688 INFO:teuthology.orchestra.run.smithi093.stdout: cephadm noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 83 k 2023-11-16T16:35:01.688 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-mirror x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 258 k 2023-11-16T16:35:01.688 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-top noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 18 k 2023-11-16T16:35:01.688 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-16T16:35:01.688 INFO:teuthology.orchestra.run.smithi093.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-16T16:35:01.688 INFO:teuthology.orchestra.run.smithi093.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-16T16:35:01.688 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs-devel x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 31 k 2023-11-16T16:35:01.688 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 771 k 2023-11-16T16:35:01.689 INFO:teuthology.orchestra.run.smithi093.stdout: librados-devel x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 139 k 2023-11-16T16:35:01.689 INFO:teuthology.orchestra.run.smithi093.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-16T16:35:01.689 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-16T16:35:01.689 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-16T16:35:01.689 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-16T16:35:01.689 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-16T16:35:01.689 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-16T16:35:01.689 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cephfs x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 197 k 2023-11-16T16:35:01.690 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rados x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 371 k 2023-11-16T16:35:01.690 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rbd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 365 k 2023-11-16T16:35:01.690 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rgw x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 107 k 2023-11-16T16:35:01.690 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-fuse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 94 k 2023-11-16T16:35:01.690 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-mirror x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.4 M 2023-11-16T16:35:01.690 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-nbd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 180 k 2023-11-16T16:35:01.691 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-16T16:35:01.691 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrading: 2023-11-16T16:35:01.691 INFO:teuthology.orchestra.run.smithi093.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-16T16:35:01.691 INFO:teuthology.orchestra.run.smithi093.stdout: librados2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.7 M 2023-11-16T16:35:01.691 INFO:teuthology.orchestra.run.smithi093.stdout: librbd1 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.6 M 2023-11-16T16:35:01.691 INFO:teuthology.orchestra.run.smithi093.stdout:Installing dependencies: 2023-11-16T16:35:01.691 INFO:teuthology.orchestra.run.smithi093.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-16T16:35:01.691 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-common x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 22 M 2023-11-16T16:35:01.692 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 23 k 2023-11-16T16:35:01.692 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mds x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 2.3 M 2023-11-16T16:35:01.692 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 240 k 2023-11-16T16:35:01.692 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mon x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 4.6 M 2023-11-16T16:35:01.692 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-osd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 18 M 2023-11-16T16:35:01.692 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 15 k 2023-11-16T16:35:01.692 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-selinux x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 24 k 2023-11-16T16:35:01.692 INFO:teuthology.orchestra.run.smithi093.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-16T16:35:01.693 INFO:teuthology.orchestra.run.smithi093.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-16T16:35:01.693 INFO:teuthology.orchestra.run.smithi093.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-16T16:35:01.693 INFO:teuthology.orchestra.run.smithi093.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-16T16:35:01.693 INFO:teuthology.orchestra.run.smithi093.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-16T16:35:01.693 INFO:teuthology.orchestra.run.smithi093.stdout: libcephsqlite x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 137 k 2023-11-16T16:35:01.693 INFO:teuthology.orchestra.run.smithi093.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-16T16:35:01.693 INFO:teuthology.orchestra.run.smithi093.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-16T16:35:01.694 INFO:teuthology.orchestra.run.smithi093.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-16T16:35:01.694 INFO:teuthology.orchestra.run.smithi093.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-16T16:35:01.694 INFO:teuthology.orchestra.run.smithi093.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-16T16:35:01.694 INFO:teuthology.orchestra.run.smithi093.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-16T16:35:01.694 INFO:teuthology.orchestra.run.smithi093.stdout: libradosstriper1 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 473 k 2023-11-16T16:35:01.694 INFO:teuthology.orchestra.run.smithi093.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-16T16:35:01.694 INFO:teuthology.orchestra.run.smithi093.stdout: librgw2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 6.5 M 2023-11-16T16:35:01.695 INFO:teuthology.orchestra.run.smithi093.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-16T16:35:01.695 INFO:teuthology.orchestra.run.smithi093.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-16T16:35:01.695 INFO:teuthology.orchestra.run.smithi093.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-16T16:35:01.695 INFO:teuthology.orchestra.run.smithi093.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-16T16:35:01.695 INFO:teuthology.orchestra.run.smithi093.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-16T16:35:01.695 INFO:teuthology.orchestra.run.smithi093.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-16T16:35:01.695 INFO:teuthology.orchestra.run.smithi093.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-16T16:35:01.695 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-16T16:35:01.696 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-16T16:35:01.696 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-16T16:35:01.696 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-16T16:35:01.696 INFO:teuthology.orchestra.run.smithi093.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-16T16:35:01.696 INFO:teuthology.orchestra.run.smithi093.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-16T16:35:01.696 INFO:teuthology.orchestra.run.smithi093.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-16T16:35:01.696 INFO:teuthology.orchestra.run.smithi093.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-16T16:35:01.697 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-16T16:35:01.697 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-argparse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 45 k 2023-11-16T16:35:01.697 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-common x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 119 k 2023-11-16T16:35:01.697 INFO:teuthology.orchestra.run.smithi093.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-16T16:35:01.697 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-16T16:35:01.697 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-16T16:35:01.697 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-16T16:35:01.698 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-16T16:35:01.698 INFO:teuthology.orchestra.run.smithi093.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-16T16:35:01.698 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-16T16:35:01.698 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-16T16:35:01.698 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-16T16:35:01.698 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-16T16:35:01.698 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-16T16:35:01.698 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T16:35:01.699 INFO:teuthology.orchestra.run.smithi093.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-16T16:35:01.699 INFO:teuthology.orchestra.run.smithi093.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-16T16:35:01.699 INFO:teuthology.orchestra.run.smithi093.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-16T16:35:01.699 INFO:teuthology.orchestra.run.smithi093.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-16T16:35:01.699 INFO:teuthology.orchestra.run.smithi093.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-16T16:35:01.699 INFO:teuthology.orchestra.run.smithi093.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-16T16:35:01.699 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-16T16:35:01.699 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-16T16:35:01.700 INFO:teuthology.orchestra.run.smithi093.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-16T16:35:01.700 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-16T16:35:01.700 INFO:teuthology.orchestra.run.smithi093.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-16T16:35:01.700 INFO:teuthology.orchestra.run.smithi093.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-16T16:35:01.700 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-16T16:35:01.700 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-16T16:35:01.700 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-16T16:35:01.701 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-16T16:35:01.701 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-16T16:35:01.701 INFO:teuthology.orchestra.run.smithi093.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-16T16:35:01.701 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T16:35:01.701 INFO:teuthology.orchestra.run.smithi093.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-16T16:35:01.701 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-16T16:35:01.701 INFO:teuthology.orchestra.run.smithi093.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-16T16:35:01.701 INFO:teuthology.orchestra.run.smithi093.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-16T16:35:01.701 INFO:teuthology.orchestra.run.smithi093.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-16T16:35:01.702 INFO:teuthology.orchestra.run.smithi093.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-16T16:35:01.702 INFO:teuthology.orchestra.run.smithi093.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-16T16:35:01.702 INFO:teuthology.orchestra.run.smithi093.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-16T16:35:01.702 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-16T16:35:01.702 INFO:teuthology.orchestra.run.smithi093.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-16T16:35:01.702 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-16T16:35:01.702 INFO:teuthology.orchestra.run.smithi093.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-16T16:35:01.702 INFO:teuthology.orchestra.run.smithi093.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-16T16:35:01.703 INFO:teuthology.orchestra.run.smithi093.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-16T16:35:01.703 INFO:teuthology.orchestra.run.smithi093.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-16T16:35:01.703 INFO:teuthology.orchestra.run.smithi093.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-16T16:35:01.703 INFO:teuthology.orchestra.run.smithi093.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-16T16:35:01.703 INFO:teuthology.orchestra.run.smithi093.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-16T16:35:01.703 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling module streams: 2023-11-16T16:35:01.703 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql 10 2023-11-16T16:35:01.703 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:35:01.704 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-11-16T16:35:01.704 INFO:teuthology.orchestra.run.smithi093.stdout:=========================================================================================================================== 2023-11-16T16:35:01.704 INFO:teuthology.orchestra.run.smithi093.stdout:Install 128 Packages 2023-11-16T16:35:01.704 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrade 3 Packages 2023-11-16T16:35:01.704 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:35:01.705 INFO:teuthology.orchestra.run.smithi093.stdout:Total download size: 197 M 2023-11-16T16:35:01.705 INFO:teuthology.orchestra.run.smithi093.stdout:Downloading Packages: 2023-11-16T16:35:02.072 INFO:teuthology.orchestra.run.smithi093.stdout:(1/131): ceph-18.0.0-3905.g5e4a743a.el8.x86_64. 37 kB/s | 6.4 kB 00:00 2023-11-16T16:35:02.092 INFO:teuthology.orchestra.run.smithi159.stdout:(4/131): ceph-mds-18.0.0-3905.g5e4a743a.el8.x86 2.2 MB/s | 2.3 MB 00:01 2023-11-16T16:35:02.122 INFO:teuthology.orchestra.run.smithi125.stdout:(8/131): ceph-osd-18.0.0-3905.g5e4a743a.el8.x86 14 MB/s | 18 MB 00:01 2023-11-16T16:35:02.180 INFO:teuthology.orchestra.run.smithi125.stdout:(9/131): ceph-selinux-18.0.0-3905.g5e4a743a.el8 418 kB/s | 24 kB 00:00 2023-11-16T16:35:02.626 INFO:teuthology.orchestra.run.smithi159.stdout:(5/131): ceph-mgr-18.0.0-3905.g5e4a743a.el8.x86 2.9 MB/s | 1.5 MB 00:00 2023-11-16T16:35:02.698 INFO:teuthology.orchestra.run.smithi093.stdout:(2/131): ceph-fuse-18.0.0-3905.g5e4a743a.el8.x8 1.3 MB/s | 847 kB 00:00 2023-11-16T16:35:02.873 INFO:teuthology.orchestra.run.smithi093.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 946 kB/s | 165 kB 00:00 2023-11-16T16:35:04.134 INFO:teuthology.orchestra.run.smithi159.stdout:(6/131): ceph-mon-18.0.0-3905.g5e4a743a.el8.x86 3.1 MB/s | 4.6 MB 00:01 2023-11-16T16:35:04.253 INFO:teuthology.orchestra.run.smithi093.stdout:(4/131): ceph-mds-18.0.0-3905.g5e4a743a.el8.x86 1.7 MB/s | 2.3 MB 00:01 2023-11-16T16:35:04.909 INFO:teuthology.orchestra.run.smithi159.stdout:(7/131): ceph-common-18.0.0-3905.g5e4a743a.el8. 4.7 MB/s | 22 MB 00:04 2023-11-16T16:35:05.123 INFO:teuthology.orchestra.run.smithi093.stdout:(5/131): ceph-mgr-18.0.0-3905.g5e4a743a.el8.x86 1.8 MB/s | 1.5 MB 00:00 2023-11-16T16:35:07.263 INFO:teuthology.orchestra.run.smithi125.stdout:(10/131): ceph-radosgw-18.0.0-3905.g5e4a743a.el 2.3 MB/s | 13 MB 00:05 2023-11-16T16:35:07.655 INFO:teuthology.orchestra.run.smithi125.stdout:(11/131): ceph-test-18.0.0-3905.g5e4a743a.el8.x 7.4 MB/s | 41 MB 00:05 2023-11-16T16:35:07.680 INFO:teuthology.orchestra.run.smithi125.stdout:(12/131): cephfs-mirror-18.0.0-3905.g5e4a743a.e 620 kB/s | 258 kB 00:00 2023-11-16T16:35:07.739 INFO:teuthology.orchestra.run.smithi125.stdout:(13/131): libcephfs-devel-18.0.0-3905.g5e4a743a 373 kB/s | 31 kB 00:00 2023-11-16T16:35:07.839 INFO:teuthology.orchestra.run.smithi125.stdout:(14/131): libcephsqlite-18.0.0-3905.g5e4a743a.e 1.3 MB/s | 137 kB 00:00 2023-11-16T16:35:07.859 INFO:teuthology.orchestra.run.smithi159.stdout:(8/131): ceph-base-18.0.0-3905.g5e4a743a.el8.x8 718 kB/s | 5.3 MB 00:07 2023-11-16T16:35:07.948 INFO:teuthology.orchestra.run.smithi093.stdout:(6/131): ceph-mon-18.0.0-3905.g5e4a743a.el8.x86 1.6 MB/s | 4.6 MB 00:02 2023-11-16T16:35:07.956 INFO:teuthology.orchestra.run.smithi125.stdout:(15/131): librados-devel-18.0.0-3905.g5e4a743a. 1.2 MB/s | 139 kB 00:00 2023-11-16T16:35:07.967 INFO:teuthology.orchestra.run.smithi159.stdout:(9/131): ceph-selinux-18.0.0-3905.g5e4a743a.el8 225 kB/s | 24 kB 00:00 2023-11-16T16:35:08.081 INFO:teuthology.orchestra.run.smithi125.stdout:(16/131): libcephfs2-18.0.0-3905.g5e4a743a.el8. 1.9 MB/s | 771 kB 00:00 2023-11-16T16:35:08.132 INFO:teuthology.orchestra.run.smithi125.stdout:(17/131): libradosstriper1-18.0.0-3905.g5e4a743 2.6 MB/s | 473 kB 00:00 2023-11-16T16:35:08.257 INFO:teuthology.orchestra.run.smithi125.stdout:(18/131): python3-ceph-argparse-18.0.0-3905.g5e 357 kB/s | 45 kB 00:00 2023-11-16T16:35:08.349 INFO:teuthology.orchestra.run.smithi125.stdout:(19/131): python3-ceph-common-18.0.0-3905.g5e4a 1.3 MB/s | 119 kB 00:00 2023-11-16T16:35:08.441 INFO:teuthology.orchestra.run.smithi125.stdout:(20/131): python3-cephfs-18.0.0-3905.g5e4a743a. 2.1 MB/s | 197 kB 00:00 2023-11-16T16:35:08.600 INFO:teuthology.orchestra.run.smithi125.stdout:(21/131): python3-rados-18.0.0-3905.g5e4a743a.e 2.3 MB/s | 371 kB 00:00 2023-11-16T16:35:08.700 INFO:teuthology.orchestra.run.smithi125.stdout:(22/131): python3-rbd-18.0.0-3905.g5e4a743a.el8 3.6 MB/s | 365 kB 00:00 2023-11-16T16:35:08.824 INFO:teuthology.orchestra.run.smithi125.stdout:(23/131): python3-rgw-18.0.0-3905.g5e4a743a.el8 863 kB/s | 107 kB 00:00 2023-11-16T16:35:08.925 INFO:teuthology.orchestra.run.smithi125.stdout:(24/131): rbd-fuse-18.0.0-3905.g5e4a743a.el8.x8 931 kB/s | 94 kB 00:00 2023-11-16T16:35:09.151 INFO:teuthology.orchestra.run.smithi159.stdout:(10/131): ceph-radosgw-18.0.0-3905.g5e4a743a.el 3.1 MB/s | 13 MB 00:04 2023-11-16T16:35:09.326 INFO:teuthology.orchestra.run.smithi159.stdout:(11/131): cephfs-mirror-18.0.0-3905.g5e4a743a.e 1.4 MB/s | 258 kB 00:00 2023-11-16T16:35:09.410 INFO:teuthology.orchestra.run.smithi159.stdout:(12/131): libcephfs-devel-18.0.0-3905.g5e4a743a 372 kB/s | 31 kB 00:00 2023-11-16T16:35:09.719 INFO:teuthology.orchestra.run.smithi159.stdout:(13/131): libcephfs2-18.0.0-3905.g5e4a743a.el8. 2.4 MB/s | 771 kB 00:00 2023-11-16T16:35:09.759 INFO:teuthology.orchestra.run.smithi125.stdout:(25/131): rbd-mirror-18.0.0-3905.g5e4a743a.el8. 4.1 MB/s | 3.4 MB 00:00 2023-11-16T16:35:09.786 INFO:teuthology.orchestra.run.smithi159.stdout:(14/131): libcephsqlite-18.0.0-3905.g5e4a743a.e 2.0 MB/s | 137 kB 00:00 2023-11-16T16:35:09.860 INFO:teuthology.orchestra.run.smithi125.stdout:(26/131): rbd-nbd-18.0.0-3905.g5e4a743a.el8.x86 1.8 MB/s | 180 kB 00:00 2023-11-16T16:35:09.869 INFO:teuthology.orchestra.run.smithi159.stdout:(15/131): librados-devel-18.0.0-3905.g5e4a743a. 1.6 MB/s | 139 kB 00:00 2023-11-16T16:35:09.960 INFO:teuthology.orchestra.run.smithi125.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3905.g 231 kB/s | 23 kB 00:00 2023-11-16T16:35:10.060 INFO:teuthology.orchestra.run.smithi125.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3905.g5e4a743 1.2 MB/s | 127 kB 00:00 2023-11-16T16:35:10.095 INFO:teuthology.orchestra.run.smithi159.stdout:(16/131): libradosstriper1-18.0.0-3905.g5e4a743 2.1 MB/s | 473 kB 00:00 2023-11-16T16:35:10.262 INFO:teuthology.orchestra.run.smithi159.stdout:(17/131): ceph-osd-18.0.0-3905.g5e4a743a.el8.x8 2.9 MB/s | 18 MB 00:06 2023-11-16T16:35:10.362 INFO:teuthology.orchestra.run.smithi159.stdout:(18/131): python3-ceph-argparse-18.0.0-3905.g5e 449 kB/s | 45 kB 00:00 2023-11-16T16:35:10.479 INFO:teuthology.orchestra.run.smithi159.stdout:(19/131): python3-ceph-common-18.0.0-3905.g5e4a 1.0 MB/s | 119 kB 00:00 2023-11-16T16:35:10.569 INFO:teuthology.orchestra.run.smithi125.stdout:(29/131): ceph-mgr-dashboard-18.0.0-3905.g5e4a7 3.4 MB/s | 1.7 MB 00:00 2023-11-16T16:35:10.571 INFO:teuthology.orchestra.run.smithi159.stdout:(20/131): python3-cephfs-18.0.0-3905.g5e4a743a. 2.1 MB/s | 197 kB 00:00 2023-11-16T16:35:10.677 INFO:teuthology.orchestra.run.smithi125.stdout:(30/131): librgw2-18.0.0-3905.g5e4a743a.el8.x86 2.5 MB/s | 6.5 MB 00:02 2023-11-16T16:35:10.738 INFO:teuthology.orchestra.run.smithi159.stdout:(21/131): python3-rados-18.0.0-3905.g5e4a743a.e 2.2 MB/s | 371 kB 00:00 2023-11-16T16:35:10.819 INFO:teuthology.orchestra.run.smithi125.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3905.g5e 1.7 MB/s | 240 kB 00:00 2023-11-16T16:35:10.936 INFO:teuthology.orchestra.run.smithi125.stdout:(32/131): ceph-mgr-rook-18.0.0-3905.g5e4a743a.e 407 kB/s | 47 kB 00:00 2023-11-16T16:35:11.061 INFO:teuthology.orchestra.run.smithi125.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3905.g5 116 kB/s | 15 kB 00:00 2023-11-16T16:35:11.096 INFO:teuthology.orchestra.run.smithi159.stdout:(22/131): python3-rbd-18.0.0-3905.g5e4a743a.el8 1.0 MB/s | 365 kB 00:00 2023-11-16T16:35:11.197 INFO:teuthology.orchestra.run.smithi159.stdout:(23/131): python3-rgw-18.0.0-3905.g5e4a743a.el8 1.0 MB/s | 107 kB 00:00 2023-11-16T16:35:11.289 INFO:teuthology.orchestra.run.smithi159.stdout:(24/131): rbd-fuse-18.0.0-3905.g5e4a743a.el8.x8 1.0 MB/s | 94 kB 00:00 2023-11-16T16:35:11.503 INFO:teuthology.orchestra.run.smithi125.stdout:(34/131): ceph-volume-18.0.0-3905.g5e4a743a.el8 599 kB/s | 265 kB 00:00 2023-11-16T16:35:11.654 INFO:teuthology.orchestra.run.smithi125.stdout:(35/131): cephadm-18.0.0-3905.g5e4a743a.el8.noa 556 kB/s | 83 kB 00:00 2023-11-16T16:35:11.762 INFO:teuthology.orchestra.run.smithi125.stdout:(36/131): cephfs-top-18.0.0-3905.g5e4a743a.el8. 166 kB/s | 18 kB 00:00 2023-11-16T16:35:12.029 INFO:teuthology.orchestra.run.smithi125.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-11-16T16:35:12.097 INFO:teuthology.orchestra.run.smithi125.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 882 kB/s | 59 kB 00:00 2023-11-16T16:35:12.197 INFO:teuthology.orchestra.run.smithi125.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 751 kB/s | 75 kB 00:00 2023-11-16T16:35:12.423 INFO:teuthology.orchestra.run.smithi093.stdout:(7/131): ceph-base-18.0.0-3905.g5e4a743a.el8.x8 518 kB/s | 5.3 MB 00:10 2023-11-16T16:35:12.435 INFO:teuthology.orchestra.run.smithi125.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.0 MB/s | 476 kB 00:00 2023-11-16T16:35:12.489 INFO:teuthology.orchestra.run.smithi125.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 813 kB/s | 44 kB 00:00 2023-11-16T16:35:12.573 INFO:teuthology.orchestra.run.smithi125.stdout:(42/131): ceph-mgr-diskprediction-local-18.0.0- 3.7 MB/s | 7.4 MB 00:02 2023-11-16T16:35:12.598 INFO:teuthology.orchestra.run.smithi125.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2023-11-16T16:35:12.656 INFO:teuthology.orchestra.run.smithi125.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 280 kB/s | 16 kB 00:00 2023-11-16T16:35:12.690 INFO:teuthology.orchestra.run.smithi125.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2023-11-16T16:35:12.765 INFO:teuthology.orchestra.run.smithi125.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2023-11-16T16:35:12.856 INFO:teuthology.orchestra.run.smithi159.stdout:(25/131): rbd-mirror-18.0.0-3905.g5e4a743a.el8. 2.2 MB/s | 3.4 MB 00:01 2023-11-16T16:35:12.874 INFO:teuthology.orchestra.run.smithi125.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.0 MB/s | 384 kB 00:00 2023-11-16T16:35:12.891 INFO:teuthology.orchestra.run.smithi125.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 649 kB/s | 81 kB 00:00 2023-11-16T16:35:12.956 INFO:teuthology.orchestra.run.smithi159.stdout:(26/131): librgw2-18.0.0-3905.g5e4a743a.el8.x86 2.3 MB/s | 6.5 MB 00:02 2023-11-16T16:35:12.958 INFO:teuthology.orchestra.run.smithi125.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 264 kB/s | 18 kB 00:00 2023-11-16T16:35:12.983 INFO:teuthology.orchestra.run.smithi125.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-11-16T16:35:13.022 INFO:teuthology.orchestra.run.smithi159.stdout:(27/131): rbd-nbd-18.0.0-3905.g5e4a743a.el8.x86 1.1 MB/s | 180 kB 00:00 2023-11-16T16:35:13.048 INFO:teuthology.orchestra.run.smithi159.stdout:(28/131): ceph-grafana-dashboards-18.0.0-3905.g 251 kB/s | 23 kB 00:00 2023-11-16T16:35:13.067 INFO:teuthology.orchestra.run.smithi125.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 582 kB/s | 49 kB 00:00 2023-11-16T16:35:13.098 INFO:teuthology.orchestra.run.smithi159.stdout:(29/131): ceph-mgr-cephadm-18.0.0-3905.g5e4a743 1.7 MB/s | 127 kB 00:00 2023-11-16T16:35:13.101 INFO:teuthology.orchestra.run.smithi125.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 8.2 MB/s | 1.2 MB 00:00 2023-11-16T16:35:13.176 INFO:teuthology.orchestra.run.smithi125.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 541 kB/s | 59 kB 00:00 2023-11-16T16:35:13.201 INFO:teuthology.orchestra.run.smithi125.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 600 kB/s | 60 kB 00:00 2023-11-16T16:35:13.251 INFO:teuthology.orchestra.run.smithi125.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2023-11-16T16:35:13.341 INFO:teuthology.orchestra.run.smithi125.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 112 kB/s | 16 kB 00:00 2023-11-16T16:35:13.427 INFO:teuthology.orchestra.run.smithi125.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 2.2 MB/s | 196 kB 00:00 2023-11-16T16:35:13.452 INFO:teuthology.orchestra.run.smithi125.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 168 kB/s | 34 kB 00:00 2023-11-16T16:35:13.503 INFO:teuthology.orchestra.run.smithi125.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 855 kB/s | 64 kB 00:00 2023-11-16T16:35:13.528 INFO:teuthology.orchestra.run.smithi125.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2023-11-16T16:35:13.553 INFO:teuthology.orchestra.run.smithi125.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 475 kB/s | 24 kB 00:00 2023-11-16T16:35:13.646 INFO:teuthology.orchestra.run.smithi125.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 299 kB/s | 27 kB 00:00 2023-11-16T16:35:13.671 INFO:teuthology.orchestra.run.smithi125.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 205 kB/s | 29 kB 00:00 2023-11-16T16:35:13.746 INFO:teuthology.orchestra.run.smithi125.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-11-16T16:35:13.771 INFO:teuthology.orchestra.run.smithi125.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-11-16T16:35:13.807 INFO:teuthology.orchestra.run.smithi159.stdout:(30/131): ceph-mgr-dashboard-18.0.0-3905.g5e4a7 2.3 MB/s | 1.7 MB 00:00 2023-11-16T16:35:13.846 INFO:teuthology.orchestra.run.smithi125.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 612 kB/s | 61 kB 00:00 2023-11-16T16:35:13.871 INFO:teuthology.orchestra.run.smithi125.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 852 kB/s | 85 kB 00:00 2023-11-16T16:35:13.897 INFO:teuthology.orchestra.run.smithi125.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 457 kB/s | 23 kB 00:00 2023-11-16T16:35:13.940 INFO:teuthology.orchestra.run.smithi159.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3905.g5e 1.8 MB/s | 240 kB 00:00 2023-11-16T16:35:13.959 INFO:teuthology.orchestra.run.smithi125.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.1 MB/s | 70 kB 00:00 2023-11-16T16:35:13.991 INFO:teuthology.orchestra.run.smithi159.stdout:(32/131): ceph-mgr-rook-18.0.0-3905.g5e4a743a.e 943 kB/s | 47 kB 00:00 2023-11-16T16:35:14.049 INFO:teuthology.orchestra.run.smithi159.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3905.g5 249 kB/s | 15 kB 00:00 2023-11-16T16:35:14.106 INFO:teuthology.orchestra.run.smithi125.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 7.4 MB/s | 1.7 MB 00:00 2023-11-16T16:35:14.217 INFO:teuthology.orchestra.run.smithi159.stdout:(34/131): ceph-volume-18.0.0-3905.g5e4a743a.el8 1.6 MB/s | 265 kB 00:00 2023-11-16T16:35:14.300 INFO:teuthology.orchestra.run.smithi159.stdout:(35/131): cephadm-18.0.0-3905.g5e4a743a.el8.noa 1.0 MB/s | 83 kB 00:00 2023-11-16T16:35:14.384 INFO:teuthology.orchestra.run.smithi159.stdout:(36/131): cephfs-top-18.0.0-3905.g5e4a743a.el8. 215 kB/s | 18 kB 00:00 2023-11-16T16:35:14.389 INFO:teuthology.orchestra.run.smithi125.stdout:(71/131): python3-scipy-1.0.0-21.module+el8.5.0 32 MB/s | 14 MB 00:00 2023-11-16T16:35:14.406 INFO:teuthology.orchestra.run.smithi125.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 367 kB/s | 110 kB 00:00 2023-11-16T16:35:14.498 INFO:teuthology.orchestra.run.smithi125.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 499 kB/s | 54 kB 00:00 2023-11-16T16:35:14.574 INFO:teuthology.orchestra.run.smithi125.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.5 MB/s | 259 kB 00:00 2023-11-16T16:35:14.599 INFO:teuthology.orchestra.run.smithi125.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.1 MB/s | 323 kB 00:00 2023-11-16T16:35:14.634 INFO:teuthology.orchestra.run.smithi159.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-11-16T16:35:14.682 INFO:teuthology.orchestra.run.smithi125.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 8.1 MB/s | 688 kB 00:00 2023-11-16T16:35:14.701 INFO:teuthology.orchestra.run.smithi159.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 883 kB/s | 59 kB 00:00 2023-11-16T16:35:14.741 INFO:teuthology.orchestra.run.smithi125.stdout:(77/131): python3-markupsafe-0.23-19.el8.x86_64 232 kB/s | 39 kB 00:00 2023-11-16T16:35:14.768 INFO:teuthology.orchestra.run.smithi159.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-11-16T16:35:14.800 INFO:teuthology.orchestra.run.smithi125.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 4.5 MB/s | 538 kB 00:00 2023-11-16T16:35:14.867 INFO:teuthology.orchestra.run.smithi125.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-11-16T16:35:14.910 INFO:teuthology.orchestra.run.smithi159.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.3 MB/s | 476 kB 00:00 2023-11-16T16:35:14.917 INFO:teuthology.orchestra.run.smithi125.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2023-11-16T16:35:14.976 INFO:teuthology.orchestra.run.smithi125.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 6.4 MB/s | 713 kB 00:00 2023-11-16T16:35:14.977 INFO:teuthology.orchestra.run.smithi159.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 654 kB/s | 44 kB 00:00 2023-11-16T16:35:15.044 INFO:teuthology.orchestra.run.smithi159.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2023-11-16T16:35:15.093 INFO:teuthology.orchestra.run.smithi125.stdout:(82/131): python3-babel-2.5.1-7.el8.noarch.rpm 27 MB/s | 4.8 MB 00:00 2023-11-16T16:35:15.111 INFO:teuthology.orchestra.run.smithi159.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 494 kB/s | 33 kB 00:00 2023-11-16T16:35:15.143 INFO:teuthology.orchestra.run.smithi125.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 159 kB/s | 27 kB 00:00 2023-11-16T16:35:15.170 INFO:teuthology.orchestra.run.smithi159.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 280 kB/s | 16 kB 00:00 2023-11-16T16:35:15.245 INFO:teuthology.orchestra.run.smithi159.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-11-16T16:35:15.268 INFO:teuthology.orchestra.run.smithi125.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 353 kB/s | 44 kB 00:00 2023-11-16T16:35:15.294 INFO:teuthology.orchestra.run.smithi125.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2023-11-16T16:35:15.363 INFO:teuthology.orchestra.run.smithi159.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2023-11-16T16:35:15.437 INFO:teuthology.orchestra.run.smithi159.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-11-16T16:35:15.477 INFO:teuthology.orchestra.run.smithi125.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.4 MB/s | 457 kB 00:00 2023-11-16T16:35:15.496 INFO:teuthology.orchestra.run.smithi159.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-11-16T16:35:15.538 INFO:teuthology.orchestra.run.smithi159.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 422 kB/s | 18 kB 00:00 2023-11-16T16:35:15.569 INFO:teuthology.orchestra.run.smithi125.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2023-11-16T16:35:15.594 INFO:teuthology.orchestra.run.smithi125.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.9 MB/s | 225 kB 00:00 2023-11-16T16:35:15.688 INFO:teuthology.orchestra.run.smithi159.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 7.8 MB/s | 1.2 MB 00:00 2023-11-16T16:35:15.703 INFO:teuthology.orchestra.run.smithi125.stdout:(89/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-11-16T16:35:15.747 INFO:teuthology.orchestra.run.smithi159.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 836 kB/s | 49 kB 00:00 2023-11-16T16:35:15.770 INFO:teuthology.orchestra.run.smithi125.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 319 kB/s | 64 kB 00:00 2023-11-16T16:35:15.814 INFO:teuthology.orchestra.run.smithi159.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 881 kB/s | 59 kB 00:00 2023-11-16T16:35:15.837 INFO:teuthology.orchestra.run.smithi125.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 150 kB/s | 20 kB 00:00 2023-11-16T16:35:15.856 INFO:teuthology.orchestra.run.smithi159.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-11-16T16:35:15.896 INFO:teuthology.orchestra.run.smithi125.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 827 kB/s | 103 kB 00:00 2023-11-16T16:35:15.906 INFO:teuthology.orchestra.run.smithi159.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-11-16T16:35:15.979 INFO:teuthology.orchestra.run.smithi125.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 4.9 MB/s | 709 kB 00:00 2023-11-16T16:35:15.990 INFO:teuthology.orchestra.run.smithi159.stdout:(55/131): ceph-mgr-diskprediction-local-18.0.0- 2.6 MB/s | 7.4 MB 00:02 2023-11-16T16:35:16.015 INFO:teuthology.orchestra.run.smithi159.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2023-11-16T16:35:16.040 INFO:teuthology.orchestra.run.smithi159.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 670 kB/s | 34 kB 00:00 2023-11-16T16:35:16.090 INFO:teuthology.orchestra.run.smithi159.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-11-16T16:35:16.121 INFO:teuthology.orchestra.run.smithi125.stdout:(94/131): openblas-threads-0.3.15-3.el8.x86_64. 22 MB/s | 4.9 MB 00:00 2023-11-16T16:35:16.146 INFO:teuthology.orchestra.run.smithi125.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-11-16T16:35:16.149 INFO:teuthology.orchestra.run.smithi159.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 320 kB/s | 19 kB 00:00 2023-11-16T16:35:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 359 kB/s | 24 kB 00:00 2023-11-16T16:35:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:(61/131): python3-routes-2.4.1-12.el8.noarch.rp 868 kB/s | 196 kB 00:00 2023-11-16T16:35:16.247 INFO:teuthology.orchestra.run.smithi125.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 2.0 MB/s | 202 kB 00:00 2023-11-16T16:35:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 500 kB/s | 29 kB 00:00 2023-11-16T16:35:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 470 kB/s | 27 kB 00:00 2023-11-16T16:35:16.323 INFO:teuthology.orchestra.run.smithi093.stdout:(8/131): ceph-osd-18.0.0-3905.g5e4a743a.el8.x86 2.1 MB/s | 18 MB 00:08 2023-11-16T16:35:16.334 INFO:teuthology.orchestra.run.smithi159.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 4.3 MB/s | 255 kB 00:00 2023-11-16T16:35:16.381 INFO:teuthology.orchestra.run.smithi093.stdout:(9/131): ceph-selinux-18.0.0-3905.g5e4a743a.el8 420 kB/s | 24 kB 00:00 2023-11-16T16:35:16.388 INFO:teuthology.orchestra.run.smithi125.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-11-16T16:35:16.393 INFO:teuthology.orchestra.run.smithi159.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-11-16T16:35:16.410 INFO:teuthology.orchestra.run.smithi159.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-11-16T16:35:16.422 INFO:teuthology.orchestra.run.smithi125.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.0 MB/s | 354 kB 00:00 2023-11-16T16:35:16.452 INFO:teuthology.orchestra.run.smithi159.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-11-16T16:35:16.673 INFO:teuthology.orchestra.run.smithi125.stdout:(99/131): postgresql-server-10.21-2.module+el8. 18 MB/s | 5.1 MB 00:00 2023-11-16T16:35:16.706 INFO:teuthology.orchestra.run.smithi125.stdout:(100/131): postgresql-contrib-10.21-2.module+el 2.8 MB/s | 808 kB 00:00 2023-11-16T16:35:16.873 INFO:teuthology.orchestra.run.smithi125.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 7.5 MB/s | 1.5 MB 00:00 2023-11-16T16:35:16.973 INFO:teuthology.orchestra.run.smithi125.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 175 kB/s | 47 kB 00:00 2023-11-16T16:35:17.085 INFO:teuthology.orchestra.run.smithi159.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 34 kB/s | 23 kB 00:00 2023-11-16T16:35:17.111 INFO:teuthology.orchestra.run.smithi159.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-16T16:35:17.178 INFO:teuthology.orchestra.run.smithi159.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 2.4 MB/s | 1.7 MB 00:00 2023-11-16T16:35:17.241 INFO:teuthology.orchestra.run.smithi125.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 545 kB/s | 200 kB 00:00 2023-11-16T16:35:17.266 INFO:teuthology.orchestra.run.smithi125.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 146 kB/s | 43 kB 00:00 2023-11-16T16:35:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 696 kB/s | 110 kB 00:00 2023-11-16T16:35:17.341 INFO:teuthology.orchestra.run.smithi125.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.1 MB/s | 220 kB 00:00 2023-11-16T16:35:17.383 INFO:teuthology.orchestra.run.smithi125.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 411 kB/s | 48 kB 00:00 2023-11-16T16:35:17.467 INFO:teuthology.orchestra.run.smithi125.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 694 kB/s | 87 kB 00:00 2023-11-16T16:35:17.517 INFO:teuthology.orchestra.run.smithi125.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 319 kB/s | 43 kB 00:00 2023-11-16T16:35:17.610 INFO:teuthology.orchestra.run.smithi125.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 762 kB/s | 109 kB 00:00 2023-11-16T16:35:17.659 INFO:teuthology.orchestra.run.smithi125.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2023-11-16T16:35:17.678 INFO:teuthology.orchestra.run.smithi159.stdout:(72/131): python3-scipy-1.0.0-21.module+el8.5.0 24 MB/s | 14 MB 00:00 2023-11-16T16:35:17.735 INFO:teuthology.orchestra.run.smithi125.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.4 MB/s | 560 kB 00:00 2023-11-16T16:35:17.785 INFO:teuthology.orchestra.run.smithi125.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 149 kB/s | 19 kB 00:00 2023-11-16T16:35:17.879 INFO:teuthology.orchestra.run.smithi159.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 100 kB/s | 54 kB 00:00 2023-11-16T16:35:17.912 INFO:teuthology.orchestra.run.smithi159.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.1 MB/s | 259 kB 00:00 2023-11-16T16:35:17.923 INFO:teuthology.orchestra.run.smithi125.stdout:(113/131): python3-cryptography-3.2.1-5.el8.x86 4.0 MB/s | 559 kB 00:00 2023-11-16T16:35:17.960 INFO:teuthology.orchestra.run.smithi125.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 687 kB/s | 155 kB 00:00 2023-11-16T16:35:17.988 INFO:teuthology.orchestra.run.smithi159.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2023-11-16T16:35:18.013 INFO:teuthology.orchestra.run.smithi159.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 387 kB/s | 39 kB 00:00 2023-11-16T16:35:18.094 INFO:teuthology.orchestra.run.smithi125.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 105 kB/s | 18 kB 00:00 2023-11-16T16:35:18.128 INFO:teuthology.orchestra.run.smithi125.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.5 MB/s | 605 kB 00:00 2023-11-16T16:35:18.146 INFO:teuthology.orchestra.run.smithi159.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 4.2 MB/s | 688 kB 00:00 2023-11-16T16:35:18.180 INFO:teuthology.orchestra.run.smithi159.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.1 MB/s | 538 kB 00:00 2023-11-16T16:35:18.270 INFO:teuthology.orchestra.run.smithi125.stdout:(117/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 16 MB/s | 2.3 MB 00:00 2023-11-16T16:35:18.295 INFO:teuthology.orchestra.run.smithi125.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2023-11-16T16:35:18.306 INFO:teuthology.orchestra.run.smithi159.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 990 kB/s | 157 kB 00:00 2023-11-16T16:35:18.331 INFO:teuthology.orchestra.run.smithi159.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2023-11-16T16:35:18.445 INFO:teuthology.orchestra.run.smithi125.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 413 kB/s | 72 kB 00:00 2023-11-16T16:35:18.481 INFO:teuthology.orchestra.run.smithi159.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 4.0 MB/s | 713 kB 00:00 2023-11-16T16:35:18.523 INFO:teuthology.orchestra.run.smithi125.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 364 kB/s | 83 kB 00:00 2023-11-16T16:35:18.579 INFO:teuthology.orchestra.run.smithi125.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 448 kB/s | 60 kB 00:00 2023-11-16T16:35:18.615 INFO:teuthology.orchestra.run.smithi159.stdout:(82/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-11-16T16:35:18.629 INFO:teuthology.orchestra.run.smithi125.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 924 kB/s | 98 kB 00:00 2023-11-16T16:35:18.640 INFO:teuthology.orchestra.run.smithi159.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 168 kB/s | 27 kB 00:00 2023-11-16T16:35:18.655 INFO:teuthology.orchestra.run.smithi125.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 768 kB/s | 58 kB 00:00 2023-11-16T16:35:18.780 INFO:teuthology.orchestra.run.smithi125.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 257 kB/s | 39 kB 00:00 2023-11-16T16:35:18.782 INFO:teuthology.orchestra.run.smithi159.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 1.9 MB/s | 320 kB 00:00 2023-11-16T16:35:18.807 INFO:teuthology.orchestra.run.smithi159.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 263 kB/s | 44 kB 00:00 2023-11-16T16:35:18.847 INFO:teuthology.orchestra.run.smithi125.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 3.3 MB/s | 644 kB 00:00 2023-11-16T16:35:18.947 INFO:teuthology.orchestra.run.smithi125.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.0 MB/s | 171 kB 00:00 2023-11-16T16:35:18.974 INFO:teuthology.orchestra.run.smithi159.stdout:(86/131): python3-numpy-1.14.3-10.el8.x86_64.rp 19 MB/s | 3.7 MB 00:00 2023-11-16T16:35:19.008 INFO:teuthology.orchestra.run.smithi159.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.2 MB/s | 457 kB 00:00 2023-11-16T16:35:19.014 INFO:teuthology.orchestra.run.smithi125.stdout:(127/131): NetworkManager-initscripts-updown-1. 825 kB/s | 138 kB 00:00 2023-11-16T16:35:19.125 INFO:teuthology.orchestra.run.smithi159.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2023-11-16T16:35:19.151 INFO:teuthology.orchestra.run.smithi159.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 450 kB/s | 64 kB 00:00 2023-11-16T16:35:19.276 INFO:teuthology.orchestra.run.smithi159.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 841 kB/s | 126 kB 00:00 2023-11-16T16:35:19.301 INFO:teuthology.orchestra.run.smithi159.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 133 kB/s | 20 kB 00:00 2023-11-16T16:35:19.418 INFO:teuthology.orchestra.run.smithi159.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 731 kB/s | 103 kB 00:00 2023-11-16T16:35:19.468 INFO:teuthology.orchestra.run.smithi159.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 4.1 MB/s | 709 kB 00:00 2023-11-16T16:35:19.531 INFO:teuthology.orchestra.run.smithi125.stdout:(128/131): ceph-common-18.0.0-3905.g5e4a743a.el 1.1 MB/s | 22 MB 00:19 2023-11-16T16:35:19.610 INFO:teuthology.orchestra.run.smithi159.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2023-11-16T16:35:19.656 INFO:teuthology.orchestra.run.smithi125.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 6.3 MB/s | 798 kB 00:00 2023-11-16T16:35:19.677 INFO:teuthology.orchestra.run.smithi159.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 19 MB/s | 4.9 MB 00:00 2023-11-16T16:35:19.802 INFO:teuthology.orchestra.run.smithi159.stdout:(96/131): openblas-0.3.15-3.el8.x86_64.rpm 25 MB/s | 4.7 MB 00:00 2023-11-16T16:35:19.844 INFO:teuthology.orchestra.run.smithi159.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2023-11-16T16:35:19.936 INFO:teuthology.orchestra.run.smithi159.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-11-16T16:35:20.045 INFO:teuthology.orchestra.run.smithi159.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 7.3 MB/s | 808 kB 00:00 2023-11-16T16:35:20.237 INFO:teuthology.orchestra.run.smithi159.stdout:(100/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-11-16T16:35:20.279 INFO:teuthology.orchestra.run.smithi159.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 6.4 MB/s | 1.5 MB 00:00 2023-11-16T16:35:20.371 INFO:teuthology.orchestra.run.smithi159.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 350 kB/s | 47 kB 00:00 2023-11-16T16:35:20.429 INFO:teuthology.orchestra.run.smithi159.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.3 MB/s | 200 kB 00:00 2023-11-16T16:35:20.455 INFO:teuthology.orchestra.run.smithi159.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 510 kB/s | 43 kB 00:00 2023-11-16T16:35:20.538 INFO:teuthology.orchestra.run.smithi159.stdout:(105/131): keyutils-libs-devel-1.5.10-9.el8.x86 577 kB/s | 48 kB 00:00 2023-11-16T16:35:20.572 INFO:teuthology.orchestra.run.smithi159.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2023-11-16T16:35:20.623 INFO:teuthology.orchestra.run.smithi125.stdout:(130/131): librbd1-18.0.0-3905.g5e4a743a.el8.x8 2.2 MB/s | 3.6 MB 00:01 2023-11-16T16:35:20.681 INFO:teuthology.orchestra.run.smithi159.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 613 kB/s | 87 kB 00:00 2023-11-16T16:35:20.747 INFO:teuthology.orchestra.run.smithi159.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 243 kB/s | 43 kB 00:00 2023-11-16T16:35:20.840 INFO:teuthology.orchestra.run.smithi159.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 688 kB/s | 109 kB 00:00 2023-11-16T16:35:20.906 INFO:teuthology.orchestra.run.smithi159.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.4 MB/s | 229 kB 00:00 2023-11-16T16:35:21.015 INFO:teuthology.orchestra.run.smithi159.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.1 MB/s | 560 kB 00:00 2023-11-16T16:35:21.082 INFO:teuthology.orchestra.run.smithi159.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 106 kB/s | 19 kB 00:00 2023-11-16T16:35:21.158 INFO:teuthology.orchestra.run.smithi159.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2023-11-16T16:35:21.191 INFO:teuthology.orchestra.run.smithi159.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 5.0 MB/s | 559 kB 00:00 2023-11-16T16:35:21.308 INFO:teuthology.orchestra.run.smithi159.stdout:(115/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.1 MB/s | 605 kB 00:00 2023-11-16T16:35:21.334 INFO:teuthology.orchestra.run.smithi159.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 102 kB/s | 18 kB 00:00 2023-11-16T16:35:21.425 INFO:teuthology.orchestra.run.smithi159.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.0 MB/s | 238 kB 00:00 2023-11-16T16:35:21.465 INFO:teuthology.orchestra.run.smithi125.stdout:(131/131): librados2-18.0.0-3905.g5e4a743a.el8. 1.5 MB/s | 3.7 MB 00:02 2023-11-16T16:35:21.468 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:35:21.468 INFO:teuthology.orchestra.run.smithi125.stdout:Total 9.0 MB/s | 197 MB 00:21 2023-11-16T16:35:21.584 INFO:teuthology.orchestra.run.smithi159.stdout:(118/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 9.3 MB/s | 2.3 MB 00:00 2023-11-16T16:35:21.610 INFO:teuthology.orchestra.run.smithi159.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 393 kB/s | 72 kB 00:00 2023-11-16T16:35:21.693 INFO:teuthology.orchestra.run.smithi159.stdout:(120/131): elfutils-libelf-devel-0.186-1.el8.x8 720 kB/s | 60 kB 00:00 2023-11-16T16:35:21.718 INFO:teuthology.orchestra.run.smithi159.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 619 kB/s | 83 kB 00:00 2023-11-16T16:35:21.819 INFO:teuthology.orchestra.run.smithi159.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 783 kB/s | 98 kB 00:00 2023-11-16T16:35:21.902 INFO:teuthology.orchestra.run.smithi159.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 314 kB/s | 58 kB 00:00 2023-11-16T16:35:21.961 INFO:teuthology.orchestra.run.smithi159.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 271 kB/s | 39 kB 00:00 2023-11-16T16:35:22.061 INFO:teuthology.orchestra.run.smithi159.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 4.0 MB/s | 644 kB 00:00 2023-11-16T16:35:22.087 INFO:teuthology.orchestra.run.smithi159.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.3 MB/s | 171 kB 00:00 2023-11-16T16:35:22.212 INFO:teuthology.orchestra.run.smithi159.stdout:(127/131): NetworkManager-initscripts-updown-1. 916 kB/s | 138 kB 00:00 2023-11-16T16:35:22.314 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-11-16T16:35:22.434 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-11-16T16:35:22.434 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-11-16T16:35:23.198 INFO:teuthology.orchestra.run.smithi093.stdout:(10/131): ceph-common-18.0.0-3905.g5e4a743a.el8 1.0 MB/s | 22 MB 00:21 2023-11-16T16:35:23.279 INFO:teuthology.orchestra.run.smithi159.stdout:(128/131): librados2-18.0.0-3905.g5e4a743a.el8. 3.1 MB/s | 3.7 MB 00:01 2023-11-16T16:35:23.423 INFO:teuthology.orchestra.run.smithi093.stdout:(11/131): cephfs-mirror-18.0.0-3905.g5e4a743a.e 1.1 MB/s | 258 kB 00:00 2023-11-16T16:35:23.513 INFO:teuthology.orchestra.run.smithi159.stdout:(129/131): librbd1-18.0.0-3905.g5e4a743a.el8.x8 2.7 MB/s | 3.6 MB 00:01 2023-11-16T16:35:23.523 INFO:teuthology.orchestra.run.smithi093.stdout:(12/131): libcephfs-devel-18.0.0-3905.g5e4a743a 311 kB/s | 31 kB 00:00 2023-11-16T16:35:23.538 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-11-16T16:35:23.542 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-11-16T16:35:23.546 INFO:teuthology.orchestra.run.smithi159.stdout:(130/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 2.9 MB/s | 798 kB 00:00 2023-11-16T16:35:23.915 INFO:teuthology.orchestra.run.smithi093.stdout:(13/131): libcephfs2-18.0.0-3905.g5e4a743a.el8. 1.9 MB/s | 771 kB 00:00 2023-11-16T16:35:24.091 INFO:teuthology.orchestra.run.smithi093.stdout:(14/131): ceph-radosgw-18.0.0-3905.g5e4a743a.el 1.1 MB/s | 13 MB 00:11 2023-11-16T16:35:24.124 INFO:teuthology.orchestra.run.smithi093.stdout:(15/131): libcephsqlite-18.0.0-3905.g5e4a743a.e 657 kB/s | 137 kB 00:00 2023-11-16T16:35:24.241 INFO:teuthology.orchestra.run.smithi093.stdout:(16/131): librados-devel-18.0.0-3905.g5e4a743a. 936 kB/s | 139 kB 00:00 2023-11-16T16:35:24.416 INFO:teuthology.orchestra.run.smithi093.stdout:(17/131): libradosstriper1-18.0.0-3905.g5e4a743 1.6 MB/s | 473 kB 00:00 2023-11-16T16:35:24.524 INFO:teuthology.orchestra.run.smithi093.stdout:(18/131): python3-ceph-argparse-18.0.0-3905.g5e 413 kB/s | 45 kB 00:00 2023-11-16T16:35:24.616 INFO:teuthology.orchestra.run.smithi093.stdout:(19/131): python3-ceph-common-18.0.0-3905.g5e4a 1.3 MB/s | 119 kB 00:00 2023-11-16T16:35:24.767 INFO:teuthology.orchestra.run.smithi093.stdout:(20/131): python3-cephfs-18.0.0-3905.g5e4a743a. 1.3 MB/s | 197 kB 00:00 2023-11-16T16:35:25.000 INFO:teuthology.orchestra.run.smithi093.stdout:(21/131): python3-rados-18.0.0-3905.g5e4a743a.e 1.6 MB/s | 371 kB 00:00 2023-11-16T16:35:25.292 INFO:teuthology.orchestra.run.smithi093.stdout:(22/131): python3-rbd-18.0.0-3905.g5e4a743a.el8 1.2 MB/s | 365 kB 00:00 2023-11-16T16:35:25.384 INFO:teuthology.orchestra.run.smithi093.stdout:(23/131): python3-rgw-18.0.0-3905.g5e4a743a.el8 1.1 MB/s | 107 kB 00:00 2023-11-16T16:35:25.459 INFO:teuthology.orchestra.run.smithi093.stdout:(24/131): rbd-fuse-18.0.0-3905.g5e4a743a.el8.x8 1.2 MB/s | 94 kB 00:00 2023-11-16T16:35:26.985 INFO:teuthology.orchestra.run.smithi093.stdout:(25/131): rbd-mirror-18.0.0-3905.g5e4a743a.el8. 2.3 MB/s | 3.4 MB 00:01 2023-11-16T16:35:27.151 INFO:teuthology.orchestra.run.smithi093.stdout:(26/131): rbd-nbd-18.0.0-3905.g5e4a743a.el8.x86 1.1 MB/s | 180 kB 00:00 2023-11-16T16:35:27.285 INFO:teuthology.orchestra.run.smithi093.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3905.g 173 kB/s | 23 kB 00:00 2023-11-16T16:35:27.427 INFO:teuthology.orchestra.run.smithi093.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3905.g5e4a743 900 kB/s | 127 kB 00:00 2023-11-16T16:35:27.602 INFO:teuthology.orchestra.run.smithi093.stdout:(29/131): librgw2-18.0.0-3905.g5e4a743a.el8.x86 1.9 MB/s | 6.5 MB 00:03 2023-11-16T16:35:28.453 INFO:teuthology.orchestra.run.smithi093.stdout:(30/131): ceph-test-18.0.0-3905.g5e4a743a.el8.x 3.4 MB/s | 41 MB 00:12 2023-11-16T16:35:28.593 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-11-16T16:35:28.778 INFO:teuthology.orchestra.run.smithi093.stdout:(31/131): ceph-mgr-dashboard-18.0.0-3905.g5e4a7 1.3 MB/s | 1.7 MB 00:01 2023-11-16T16:35:28.788 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-16T16:35:28.820 INFO:teuthology.orchestra.run.smithi093.stdout:(32/131): ceph-mgr-modules-core-18.0.0-3905.g5e 655 kB/s | 240 kB 00:00 2023-11-16T16:35:28.887 INFO:teuthology.orchestra.run.smithi093.stdout:(33/131): ceph-mgr-rook-18.0.0-3905.g5e4a743a.e 436 kB/s | 47 kB 00:00 2023-11-16T16:35:28.912 INFO:teuthology.orchestra.run.smithi093.stdout:(34/131): ceph-prometheus-alerts-18.0.0-3905.g5 158 kB/s | 15 kB 00:00 2023-11-16T16:35:28.989 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-16T16:35:29.037 INFO:teuthology.orchestra.run.smithi093.stdout:(35/131): cephadm-18.0.0-3905.g5e4a743a.el8.noa 669 kB/s | 83 kB 00:00 2023-11-16T16:35:29.138 INFO:teuthology.orchestra.run.smithi093.stdout:(36/131): ceph-volume-18.0.0-3905.g5e4a743a.el8 1.0 MB/s | 265 kB 00:00 2023-11-16T16:35:29.163 INFO:teuthology.orchestra.run.smithi093.stdout:(37/131): cephfs-top-18.0.0-3905.g5e4a743a.el8. 143 kB/s | 18 kB 00:00 2023-11-16T16:35:29.230 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-16T16:35:29.347 INFO:teuthology.orchestra.run.smithi093.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 321 kB/s | 59 kB 00:00 2023-11-16T16:35:29.407 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T16:35:29.486 INFO:teuthology.orchestra.run.smithi093.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 541 kB/s | 75 kB 00:00 2023-11-16T16:35:29.522 INFO:teuthology.orchestra.run.smithi093.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 798 kB/s | 306 kB 00:00 2023-11-16T16:35:29.545 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T16:35:29.589 INFO:teuthology.orchestra.run.smithi093.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 656 kB/s | 44 kB 00:00 2023-11-16T16:35:29.656 INFO:teuthology.orchestra.run.smithi093.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.7 MB/s | 476 kB 00:00 2023-11-16T16:35:29.682 INFO:teuthology.orchestra.run.smithi093.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-11-16T16:35:29.715 INFO:teuthology.orchestra.run.smithi093.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 566 kB/s | 33 kB 00:00 2023-11-16T16:35:29.740 INFO:teuthology.orchestra.run.smithi093.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 278 kB/s | 16 kB 00:00 2023-11-16T16:35:29.782 INFO:teuthology.orchestra.run.smithi093.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2023-11-16T16:35:29.815 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T16:35:29.841 INFO:teuthology.orchestra.run.smithi093.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.7 MB/s | 384 kB 00:00 2023-11-16T16:35:29.866 INFO:teuthology.orchestra.run.smithi093.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 974 kB/s | 81 kB 00:00 2023-11-16T16:35:29.925 INFO:teuthology.orchestra.run.smithi093.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2023-11-16T16:35:29.950 INFO:teuthology.orchestra.run.smithi093.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2023-11-16T16:35:30.017 INFO:teuthology.orchestra.run.smithi093.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 734 kB/s | 49 kB 00:00 2023-11-16T16:35:30.059 INFO:teuthology.orchestra.run.smithi093.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 8.7 MB/s | 1.2 MB 00:00 2023-11-16T16:35:30.084 INFO:teuthology.orchestra.run.smithi093.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 879 kB/s | 59 kB 00:00 2023-11-16T16:35:30.118 INFO:teuthology.orchestra.run.smithi093.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-11-16T16:35:30.160 INFO:teuthology.orchestra.run.smithi093.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2023-11-16T16:35:30.177 INFO:teuthology.orchestra.run.smithi093.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 268 kB/s | 16 kB 00:00 2023-11-16T16:35:30.219 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T16:35:30.219 INFO:teuthology.orchestra.run.smithi093.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 576 kB/s | 34 kB 00:00 2023-11-16T16:35:30.244 INFO:teuthology.orchestra.run.smithi093.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-11-16T16:35:30.286 INFO:teuthology.orchestra.run.smithi093.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 964 kB/s | 64 kB 00:00 2023-11-16T16:35:30.336 INFO:teuthology.orchestra.run.smithi093.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 204 kB/s | 19 kB 00:00 2023-11-16T16:35:30.370 INFO:teuthology.orchestra.run.smithi093.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 285 kB/s | 24 kB 00:00 2023-11-16T16:35:30.387 INFO:teuthology.orchestra.run.smithi093.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 581 kB/s | 29 kB 00:00 2023-11-16T16:35:30.421 INFO:teuthology.orchestra.run.smithi093.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 551 kB/s | 27 kB 00:00 2023-11-16T16:35:30.431 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:35:30.463 INFO:teuthology.orchestra.run.smithi093.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-11-16T16:35:30.488 INFO:teuthology.orchestra.run.smithi093.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.7 MB/s | 251 kB 00:00 2023-11-16T16:35:30.522 INFO:teuthology.orchestra.run.smithi093.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-11-16T16:35:30.555 INFO:teuthology.orchestra.run.smithi093.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.3 MB/s | 85 kB 00:00 2023-11-16T16:35:30.580 INFO:teuthology.orchestra.run.smithi093.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 392 kB/s | 23 kB 00:00 2023-11-16T16:35:30.606 INFO:teuthology.orchestra.run.smithi093.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-11-16T16:35:30.681 INFO:teuthology.orchestra.run.smithi093.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2023-11-16T16:35:30.832 INFO:teuthology.orchestra.run.smithi093.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 734 kB/s | 110 kB 00:00 2023-11-16T16:35:30.833 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:35:30.982 INFO:teuthology.orchestra.run.smithi093.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 362 kB/s | 54 kB 00:00 2023-11-16T16:35:31.056 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:35:31.164 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:35:31.274 INFO:teuthology.orchestra.run.smithi093.stdout:(73/131): python3-scipy-1.0.0-21.module+el8.5.0 21 MB/s | 14 MB 00:00 2023-11-16T16:35:31.307 INFO:teuthology.orchestra.run.smithi093.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 797 kB/s | 259 kB 00:00 2023-11-16T16:35:31.364 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:35:31.457 INFO:teuthology.orchestra.run.smithi093.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.7 MB/s | 323 kB 00:00 2023-11-16T16:35:31.462 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:35:31.482 INFO:teuthology.orchestra.run.smithi093.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 222 kB/s | 39 kB 00:00 2023-11-16T16:35:31.641 INFO:teuthology.orchestra.run.smithi093.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.3 MB/s | 538 kB 00:00 2023-11-16T16:35:31.665 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 9/134 2023-11-16T16:35:31.675 INFO:teuthology.orchestra.run.smithi093.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 3.1 MB/s | 688 kB 00:00 2023-11-16T16:35:31.809 INFO:teuthology.orchestra.run.smithi093.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 940 kB/s | 157 kB 00:00 2023-11-16T16:35:31.834 INFO:teuthology.orchestra.run.smithi093.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2023-11-16T16:35:31.893 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-16T16:35:31.993 INFO:teuthology.orchestra.run.smithi093.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 3.8 MB/s | 713 kB 00:00 2023-11-16T16:35:32.109 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-16T16:35:32.167 INFO:teuthology.orchestra.run.smithi093.stdout:(82/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-11-16T16:35:32.193 INFO:teuthology.orchestra.run.smithi093.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 133 kB/s | 27 kB 00:00 2023-11-16T16:35:32.301 INFO:teuthology.orchestra.run.smithi093.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2023-11-16T16:35:32.335 INFO:teuthology.orchestra.run.smithi093.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 310 kB/s | 44 kB 00:00 2023-11-16T16:35:32.477 INFO:teuthology.orchestra.run.smithi093.stdout:(86/131): ceph-mgr-diskprediction-local-18.0.0- 1.5 MB/s | 7.4 MB 00:04 2023-11-16T16:35:32.519 INFO:teuthology.orchestra.run.smithi093.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.4 MB/s | 457 kB 00:00 2023-11-16T16:35:32.551 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-16T16:35:32.619 INFO:teuthology.orchestra.run.smithi093.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2023-11-16T16:35:32.653 INFO:teuthology.orchestra.run.smithi093.stdout:(89/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2023-11-16T16:35:32.695 INFO:teuthology.orchestra.run.smithi093.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 376 kB/s | 64 kB 00:00 2023-11-16T16:35:32.753 INFO:teuthology.orchestra.run.smithi093.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 199 kB/s | 20 kB 00:00 2023-11-16T16:35:32.804 INFO:teuthology.orchestra.run.smithi093.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 952 kB/s | 103 kB 00:00 2023-11-16T16:35:32.818 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-16T16:35:32.829 INFO:teuthology.orchestra.run.smithi093.stdout:(93/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 604 kB/s | 126 kB 00:00 2023-11-16T16:35:32.888 INFO:teuthology.orchestra.run.smithi093.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2023-11-16T16:35:32.955 INFO:teuthology.orchestra.run.smithi093.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.5 MB/s | 198 kB 00:00 2023-11-16T16:35:32.956 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 14/134 2023-11-16T16:35:33.038 INFO:teuthology.orchestra.run.smithi093.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 2.4 MB/s | 202 kB 00:00 2023-11-16T16:35:33.058 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 14/134 2023-11-16T16:35:33.114 INFO:teuthology.orchestra.run.smithi093.stdout:(97/131): openblas-threads-0.3.15-3.el8.x86_64. 16 MB/s | 4.9 MB 00:00 2023-11-16T16:35:33.231 INFO:teuthology.orchestra.run.smithi093.stdout:(98/131): openblas-0.3.15-3.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2023-11-16T16:35:33.231 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 15/134 2023-11-16T16:35:33.264 INFO:teuthology.orchestra.run.smithi093.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.5 MB/s | 354 kB 00:00 2023-11-16T16:35:33.397 INFO:teuthology.orchestra.run.smithi093.stdout:(100/131): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2023-11-16T16:35:33.399 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 15/134 2023-11-16T16:35:33.448 INFO:teuthology.orchestra.run.smithi093.stdout:(101/131): postgresql-contrib-10.21-2.module+el 3.6 MB/s | 808 kB 00:00 2023-11-16T16:35:33.490 INFO:teuthology.orchestra.run.smithi093.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 6.7 MB/s | 1.5 MB 00:00 2023-11-16T16:35:33.532 INFO:teuthology.orchestra.run.smithi093.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 350 kB/s | 47 kB 00:00 2023-11-16T16:35:33.557 INFO:teuthology.orchestra.run.smithi093.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.8 MB/s | 200 kB 00:00 2023-11-16T16:35:33.649 INFO:teuthology.orchestra.run.smithi093.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 268 kB/s | 43 kB 00:00 2023-11-16T16:35:33.674 INFO:teuthology.orchestra.run.smithi093.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2023-11-16T16:35:33.699 INFO:teuthology.orchestra.run.smithi093.stdout:(107/131): keyutils-libs-devel-1.5.10-9.el8.x86 338 kB/s | 48 kB 00:00 2023-11-16T16:35:33.747 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T16:35:33.758 INFO:teuthology.orchestra.run.smithi093.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 800 kB/s | 87 kB 00:00 2023-11-16T16:35:33.800 INFO:teuthology.orchestra.run.smithi093.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 340 kB/s | 43 kB 00:00 2023-11-16T16:35:33.850 INFO:teuthology.orchestra.run.smithi093.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 725 kB/s | 109 kB 00:00 2023-11-16T16:35:33.880 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T16:35:33.892 INFO:teuthology.orchestra.run.smithi093.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.7 MB/s | 229 kB 00:00 2023-11-16T16:35:33.934 INFO:teuthology.orchestra.run.smithi093.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-11-16T16:35:33.993 INFO:teuthology.orchestra.run.smithi093.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 131 kB/s | 19 kB 00:00 2023-11-16T16:35:34.051 INFO:teuthology.orchestra.run.smithi093.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 975 kB/s | 155 kB 00:00 2023-11-16T16:35:34.064 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T16:35:34.085 INFO:teuthology.orchestra.run.smithi093.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 3.6 MB/s | 559 kB 00:00 2023-11-16T16:35:34.135 INFO:teuthology.orchestra.run.smithi093.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 126 kB/s | 18 kB 00:00 2023-11-16T16:35:34.261 INFO:teuthology.orchestra.run.smithi093.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 2.8 MB/s | 605 kB 00:00 2023-11-16T16:35:34.294 INFO:teuthology.orchestra.run.smithi093.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.1 MB/s | 238 kB 00:00 2023-11-16T16:35:34.370 INFO:teuthology.orchestra.run.smithi093.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 9.9 MB/s | 2.3 MB 00:00 2023-11-16T16:35:34.403 INFO:teuthology.orchestra.run.smithi093.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 508 kB/s | 72 kB 00:00 2023-11-16T16:35:34.453 INFO:teuthology.orchestra.run.smithi093.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 521 kB/s | 83 kB 00:00 2023-11-16T16:35:34.504 INFO:teuthology.orchestra.run.smithi093.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 447 kB/s | 60 kB 00:00 2023-11-16T16:35:34.554 INFO:teuthology.orchestra.run.smithi093.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 653 kB/s | 98 kB 00:00 2023-11-16T16:35:34.579 INFO:teuthology.orchestra.run.smithi093.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 459 kB/s | 58 kB 00:00 2023-11-16T16:35:34.671 INFO:teuthology.orchestra.run.smithi093.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 231 kB/s | 39 kB 00:00 2023-11-16T16:35:34.680 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T16:35:34.713 INFO:teuthology.orchestra.run.smithi093.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 4.0 MB/s | 644 kB 00:00 2023-11-16T16:35:34.755 INFO:teuthology.orchestra.run.smithi093.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 977 kB/s | 171 kB 00:00 2023-11-16T16:35:34.851 INFO:teuthology.orchestra.run.smithi093.stdout:(128/131): NetworkManager-initscripts-updown-1. 768 kB/s | 138 kB 00:00 2023-11-16T16:35:34.864 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T16:35:34.972 INFO:teuthology.orchestra.run.smithi093.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 6.4 MB/s | 798 kB 00:00 2023-11-16T16:35:35.472 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T16:35:35.697 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T16:35:36.331 INFO:teuthology.orchestra.run.smithi093.stdout:(130/131): librados2-18.0.0-3905.g5e4a743a.el8. 2.3 MB/s | 3.7 MB 00:01 2023-11-16T16:35:36.439 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T16:35:36.581 INFO:teuthology.orchestra.run.smithi093.stdout:(131/131): librbd1-18.0.0-3905.g5e4a743a.el8.x8 1.9 MB/s | 3.6 MB 00:01 2023-11-16T16:35:36.584 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:35:36.585 INFO:teuthology.orchestra.run.smithi093.stdout:Total 5.6 MB/s | 197 MB 00:34 2023-11-16T16:35:36.720 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-16T16:35:36.932 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-16T16:35:37.176 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-16T16:35:37.403 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-16T16:35:37.613 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-16T16:35:37.874 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-16T16:35:38.078 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8. 26/134 2023-11-16T16:35:38.199 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el 27/134 2023-11-16T16:35:38.248 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction check 2023-11-16T16:35:38.383 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction check succeeded. 2023-11-16T16:35:38.383 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction test 2023-11-16T16:35:39.116 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_6 28/134 2023-11-16T16:35:39.419 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-16T16:35:39.683 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-16T16:35:39.926 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction test succeeded. 2023-11-16T16:35:39.931 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction 2023-11-16T16:35:39.949 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-16T16:35:40.205 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T16:35:40.686 INFO:teuthology.orchestra.run.smithi159.stdout:(131/131): ceph-test-18.0.0-3905.g5e4a743a.el8. 1.2 MB/s | 41 MB 00:32 2023-11-16T16:35:40.688 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:35:40.689 INFO:teuthology.orchestra.run.smithi159.stdout:Total 4.8 MB/s | 197 MB 00:40 2023-11-16T16:35:41.482 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-11-16T16:35:41.602 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-11-16T16:35:41.602 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-11-16T16:35:41.973 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T16:35:42.331 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-16T16:35:42.592 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-16T16:35:42.691 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-11-16T16:35:42.695 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-11-16T16:35:42.781 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-16T16:35:42.994 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 36/134 2023-11-16T16:35:43.208 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_6 37/134 2023-11-16T16:35:43.369 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-16T16:35:43.589 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-16T16:35:43.832 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-16T16:35:43.999 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-16T16:35:44.262 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-16T16:35:44.509 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-16T16:35:44.907 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-16T16:35:44.937 INFO:teuthology.orchestra.run.smithi093.stdout: Preparing : 1/1 2023-11-16T16:35:45.109 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-16T16:35:45.257 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-16T16:35:45.323 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-16T16:35:45.455 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-16T16:35:45.536 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-16T16:35:45.716 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-16T16:35:45.735 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T16:35:45.868 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T16:35:46.000 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-16T16:35:46.110 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T16:35:46.363 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-16T16:35:46.573 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T16:35:46.603 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-16T16:35:46.768 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:35:46.846 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-16T16:35:47.066 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-16T16:35:47.239 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:35:47.282 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-16T16:35:47.393 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:35:47.482 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-16T16:35:47.546 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:35:47.743 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:35:47.772 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-16T16:35:47.801 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-11-16T16:35:47.849 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:35:48.023 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-16T16:35:48.033 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 9/134 2023-11-16T16:35:48.034 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-16T16:35:48.237 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-16T16:35:48.276 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-16T16:35:48.283 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-16T16:35:48.457 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-16T16:35:48.518 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-16T16:35:48.532 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-16T16:35:48.683 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T16:35:48.727 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-16T16:35:48.801 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T16:35:48.913 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-16T16:35:48.925 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-16T16:35:49.075 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T16:35:49.111 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T16:35:49.206 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T16:35:49.406 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-16T16:35:49.456 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-16T16:35:49.475 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T16:35:49.551 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 14/134 2023-11-16T16:35:49.681 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 14/134 2023-11-16T16:35:49.681 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-16T16:35:49.700 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:35:49.851 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 15/134 2023-11-16T16:35:49.878 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T16:35:49.951 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T16:35:49.957 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 15/134 2023-11-16T16:35:50.093 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T16:35:50.096 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:35:50.216 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-16T16:35:50.268 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T16:35:50.350 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:35:50.395 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-16T16:35:50.418 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T16:35:50.461 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:35:50.611 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-16T16:35:50.641 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:35:50.759 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:35:50.831 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-16T16:35:50.975 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 9/134 2023-11-16T16:35:51.103 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T16:35:51.121 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-16T16:35:51.245 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-16T16:35:51.293 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T16:35:51.376 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-16T16:35:51.436 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-16T16:35:51.515 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-16T16:35:51.689 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-16T16:35:51.859 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-16T16:35:51.878 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-16T16:35:51.987 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T16:35:52.106 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-16T16:35:52.128 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-16T16:35:52.201 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T16:35:52.275 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 14/134 2023-11-16T16:35:52.323 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-16T16:35:52.368 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 14/134 2023-11-16T16:35:52.489 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-16T16:35:52.533 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 15/134 2023-11-16T16:35:52.595 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 15/134 2023-11-16T16:35:52.692 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-16T16:35:52.791 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T16:35:52.917 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T16:35:52.957 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-16T16:35:53.032 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T16:35:53.083 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T16:35:53.215 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-16T16:35:53.283 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-16T16:35:53.400 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-16T16:35:53.544 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-16T16:35:53.673 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-16T16:35:53.695 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T16:35:53.757 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-16T16:35:53.874 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T16:35:53.969 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-16T16:35:53.977 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-16T16:35:54.168 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-16T16:35:54.219 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-16T16:35:54.378 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-16T16:35:54.471 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T16:35:54.480 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-16T16:35:54.658 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T16:35:54.674 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8. 26/134 2023-11-16T16:35:54.788 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el 27/134 2023-11-16T16:35:55.361 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T16:35:55.512 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-16T16:35:55.664 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-16T16:35:55.788 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_6 28/134 2023-11-16T16:35:55.822 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-16T16:35:55.891 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-16T16:35:56.109 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-16T16:35:56.153 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-16T16:35:56.155 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-16T16:35:56.363 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-16T16:35:56.452 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-16T16:35:56.459 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-16T16:35:56.597 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-16T16:35:56.732 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-16T16:35:56.750 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-16T16:35:56.825 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-16T16:35:56.925 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T16:35:56.975 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-16T16:35:57.022 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8. 26/134 2023-11-16T16:35:57.151 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el 27/134 2023-11-16T16:35:57.761 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-16T16:35:57.943 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 92/134 2023-11-16T16:35:58.003 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 92/134 2023-11-16T16:35:58.111 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_6 28/134 2023-11-16T16:35:58.218 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 93/134 2023-11-16T16:35:58.455 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-16T16:35:58.759 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-16T16:35:59.010 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T16:35:59.057 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-16T16:35:59.257 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T16:35:59.375 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-16T16:35:59.579 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-16T16:35:59.789 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-16T16:36:00.027 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 36/134 2023-11-16T16:36:00.213 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_6 37/134 2023-11-16T16:36:00.375 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-16T16:36:00.595 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-16T16:36:00.778 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-16T16:36:00.956 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-16T16:36:01.019 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T16:36:01.163 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-16T16:36:01.350 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-16T16:36:01.398 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-16T16:36:01.586 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-16T16:36:01.767 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-16T16:36:01.810 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-16T16:36:01.951 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:36:02.029 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 36/134 2023-11-16T16:36:02.136 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-16T16:36:02.235 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:36:02.235 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_6 37/134 2023-11-16T16:36:02.340 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-16T16:36:02.447 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-16T16:36:02.571 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-16T16:36:02.616 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-16T16:36:02.819 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-16T16:36:02.829 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:36:02.834 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-16T16:36:02.977 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-16T16:36:03.150 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 95/134 2023-11-16T16:36:03.202 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-16T16:36:03.223 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-16T16:36:03.262 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 95/134 2023-11-16T16:36:03.410 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 96/134 2023-11-16T16:36:03.414 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-16T16:36:03.434 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-16T16:36:03.619 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-16T16:36:03.841 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-16T16:36:03.851 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-16T16:36:04.023 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-16T16:36:04.176 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-16T16:36:04.180 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-16T16:36:04.391 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-16T16:36:04.509 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-16T16:36:04.606 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-16T16:36:04.814 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-16T16:36:04.869 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-16T16:36:05.081 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-16T16:36:05.165 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-16T16:36:05.291 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-16T16:36:05.380 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-16T16:36:05.483 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-16T16:36:05.624 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-16T16:36:05.660 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-16T16:36:05.860 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-16T16:36:05.907 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T16:36:05.999 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T16:36:06.085 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-16T16:36:06.213 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-16T16:36:06.267 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-16T16:36:06.430 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-16T16:36:06.582 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T16:36:06.614 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-16T16:36:06.676 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T16:36:06.870 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-16T16:36:06.919 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-16T16:36:07.094 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-16T16:36:07.152 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-16T16:36:07.333 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-16T16:36:07.379 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-16T16:36:07.546 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-16T16:36:07.565 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-16T16:36:07.724 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-16T16:36:07.816 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-16T16:36:07.930 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T16:36:07.996 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T16:36:08.099 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-16T16:36:08.237 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-16T16:36:08.303 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-16T16:36:08.420 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-16T16:36:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-16T16:36:08.613 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-16T16:36:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T16:36:08.783 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T16:36:08.805 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-16T16:36:09.004 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-16T16:36:09.050 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-16T16:36:09.168 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-16T16:36:09.239 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-16T16:36:09.339 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-16T16:36:09.438 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-16T16:36:09.616 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-16T16:36:09.633 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-16T16:36:09.863 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-16T16:36:09.931 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-16T16:36:10.089 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-16T16:36:10.154 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-16T16:36:10.334 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-16T16:36:10.340 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-16T16:36:10.483 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-16T16:36:10.567 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-16T16:36:10.732 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-16T16:36:10.743 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-16T16:36:10.908 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-16T16:36:11.016 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-16T16:36:11.123 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-16T16:36:11.291 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-16T16:36:11.461 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-16T16:36:11.681 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-16T16:36:11.941 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-16T16:36:12.127 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-16T16:36:12.343 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-16T16:36:12.404 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-16T16:36:12.620 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-16T16:36:12.644 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-16T16:36:12.811 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-16T16:36:12.904 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-16T16:36:13.001 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-16T16:36:13.258 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-16T16:36:13.591 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-16T16:36:13.801 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-16T16:36:14.114 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-16T16:36:14.416 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-16T16:36:14.628 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-16T16:36:14.638 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-16T16:36:14.814 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 92/134 2023-11-16T16:36:14.910 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 92/134 2023-11-16T16:36:15.047 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-16T16:36:15.123 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 93/134 2023-11-16T16:36:15.352 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-16T16:36:15.558 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-16T16:36:16.364 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-16T16:36:16.604 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 92/134 2023-11-16T16:36:16.663 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 92/134 2023-11-16T16:36:16.880 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 93/134 2023-11-16T16:36:20.028 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:36:20.674 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:36:20.706 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:36:20.938 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:36:21.377 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:36:21.535 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:36:21.643 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 95/134 2023-11-16T16:36:21.810 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 95/134 2023-11-16T16:36:21.816 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 95/134 2023-11-16T16:36:21.905 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 96/134 2023-11-16T16:36:21.923 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 95/134 2023-11-16T16:36:22.062 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 96/134 2023-11-16T16:36:22.193 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 96/134 2023-11-16T16:36:22.193 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /sys 2023-11-16T16:36:22.193 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /proc 2023-11-16T16:36:22.193 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /dev 2023-11-16T16:36:22.193 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /run 2023-11-16T16:36:22.193 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /mnt 2023-11-16T16:36:22.194 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /var/tmp 2023-11-16T16:36:22.194 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /home 2023-11-16T16:36:22.194 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /tmp 2023-11-16T16:36:22.194 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /dev 2023-11-16T16:36:22.194 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:36:22.581 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 97/134 2023-11-16T16:36:24.139 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 97/134 2023-11-16T16:36:24.355 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 98/134 2023-11-16T16:36:24.573 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 98/134 2023-11-16T16:36:24.806 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 99/134 2023-11-16T16:36:25.254 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 99/134 2023-11-16T16:36:25.456 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 100/134 2023-11-16T16:36:25.473 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 100/134 2023-11-16T16:36:27.216 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 101/134 2023-11-16T16:36:27.407 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 101/134 2023-11-16T16:36:27.578 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.e 102/134 2023-11-16T16:36:27.770 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a. 103/134 2023-11-16T16:36:27.980 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 104/134 2023-11-16T16:36:28.195 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 104/134 2023-11-16T16:36:28.470 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 105/134 2023-11-16T16:36:29.430 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 105/134 2023-11-16T16:36:29.638 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 106/134 2023-11-16T16:36:29.728 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 106/134 2023-11-16T16:36:30.124 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el 107/134 2023-11-16T16:36:30.389 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 108/134 2023-11-16T16:36:30.425 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 108/134 2023-11-16T16:36:30.621 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 109/134 2023-11-16T16:36:30.633 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 109/134 2023-11-16T16:36:31.877 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 110/134 2023-11-16T16:36:32.106 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 111/134 2023-11-16T16:36:32.141 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 111/134 2023-11-16T16:36:32.329 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 112/134 2023-11-16T16:36:32.379 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 112/134 2023-11-16T16:36:32.546 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 113/134 2023-11-16T16:36:32.892 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 113/134 2023-11-16T16:36:33.071 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 114/134 2023-11-16T16:36:36.985 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 114/134 2023-11-16T16:36:37.311 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 115/134 2023-11-16T16:36:37.520 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T16:36:37.721 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T16:36:38.618 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-16T16:36:38.959 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-16T16:36:39.189 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 119/134 2023-11-16T16:36:39.432 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T16:36:39.632 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_ 121/134 2023-11-16T16:36:39.779 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T16:36:41.021 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 96/134 2023-11-16T16:36:41.021 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2023-11-16T16:36:41.021 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2023-11-16T16:36:41.021 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /dev 2023-11-16T16:36:41.021 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /run 2023-11-16T16:36:41.021 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2023-11-16T16:36:41.021 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2023-11-16T16:36:41.021 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2023-11-16T16:36:41.021 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2023-11-16T16:36:41.022 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /dev 2023-11-16T16:36:41.022 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:36:41.359 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 97/134 2023-11-16T16:36:42.046 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:36:42.294 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 96/134 2023-11-16T16:36:42.294 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /sys 2023-11-16T16:36:42.294 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /proc 2023-11-16T16:36:42.294 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /dev 2023-11-16T16:36:42.294 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /run 2023-11-16T16:36:42.294 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /mnt 2023-11-16T16:36:42.294 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /var/tmp 2023-11-16T16:36:42.294 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /home 2023-11-16T16:36:42.294 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /tmp 2023-11-16T16:36:42.295 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /dev 2023-11-16T16:36:42.295 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:36:42.344 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:36:42.362 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:36:42.645 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 124/134 2023-11-16T16:36:42.719 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 97/134 2023-11-16T16:36:42.910 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 125/134 2023-11-16T16:36:42.920 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 97/134 2023-11-16T16:36:43.113 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 98/134 2023-11-16T16:36:43.214 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 126/134 2023-11-16T16:36:43.331 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 98/134 2023-11-16T16:36:43.525 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 99/134 2023-11-16T16:36:43.579 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-16T16:36:43.710 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:36:43.719 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:36:43.935 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-16T16:36:43.975 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 99/134 2023-11-16T16:36:44.142 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 100/134 2023-11-16T16:36:44.159 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 100/134 2023-11-16T16:36:44.160 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T16:36:44.240 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T16:36:44.416 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 97/134 2023-11-16T16:36:44.460 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T16:36:44.473 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T16:36:44.473 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:36:44.590 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 98/134 2023-11-16T16:36:44.663 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:36:44.663 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T16:36:44.855 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 98/134 2023-11-16T16:36:44.915 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T16:36:45.018 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 99/134 2023-11-16T16:36:45.143 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:36:45.534 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 99/134 2023-11-16T16:36:45.646 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:36:45.647 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 1/134 2023-11-16T16:36:45.647 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 2/134 2023-11-16T16:36:45.647 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 3/134 2023-11-16T16:36:45.647 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 4/134 2023-11-16T16:36:45.647 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 5/134 2023-11-16T16:36:45.647 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:36:45.647 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:36:45.648 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:36:45.648 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 9/134 2023-11-16T16:36:45.648 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 10/134 2023-11-16T16:36:45.648 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 11/134 2023-11-16T16:36:45.648 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 12/134 2023-11-16T16:36:45.648 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 13/134 2023-11-16T16:36:45.648 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_ 14/134 2023-11-16T16:36:45.648 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 15/134 2023-11-16T16:36:45.648 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 16/134 2023-11-16T16:36:45.648 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_6 17/134 2023-11-16T16:36:45.649 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 18/134 2023-11-16T16:36:45.649 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 19/134 2023-11-16T16:36:45.649 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el 20/134 2023-11-16T16:36:45.649 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8. 21/134 2023-11-16T16:36:45.649 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_6 22/134 2023-11-16T16:36:45.649 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 23/134 2023-11-16T16:36:45.649 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 24/134 2023-11-16T16:36:45.649 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 25/134 2023-11-16T16:36:45.649 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 26/134 2023-11-16T16:36:45.650 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 27/134 2023-11-16T16:36:45.650 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 28/134 2023-11-16T16:36:45.650 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a. 29/134 2023-11-16T16:36:45.660 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 30/134 2023-11-16T16:36:45.660 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 31/134 2023-11-16T16:36:45.660 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 32/134 2023-11-16T16:36:45.660 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el 33/134 2023-11-16T16:36:45.660 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 34/134 2023-11-16T16:36:45.660 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.e 35/134 2023-11-16T16:36:45.660 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 36/134 2023-11-16T16:36:45.661 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 37/134 2023-11-16T16:36:45.661 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 38/134 2023-11-16T16:36:45.661 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-16T16:36:45.661 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-16T16:36:45.661 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-16T16:36:45.661 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-16T16:36:45.661 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-16T16:36:45.661 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-16T16:36:45.661 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-16T16:36:45.662 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-16T16:36:45.662 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-16T16:36:45.662 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-16T16:36:45.662 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-16T16:36:45.662 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-16T16:36:45.662 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-16T16:36:45.662 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-16T16:36:45.662 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-16T16:36:45.662 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-16T16:36:45.663 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-16T16:36:45.663 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-16T16:36:45.663 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-16T16:36:45.663 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-16T16:36:45.663 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-16T16:36:45.663 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-16T16:36:45.663 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-16T16:36:45.663 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-16T16:36:45.663 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-16T16:36:45.663 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-16T16:36:45.664 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-16T16:36:45.664 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-16T16:36:45.664 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-16T16:36:45.664 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-16T16:36:45.664 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-16T16:36:45.664 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-16T16:36:45.664 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-16T16:36:45.664 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-16T16:36:45.664 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-16T16:36:45.665 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-16T16:36:45.665 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-16T16:36:45.665 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-16T16:36:45.665 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-16T16:36:45.665 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-16T16:36:45.665 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-16T16:36:45.665 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-16T16:36:45.665 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-16T16:36:45.665 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-16T16:36:45.665 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-16T16:36:45.666 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-16T16:36:45.666 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-16T16:36:45.666 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-16T16:36:45.666 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-16T16:36:45.666 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-16T16:36:45.666 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-16T16:36:45.666 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-16T16:36:45.666 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-16T16:36:45.666 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-16T16:36:45.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-16T16:36:45.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-16T16:36:45.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-16T16:36:45.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-16T16:36:45.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-16T16:36:45.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-16T16:36:45.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-16T16:36:45.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-16T16:36:45.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-16T16:36:45.667 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-16T16:36:45.668 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-16T16:36:45.668 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-16T16:36:45.668 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-16T16:36:45.668 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-16T16:36:45.668 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-16T16:36:45.668 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-16T16:36:45.668 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-16T16:36:45.668 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-16T16:36:45.668 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-16T16:36:45.668 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-16T16:36:45.669 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-16T16:36:45.669 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-16T16:36:45.669 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-16T16:36:45.669 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-16T16:36:45.669 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-16T16:36:45.669 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-16T16:36:45.669 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-16T16:36:45.669 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T16:36:45.669 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-16T16:36:45.670 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T16:36:45.670 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-16T16:36:45.670 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-16T16:36:45.670 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-16T16:36:45.670 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-16T16:36:45.670 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-16T16:36:45.670 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:36:45.670 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 129/134 2023-11-16T16:36:45.670 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-16T16:36:45.670 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 131/134 2023-11-16T16:36:45.671 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:36:45.671 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-16T16:36:45.735 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 100/134 2023-11-16T16:36:45.777 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 100/134 2023-11-16T16:36:45.813 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 101/134 2023-11-16T16:36:46.057 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 101/134 2023-11-16T16:36:46.172 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.e 102/134 2023-11-16T16:36:46.389 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a. 103/134 2023-11-16T16:36:46.608 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 104/134 2023-11-16T16:36:46.822 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 104/134 2023-11-16T16:36:47.095 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 105/134 2023-11-16T16:36:48.058 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 105/134 2023-11-16T16:36:48.249 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 106/134 2023-11-16T16:36:48.285 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:36:48.285 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:01 ago on Thu 16 Nov 2023 04:34:47 PM UTC. 2023-11-16T16:36:48.340 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 106/134 2023-11-16T16:36:48.611 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 101/134 2023-11-16T16:36:48.749 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el 107/134 2023-11-16T16:36:48.817 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 101/134 2023-11-16T16:36:48.925 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.e 102/134 2023-11-16T16:36:49.051 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 108/134 2023-11-16T16:36:49.085 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 108/134 2023-11-16T16:36:49.105 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a. 103/134 2023-11-16T16:36:49.258 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 109/134 2023-11-16T16:36:49.269 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 109/134 2023-11-16T16:36:49.309 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 104/134 2023-11-16T16:36:49.571 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 104/134 2023-11-16T16:36:49.789 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 105/134 2023-11-16T16:36:50.559 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 110/134 2023-11-16T16:36:50.796 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 111/134 2023-11-16T16:36:50.829 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 111/134 2023-11-16T16:36:50.880 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 105/134 2023-11-16T16:36:51.007 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 112/134 2023-11-16T16:36:51.034 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 106/134 2023-11-16T16:36:51.062 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 112/134 2023-11-16T16:36:51.163 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 106/134 2023-11-16T16:36:51.224 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 113/134 2023-11-16T16:36:51.569 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 113/134 2023-11-16T16:36:51.578 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el 107/134 2023-11-16T16:36:51.741 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 114/134 2023-11-16T16:36:51.822 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 108/134 2023-11-16T16:36:51.888 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 108/134 2023-11-16T16:36:52.059 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 109/134 2023-11-16T16:36:52.087 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 109/134 2023-11-16T16:36:53.208 INFO:teuthology.orchestra.run.smithi125.stdout:Installed products updated. 2023-11-16T16:36:53.422 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:36:53.422 INFO:teuthology.orchestra.run.smithi125.stdout:Upgraded: 2023-11-16T16:36:53.422 INFO:teuthology.orchestra.run.smithi125.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-16T16:36:53.422 INFO:teuthology.orchestra.run.smithi125.stdout: librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.422 INFO:teuthology.orchestra.run.smithi125.stdout: librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.422 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-11-16T16:36:53.422 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-16T16:36:53.422 INFO:teuthology.orchestra.run.smithi125.stdout: automake-1.16.1-7.el8.noarch 2023-11-16T16:36:53.423 INFO:teuthology.orchestra.run.smithi125.stdout: bison-3.0.4-10.el8.x86_64 2023-11-16T16:36:53.423 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.423 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.423 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.423 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.423 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:36:53.423 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-immutable-object-cache-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.423 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.423 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.423 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:36:53.424 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:36:53.424 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:36:53.424 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:36:53.424 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:36:53.424 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.424 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.424 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:36:53.424 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.424 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.424 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.425 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:36:53.425 INFO:teuthology.orchestra.run.smithi125.stdout: cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:36:53.425 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.425 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:36:53.425 INFO:teuthology.orchestra.run.smithi125.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-16T16:36:53.425 INFO:teuthology.orchestra.run.smithi125.stdout: flex-2.6.1-9.el8.x86_64 2023-11-16T16:36:53.425 INFO:teuthology.orchestra.run.smithi125.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-16T16:36:53.425 INFO:teuthology.orchestra.run.smithi125.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-16T16:36:53.425 INFO:teuthology.orchestra.run.smithi125.stdout: jq-1.6-3.el8.x86_64 2023-11-16T16:36:53.425 INFO:teuthology.orchestra.run.smithi125.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-16T16:36:53.426 INFO:teuthology.orchestra.run.smithi125.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-16T16:36:53.426 INFO:teuthology.orchestra.run.smithi125.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-16T16:36:53.426 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.426 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.426 INFO:teuthology.orchestra.run.smithi125.stdout: libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.426 INFO:teuthology.orchestra.run.smithi125.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-16T16:36:53.426 INFO:teuthology.orchestra.run.smithi125.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-16T16:36:53.426 INFO:teuthology.orchestra.run.smithi125.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-16T16:36:53.426 INFO:teuthology.orchestra.run.smithi125.stdout: libpq-13.5-1.el8.x86_64 2023-11-16T16:36:53.427 INFO:teuthology.orchestra.run.smithi125.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-16T16:36:53.427 INFO:teuthology.orchestra.run.smithi125.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-16T16:36:53.427 INFO:teuthology.orchestra.run.smithi125.stdout: librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.427 INFO:teuthology.orchestra.run.smithi125.stdout: libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.427 INFO:teuthology.orchestra.run.smithi125.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-16T16:36:53.427 INFO:teuthology.orchestra.run.smithi125.stdout: librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.427 INFO:teuthology.orchestra.run.smithi125.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-16T16:36:53.427 INFO:teuthology.orchestra.run.smithi125.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-16T16:36:53.427 INFO:teuthology.orchestra.run.smithi125.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-16T16:36:53.427 INFO:teuthology.orchestra.run.smithi125.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-16T16:36:53.428 INFO:teuthology.orchestra.run.smithi125.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-16T16:36:53.428 INFO:teuthology.orchestra.run.smithi125.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-16T16:36:53.428 INFO:teuthology.orchestra.run.smithi125.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-16T16:36:53.428 INFO:teuthology.orchestra.run.smithi125.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-16T16:36:53.428 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-16T16:36:53.428 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-16T16:36:53.428 INFO:teuthology.orchestra.run.smithi125.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-16T16:36:53.428 INFO:teuthology.orchestra.run.smithi125.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-16T16:36:53.428 INFO:teuthology.orchestra.run.smithi125.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-16T16:36:53.428 INFO:teuthology.orchestra.run.smithi125.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-16T16:36:53.429 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:36:53.429 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:36:53.429 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:36:53.429 INFO:teuthology.orchestra.run.smithi125.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-16T16:36:53.429 INFO:teuthology.orchestra.run.smithi125.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-16T16:36:53.429 INFO:teuthology.orchestra.run.smithi125.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-16T16:36:53.429 INFO:teuthology.orchestra.run.smithi125.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-16T16:36:53.429 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-16T16:36:53.429 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.429 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.430 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.430 INFO:teuthology.orchestra.run.smithi125.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-16T16:36:53.430 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-16T16:36:53.430 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-16T16:36:53.430 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-16T16:36:53.430 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-16T16:36:53.430 INFO:teuthology.orchestra.run.smithi125.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-16T16:36:53.430 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-16T16:36:53.430 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-16T16:36:53.430 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-16T16:36:53.431 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-16T16:36:53.431 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-16T16:36:53.431 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-16T16:36:53.431 INFO:teuthology.orchestra.run.smithi125.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-16T16:36:53.431 INFO:teuthology.orchestra.run.smithi125.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-16T16:36:53.431 INFO:teuthology.orchestra.run.smithi125.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-16T16:36:53.431 INFO:teuthology.orchestra.run.smithi125.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-16T16:36:53.431 INFO:teuthology.orchestra.run.smithi125.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-16T16:36:53.431 INFO:teuthology.orchestra.run.smithi125.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-16T16:36:53.431 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-16T16:36:53.432 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-16T16:36:53.432 INFO:teuthology.orchestra.run.smithi125.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-16T16:36:53.432 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-16T16:36:53.432 INFO:teuthology.orchestra.run.smithi125.stdout: python3-portend-2.6-1.el8.noarch 2023-11-16T16:36:53.432 INFO:teuthology.orchestra.run.smithi125.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-16T16:36:53.432 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-16T16:36:53.432 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-16T16:36:53.432 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-16T16:36:53.432 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-16T16:36:53.432 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-16T16:36:53.433 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.433 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.433 INFO:teuthology.orchestra.run.smithi125.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-16T16:36:53.433 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-16T16:36:53.433 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.433 INFO:teuthology.orchestra.run.smithi125.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-16T16:36:53.433 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-16T16:36:53.433 INFO:teuthology.orchestra.run.smithi125.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-16T16:36:53.433 INFO:teuthology.orchestra.run.smithi125.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-16T16:36:53.433 INFO:teuthology.orchestra.run.smithi125.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-16T16:36:53.434 INFO:teuthology.orchestra.run.smithi125.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-16T16:36:53.434 INFO:teuthology.orchestra.run.smithi125.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-16T16:36:53.434 INFO:teuthology.orchestra.run.smithi125.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-16T16:36:53.434 INFO:teuthology.orchestra.run.smithi125.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-16T16:36:53.434 INFO:teuthology.orchestra.run.smithi125.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-16T16:36:53.434 INFO:teuthology.orchestra.run.smithi125.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-16T16:36:53.434 INFO:teuthology.orchestra.run.smithi125.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-16T16:36:53.434 INFO:teuthology.orchestra.run.smithi125.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-16T16:36:53.434 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.435 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.435 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:36:53.435 INFO:teuthology.orchestra.run.smithi125.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-16T16:36:53.435 INFO:teuthology.orchestra.run.smithi125.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-16T16:36:53.435 INFO:teuthology.orchestra.run.smithi125.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-16T16:36:53.435 INFO:teuthology.orchestra.run.smithi125.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-16T16:36:53.435 INFO:teuthology.orchestra.run.smithi125.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-16T16:36:53.435 INFO:teuthology.orchestra.run.smithi125.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-16T16:36:53.436 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-11-16T16:36:53.436 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-11-16T16:36:53.437 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 110/134 2023-11-16T16:36:53.632 DEBUG:teuthology.parallel:result is None 2023-11-16T16:36:53.660 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 111/134 2023-11-16T16:36:53.722 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 111/134 2023-11-16T16:36:53.884 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 112/134 2023-11-16T16:36:53.959 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 112/134 2023-11-16T16:36:54.109 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 113/134 2023-11-16T16:36:54.513 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 113/134 2023-11-16T16:36:54.709 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 114/134 2023-11-16T16:36:55.648 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 114/134 2023-11-16T16:36:55.950 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 115/134 2023-11-16T16:36:56.132 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T16:36:56.328 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T16:36:57.251 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-16T16:36:57.563 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-16T16:36:57.793 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 119/134 2023-11-16T16:36:57.969 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T16:36:58.177 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_ 121/134 2023-11-16T16:36:58.324 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T16:36:58.951 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 114/134 2023-11-16T16:36:59.260 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 115/134 2023-11-16T16:36:59.416 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T16:36:59.672 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T16:37:00.559 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:37:00.814 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:37:00.831 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:37:00.871 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-16T16:37:01.048 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 124/134 2023-11-16T16:37:01.163 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-16T16:37:01.355 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 119/134 2023-11-16T16:37:01.388 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 125/134 2023-11-16T16:37:01.588 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T16:37:01.699 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 126/134 2023-11-16T16:37:01.771 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_ 121/134 2023-11-16T16:37:01.909 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T16:37:02.083 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-16T16:37:02.256 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:37:02.264 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:37:02.454 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-16T16:37:02.672 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T16:37:02.757 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T16:37:02.978 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T16:37:02.991 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T16:37:02.991 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:37:03.210 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:37:03.210 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T16:37:03.496 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T16:37:03.696 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:37:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:37:04.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 1/134 2023-11-16T16:37:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 2/134 2023-11-16T16:37:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 3/134 2023-11-16T16:37:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 4/134 2023-11-16T16:37:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 5/134 2023-11-16T16:37:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:37:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:37:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:37:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 9/134 2023-11-16T16:37:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 10/134 2023-11-16T16:37:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 11/134 2023-11-16T16:37:04.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 12/134 2023-11-16T16:37:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 13/134 2023-11-16T16:37:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_ 14/134 2023-11-16T16:37:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 15/134 2023-11-16T16:37:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 16/134 2023-11-16T16:37:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_6 17/134 2023-11-16T16:37:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 18/134 2023-11-16T16:37:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 19/134 2023-11-16T16:37:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el 20/134 2023-11-16T16:37:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8. 21/134 2023-11-16T16:37:04.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_6 22/134 2023-11-16T16:37:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 23/134 2023-11-16T16:37:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 24/134 2023-11-16T16:37:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 25/134 2023-11-16T16:37:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 26/134 2023-11-16T16:37:04.271 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 27/134 2023-11-16T16:37:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 28/134 2023-11-16T16:37:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a. 29/134 2023-11-16T16:37:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 30/134 2023-11-16T16:37:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 31/134 2023-11-16T16:37:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 32/134 2023-11-16T16:37:04.280 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el 33/134 2023-11-16T16:37:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 34/134 2023-11-16T16:37:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.e 35/134 2023-11-16T16:37:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 36/134 2023-11-16T16:37:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 37/134 2023-11-16T16:37:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 38/134 2023-11-16T16:37:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-16T16:37:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-16T16:37:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-16T16:37:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-16T16:37:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-16T16:37:04.281 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-16T16:37:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-16T16:37:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-16T16:37:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-16T16:37:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-16T16:37:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-16T16:37:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-16T16:37:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-16T16:37:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-16T16:37:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-16T16:37:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-16T16:37:04.282 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-16T16:37:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-16T16:37:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-16T16:37:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-16T16:37:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-16T16:37:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-16T16:37:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-16T16:37:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-16T16:37:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-16T16:37:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-16T16:37:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-16T16:37:04.283 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-16T16:37:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-16T16:37:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-16T16:37:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-16T16:37:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-16T16:37:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-16T16:37:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-16T16:37:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-16T16:37:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-16T16:37:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-16T16:37:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-16T16:37:04.284 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-16T16:37:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-16T16:37:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-16T16:37:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-16T16:37:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-16T16:37:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-16T16:37:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-16T16:37:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-16T16:37:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-16T16:37:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-16T16:37:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-16T16:37:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-16T16:37:04.285 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-16T16:37:04.286 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-16T16:37:04.286 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-16T16:37:04.286 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-16T16:37:04.286 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-16T16:37:04.286 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-16T16:37:04.286 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-16T16:37:04.286 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-16T16:37:04.286 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-16T16:37:04.286 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-16T16:37:04.286 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-16T16:37:04.286 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-16T16:37:04.287 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-16T16:37:04.287 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-16T16:37:04.287 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-16T16:37:04.287 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-16T16:37:04.287 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-16T16:37:04.287 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-16T16:37:04.287 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-16T16:37:04.287 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-16T16:37:04.287 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-16T16:37:04.287 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-16T16:37:04.287 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-16T16:37:04.288 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-16T16:37:04.288 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-16T16:37:04.288 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-16T16:37:04.288 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-16T16:37:04.288 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-16T16:37:04.288 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-16T16:37:04.288 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-16T16:37:04.288 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-16T16:37:04.288 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T16:37:04.288 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-16T16:37:04.289 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T16:37:04.289 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-16T16:37:04.289 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-16T16:37:04.289 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-16T16:37:04.289 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-16T16:37:04.289 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-16T16:37:04.289 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:37:04.289 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 129/134 2023-11-16T16:37:04.289 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-16T16:37:04.289 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 131/134 2023-11-16T16:37:04.290 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:37:04.290 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-16T16:37:05.337 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:37:05.599 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:37:05.637 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:37:05.917 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 124/134 2023-11-16T16:37:06.149 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 125/134 2023-11-16T16:37:06.466 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 126/134 2023-11-16T16:37:06.792 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-16T16:37:06.880 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:37:06.881 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:02:18 ago on Thu 16 Nov 2023 04:34:48 PM UTC. 2023-11-16T16:37:07.007 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:37:07.030 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:37:07.234 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-16T16:37:07.456 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T16:37:07.568 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T16:37:07.798 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T16:37:07.827 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T16:37:07.827 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:37:08.046 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:37:08.047 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T16:37:08.354 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T16:37:08.540 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:37:08.828 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:37:08.828 INFO:teuthology.orchestra.run.smithi093.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-16T16:37:08.828 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:37:09.440 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 1/134 2023-11-16T16:37:09.440 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 2/134 2023-11-16T16:37:09.440 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 3/134 2023-11-16T16:37:09.440 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 4/134 2023-11-16T16:37:09.440 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 5/134 2023-11-16T16:37:09.440 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:37:09.440 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:37:09.441 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:37:09.441 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 9/134 2023-11-16T16:37:09.441 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 10/134 2023-11-16T16:37:09.441 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 11/134 2023-11-16T16:37:09.441 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 12/134 2023-11-16T16:37:09.441 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 13/134 2023-11-16T16:37:09.441 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_ 14/134 2023-11-16T16:37:09.441 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 15/134 2023-11-16T16:37:09.441 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 16/134 2023-11-16T16:37:09.441 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_6 17/134 2023-11-16T16:37:09.442 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 18/134 2023-11-16T16:37:09.442 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 19/134 2023-11-16T16:37:09.442 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el 20/134 2023-11-16T16:37:09.442 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8. 21/134 2023-11-16T16:37:09.442 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_6 22/134 2023-11-16T16:37:09.442 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 23/134 2023-11-16T16:37:09.442 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 24/134 2023-11-16T16:37:09.442 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 25/134 2023-11-16T16:37:09.442 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 26/134 2023-11-16T16:37:09.442 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 27/134 2023-11-16T16:37:09.451 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 28/134 2023-11-16T16:37:09.451 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a. 29/134 2023-11-16T16:37:09.451 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 30/134 2023-11-16T16:37:09.451 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 31/134 2023-11-16T16:37:09.451 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 32/134 2023-11-16T16:37:09.452 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el 33/134 2023-11-16T16:37:09.452 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 34/134 2023-11-16T16:37:09.452 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.e 35/134 2023-11-16T16:37:09.452 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 36/134 2023-11-16T16:37:09.452 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 37/134 2023-11-16T16:37:09.452 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 38/134 2023-11-16T16:37:09.452 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-16T16:37:09.452 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-16T16:37:09.452 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-16T16:37:09.452 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-16T16:37:09.452 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-16T16:37:09.452 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-16T16:37:09.453 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-16T16:37:09.454 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-16T16:37:09.455 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-16T16:37:09.456 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-16T16:37:09.456 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-16T16:37:09.456 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-16T16:37:09.456 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-16T16:37:09.456 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-16T16:37:09.456 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-16T16:37:09.456 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-16T16:37:09.456 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-16T16:37:09.456 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-16T16:37:09.456 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-16T16:37:09.456 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-16T16:37:09.456 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-16T16:37:09.457 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-16T16:37:09.458 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T16:37:09.459 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-16T16:37:09.459 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-16T16:37:09.459 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-16T16:37:09.459 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-16T16:37:09.459 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-16T16:37:09.459 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:37:09.459 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 129/134 2023-11-16T16:37:09.459 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-16T16:37:09.459 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 131/134 2023-11-16T16:37:09.459 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:37:09.459 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-16T16:37:12.237 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:37:12.237 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:02:23 ago on Thu 16 Nov 2023 04:34:49 PM UTC. 2023-11-16T16:37:15.378 INFO:teuthology.orchestra.run.smithi159.stdout:Installed products updated. 2023-11-16T16:37:15.596 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:37:15.596 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2023-11-16T16:37:15.596 INFO:teuthology.orchestra.run.smithi159.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-16T16:37:15.597 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.597 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.597 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-11-16T16:37:15.597 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-16T16:37:15.597 INFO:teuthology.orchestra.run.smithi159.stdout: automake-1.16.1-7.el8.noarch 2023-11-16T16:37:15.597 INFO:teuthology.orchestra.run.smithi159.stdout: bison-3.0.4-10.el8.x86_64 2023-11-16T16:37:15.598 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.598 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.598 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.598 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.598 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:15.598 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.598 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.598 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-16T16:37:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: flex-2.6.1-9.el8.x86_64 2023-11-16T16:37:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-16T16:37:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-16T16:37:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: jq-1.6-3.el8.x86_64 2023-11-16T16:37:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-16T16:37:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-16T16:37:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-16T16:37:15.602 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.602 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.602 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.602 INFO:teuthology.orchestra.run.smithi159.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-16T16:37:15.602 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-16T16:37:15.602 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-16T16:37:15.602 INFO:teuthology.orchestra.run.smithi159.stdout: libpq-13.5-1.el8.x86_64 2023-11-16T16:37:15.603 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-16T16:37:15.603 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-16T16:37:15.603 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.603 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.603 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-16T16:37:15.603 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.603 INFO:teuthology.orchestra.run.smithi159.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-16T16:37:15.603 INFO:teuthology.orchestra.run.smithi159.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-16T16:37:15.604 INFO:teuthology.orchestra.run.smithi159.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-16T16:37:15.604 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-16T16:37:15.604 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-16T16:37:15.604 INFO:teuthology.orchestra.run.smithi159.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-16T16:37:15.604 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-16T16:37:15.604 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-16T16:37:15.604 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-16T16:37:15.604 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-16T16:37:15.604 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-16T16:37:15.605 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-16T16:37:15.605 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-16T16:37:15.605 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-16T16:37:15.605 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:37:15.605 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:37:15.605 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:37:15.605 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-16T16:37:15.606 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-16T16:37:15.606 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-16T16:37:15.606 INFO:teuthology.orchestra.run.smithi159.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-16T16:37:15.606 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-16T16:37:15.606 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.606 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.606 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.607 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-16T16:37:15.607 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-16T16:37:15.607 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-16T16:37:15.607 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-16T16:37:15.607 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-16T16:37:15.607 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-16T16:37:15.608 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-16T16:37:15.608 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-16T16:37:15.608 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-16T16:37:15.608 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-16T16:37:15.608 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-16T16:37:15.608 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-16T16:37:15.608 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-16T16:37:15.608 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-16T16:37:15.608 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-16T16:37:15.609 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-16T16:37:15.609 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-16T16:37:15.609 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-16T16:37:15.609 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-16T16:37:15.609 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-16T16:37:15.609 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-16T16:37:15.609 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-16T16:37:15.610 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-2.6-1.el8.noarch 2023-11-16T16:37:15.610 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-16T16:37:15.610 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-16T16:37:15.610 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-16T16:37:15.610 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-16T16:37:15.611 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-16T16:37:15.611 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-16T16:37:15.611 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.611 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.611 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-16T16:37:15.611 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-16T16:37:15.611 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.612 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-16T16:37:15.612 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-16T16:37:15.612 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-16T16:37:15.612 INFO:teuthology.orchestra.run.smithi159.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-16T16:37:15.612 INFO:teuthology.orchestra.run.smithi159.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-16T16:37:15.612 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-16T16:37:15.612 INFO:teuthology.orchestra.run.smithi159.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-16T16:37:15.612 INFO:teuthology.orchestra.run.smithi159.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-16T16:37:15.613 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-16T16:37:15.613 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-16T16:37:15.613 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-16T16:37:15.613 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-16T16:37:15.613 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-16T16:37:15.613 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.613 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.613 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:15.613 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-16T16:37:15.614 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-16T16:37:15.614 INFO:teuthology.orchestra.run.smithi159.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-16T16:37:15.614 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-16T16:37:15.614 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-16T16:37:15.614 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-16T16:37:15.615 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:37:15.615 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-11-16T16:37:17.195 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:19.221 INFO:teuthology.orchestra.run.smithi093.stdout:Installed products updated. 2023-11-16T16:37:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:37:19.445 INFO:teuthology.orchestra.run.smithi093.stdout:Upgraded: 2023-11-16T16:37:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-16T16:37:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.445 INFO:teuthology.orchestra.run.smithi093.stdout:Installed: 2023-11-16T16:37:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-16T16:37:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: automake-1.16.1-7.el8.noarch 2023-11-16T16:37:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: bison-3.0.4-10.el8.x86_64 2023-11-16T16:37:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-immutable-object-cache-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:37:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-16T16:37:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: flex-2.6.1-9.el8.x86_64 2023-11-16T16:37:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-16T16:37:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-16T16:37:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: jq-1.6-3.el8.x86_64 2023-11-16T16:37:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-16T16:37:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-16T16:37:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-16T16:37:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-16T16:37:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: libpq-13.5-1.el8.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-16T16:37:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-16T16:37:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-16T16:37:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-16T16:37:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-16T16:37:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-16T16:37:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-16T16:37:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-16T16:37:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-16T16:37:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-16T16:37:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-portend-2.6-1.el8.noarch 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-16T16:37:19.455 INFO:teuthology.orchestra.run.smithi093.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-16T16:37:19.455 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-16T16:37:19.455 INFO:teuthology.orchestra.run.smithi093.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-16T16:37:19.455 INFO:teuthology.orchestra.run.smithi093.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-16T16:37:19.455 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.455 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.455 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:37:19.455 INFO:teuthology.orchestra.run.smithi093.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-16T16:37:19.455 INFO:teuthology.orchestra.run.smithi093.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-16T16:37:19.455 INFO:teuthology.orchestra.run.smithi093.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-16T16:37:19.455 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-16T16:37:19.455 INFO:teuthology.orchestra.run.smithi093.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-16T16:37:19.456 INFO:teuthology.orchestra.run.smithi093.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-16T16:37:19.456 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:37:19.456 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-11-16T16:37:19.733 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:19.734 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:37:20.004 DEBUG:teuthology.orchestra.run.smithi093:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-16T16:37:20.057 INFO:teuthology.orchestra.run.smithi093.stdout:18.0.0-3905.g5e4a743a.el8 2023-11-16T16:37:20.058 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3905.g5e4a743a.el8 2023-11-16T16:37:20.059 INFO:teuthology.task.install:The correct ceph version 18.0.0-3905.g5e4a743a is installed. 2023-11-16T16:37:20.059 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:37:20.281 DEBUG:teuthology.orchestra.run.smithi125:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-16T16:37:20.310 INFO:teuthology.orchestra.run.smithi125.stdout:18.0.0-3905.g5e4a743a.el8 2023-11-16T16:37:20.310 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3905.g5e4a743a.el8 2023-11-16T16:37:20.310 INFO:teuthology.task.install:The correct ceph version 18.0.0-3905.g5e4a743a is installed. 2023-11-16T16:37:20.310 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:37:20.460 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-16T16:37:20.488 INFO:teuthology.orchestra.run.smithi159.stdout:18.0.0-3905.g5e4a743a.el8 2023-11-16T16:37:20.488 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3905.g5e4a743a.el8 2023-11-16T16:37:20.488 INFO:teuthology.task.install:The correct ceph version 18.0.0-3905.g5e4a743a is installed. 2023-11-16T16:37:20.489 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-16T16:37:20.489 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:37:20.489 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-16T16:37:20.579 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-11-16T16:37:20.579 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-16T16:37:20.627 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-11-16T16:37:20.627 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-16T16:37:20.672 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-16T16:37:20.672 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:37:20.672 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/daemon-helper 2023-11-16T16:37:20.740 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-16T16:37:20.847 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-11-16T16:37:20.847 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/daemon-helper 2023-11-16T16:37:20.881 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-16T16:37:20.953 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-11-16T16:37:20.953 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2023-11-16T16:37:20.986 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-16T16:37:21.058 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-16T16:37:21.058 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:37:21.058 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-16T16:37:21.125 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-16T16:37:21.235 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-11-16T16:37:21.235 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-16T16:37:21.267 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-16T16:37:21.339 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-11-16T16:37:21.339 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-16T16:37:21.372 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-16T16:37:21.443 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-16T16:37:21.443 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:37:21.443 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/stdin-killer 2023-11-16T16:37:21.513 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-16T16:37:21.621 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-11-16T16:37:21.621 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/stdin-killer 2023-11-16T16:37:21.653 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-16T16:37:21.725 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-11-16T16:37:21.725 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2023-11-16T16:37:21.757 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-16T16:37:21.827 INFO:teuthology.run_tasks:Running task cephadm... 2023-11-16T16:37:21.902 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-11-16T16:37:21.902 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:37:21.903 INFO:tasks.cephadm:Cluster fsid is 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:37:21.903 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-11-16T16:37:21.903 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.93', 'mon.b': '172.21.15.125', 'mon.c': '172.21.15.159'} 2023-11-16T16:37:21.903 INFO:tasks.cephadm:First mon is mon.a on smithi093 2023-11-16T16:37:21.903 INFO:tasks.cephadm:First mgr is x 2023-11-16T16:37:21.903 INFO:tasks.cephadm:Normalizing hostnames... 2023-11-16T16:37:21.903 DEBUG:teuthology.orchestra.run.smithi093:> sudo hostname $(hostname -s) 2023-11-16T16:37:21.973 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname $(hostname -s) 2023-11-16T16:37:22.006 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname $(hostname -s) 2023-11-16T16:37:22.038 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-11-16T16:37:22.039 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:37:22.179 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74186/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-3905-g5e4a743a', 'node_name': '172.21.5.32+adami02', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-3905.g5e4a743a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-11-16 00:50:05.467524', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-choffman-62793', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-11-16T16:37:22.314 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-choffman-62793, sha1 5e4a743ad7fd1422ebd6c4dc950a21a303279207 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:37:22.314 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/x86_64/flavors/default/cephadm 2023-11-16T16:37:22.315 DEBUG:teuthology.orchestra.run.smithi093:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-16T16:37:22.734 INFO:teuthology.orchestra.run.smithi093.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 16 16:37 /home/ubuntu/cephtest/cephadm 2023-11-16T16:37:22.735 DEBUG:teuthology.orchestra.run.smithi125:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-16T16:37:23.327 INFO:teuthology.orchestra.run.smithi125.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 16 16:37 /home/ubuntu/cephtest/cephadm 2023-11-16T16:37:23.327 DEBUG:teuthology.orchestra.run.smithi159:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-16T16:37:23.967 INFO:teuthology.orchestra.run.smithi159.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 16 16:37 /home/ubuntu/cephtest/cephadm 2023-11-16T16:37:23.967 DEBUG:teuthology.orchestra.run.smithi093:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-16T16:37:24.018 DEBUG:teuthology.orchestra.run.smithi125:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-16T16:37:24.040 DEBUG:teuthology.orchestra.run.smithi159:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-16T16:37:24.075 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 on all hosts... 2023-11-16T16:37:24.075 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 pull 2023-11-16T16:37:24.080 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 pull 2023-11-16T16:37:24.082 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 pull 2023-11-16T16:37:25.594 INFO:teuthology.orchestra.run.smithi125.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207... 2023-11-16T16:37:27.348 INFO:teuthology.orchestra.run.smithi159.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207... 2023-11-16T16:37:28.238 INFO:teuthology.orchestra.run.smithi093.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207... 2023-11-16T16:37:59.308 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:37:59.308 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_version": "ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev)", 2023-11-16T16:37:59.308 INFO:teuthology.orchestra.run.smithi093.stdout: "image_id": "d5b1f852d682af5926771a6e4760786b6147f99518b76fd91b07a1b9e99bfe99", 2023-11-16T16:37:59.308 INFO:teuthology.orchestra.run.smithi093.stdout: "repo_digests": [ 2023-11-16T16:37:59.308 INFO:teuthology.orchestra.run.smithi093.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:209515a30c262c9be4300b2075ceb5fd8912e5a4eccd3c9021b5585705456b42" 2023-11-16T16:37:59.308 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-11-16T16:37:59.308 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:38:36.092 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-11-16T16:38:36.092 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev)", 2023-11-16T16:38:36.092 INFO:teuthology.orchestra.run.smithi125.stdout: "image_id": "d5b1f852d682af5926771a6e4760786b6147f99518b76fd91b07a1b9e99bfe99", 2023-11-16T16:38:36.092 INFO:teuthology.orchestra.run.smithi125.stdout: "repo_digests": [ 2023-11-16T16:38:36.092 INFO:teuthology.orchestra.run.smithi125.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:209515a30c262c9be4300b2075ceb5fd8912e5a4eccd3c9021b5585705456b42" 2023-11-16T16:38:36.093 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-11-16T16:38:36.093 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-11-16T16:38:45.406 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-11-16T16:38:45.407 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_version": "ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev)", 2023-11-16T16:38:45.407 INFO:teuthology.orchestra.run.smithi159.stdout: "image_id": "d5b1f852d682af5926771a6e4760786b6147f99518b76fd91b07a1b9e99bfe99", 2023-11-16T16:38:45.407 INFO:teuthology.orchestra.run.smithi159.stdout: "repo_digests": [ 2023-11-16T16:38:45.407 INFO:teuthology.orchestra.run.smithi159.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:209515a30c262c9be4300b2075ceb5fd8912e5a4eccd3c9021b5585705456b42" 2023-11-16T16:38:45.407 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2023-11-16T16:38:45.407 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-11-16T16:38:45.427 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph 2023-11-16T16:38:45.506 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /etc/ceph 2023-11-16T16:38:45.544 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph 2023-11-16T16:38:45.573 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 777 /etc/ceph 2023-11-16T16:38:45.642 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 777 /etc/ceph 2023-11-16T16:38:45.675 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /etc/ceph 2023-11-16T16:38:45.701 INFO:tasks.cephadm:Writing seed config... 2023-11-16T16:38:45.702 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-11-16T16:38:45.702 INFO:tasks.cephadm: override: [client] debug client = 20 2023-11-16T16:38:45.702 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-11-16T16:38:45.702 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-11-16T16:38:45.702 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-11-16T16:38:45.702 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-11-16T16:38:45.702 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-11-16T16:38:45.703 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-11-16T16:38:45.704 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-11-16T16:38:45.705 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-11-16T16:38:45.705 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-11-16T16:38:45.705 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-11-16T16:38:45.705 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-11-16T16:38:45.705 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-11-16T16:38:45.705 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-11-16T16:38:45.705 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-11-16T16:38:45.705 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-11-16T16:38:45.706 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:38:45.706 DEBUG:teuthology.orchestra.run.smithi093:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-11-16T16:38:45.749 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-11-16T16:38:45.749 DEBUG:teuthology.orchestra.run.smithi093:mon.a> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.a.service 2023-11-16T16:38:45.793 DEBUG:teuthology.orchestra.run.smithi093:mgr.x> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.x.service 2023-11-16T16:38:45.838 INFO:tasks.cephadm:Bootstrapping... 2023-11-16T16:38:45.838 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 -v bootstrap --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.93 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:38:45.860 INFO:journalctl@ceph.mon.a.smithi093.stdout:-- Logs begin at Thu 2023-11-16 16:19:38 UTC. -- 2023-11-16T16:38:45.905 INFO:journalctl@ceph.mgr.x.smithi093.stdout:-- Logs begin at Thu 2023-11-16 16:19:38 UTC. -- 2023-11-16T16:38:46.175 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:38:46.175 INFO:teuthology.orchestra.run.smithi093.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207', '-v', 'bootstrap', '--fsid', '6a59c1a8-849e-11ee-8dbe-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.93', '--skip-admin-label'] 2023-11-16T16:38:46.350 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stdout 4.1.1 2023-11-16T16:38:46.350 INFO:teuthology.orchestra.run.smithi093.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-11-16T16:38:46.351 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying podman|docker is present... 2023-11-16T16:38:46.522 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stdout 4.1.1 2023-11-16T16:38:46.522 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying lvm2 is present... 2023-11-16T16:38:46.522 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying time synchronization is in place... 2023-11-16T16:38:46.536 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-16T16:38:46.537 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-16T16:38:46.549 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-16T16:38:46.549 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-11-16T16:38:46.563 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout enabled 2023-11-16T16:38:46.576 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout active 2023-11-16T16:38:46.576 INFO:teuthology.orchestra.run.smithi093.stdout:Unit chronyd.service is enabled and running 2023-11-16T16:38:46.576 INFO:teuthology.orchestra.run.smithi093.stdout:Repeating the final host check... 2023-11-16T16:38:46.747 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stdout 4.1.1 2023-11-16T16:38:46.747 INFO:teuthology.orchestra.run.smithi093.stdout:podman (/bin/podman) version 4.1.1 is present 2023-11-16T16:38:46.747 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl is present 2023-11-16T16:38:46.747 INFO:teuthology.orchestra.run.smithi093.stdout:lvcreate is present 2023-11-16T16:38:46.760 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-16T16:38:46.760 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-16T16:38:46.773 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-16T16:38:46.773 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-11-16T16:38:46.786 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout enabled 2023-11-16T16:38:46.799 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout active 2023-11-16T16:38:46.799 INFO:teuthology.orchestra.run.smithi093.stdout:Unit chronyd.service is enabled and running 2023-11-16T16:38:46.799 INFO:teuthology.orchestra.run.smithi093.stdout:Host looks OK 2023-11-16T16:38:46.799 INFO:teuthology.orchestra.run.smithi093.stdout:Cluster fsid: 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:38:46.799 INFO:teuthology.orchestra.run.smithi093.stdout:Acquiring lock 140274258520552 on /run/cephadm/6a59c1a8-849e-11ee-8dbe-212e2dc638e7.lock 2023-11-16T16:38:46.800 INFO:teuthology.orchestra.run.smithi093.stdout:Lock 140274258520552 acquired on /run/cephadm/6a59c1a8-849e-11ee-8dbe-212e2dc638e7.lock 2023-11-16T16:38:46.800 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying IP 172.21.15.93 port 3300 ... 2023-11-16T16:38:46.800 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying IP 172.21.15.93 port 6789 ... 2023-11-16T16:38:46.800 INFO:teuthology.orchestra.run.smithi093.stdout:Base mon IP(s) is [172.21.15.93:3300, 172.21.15.93:6789], mon addrv is [v2:172.21.15.93:3300,v1:172.21.15.93:6789] 2023-11-16T16:38:46.808 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-11-16T16:38:46.808 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.93 metric 100 2023-11-16T16:38:46.815 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-11-16T16:38:46.815 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-11-16T16:38:46.815 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-11-16T16:38:46.822 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-11-16T16:38:46.822 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-11-16T16:38:46.822 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-16T16:38:46.822 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-11-16T16:38:46.822 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout inet6 fe80::2b1e:922d:25d9:39b2/64 scope link noprefixroute 2023-11-16T16:38:46.822 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-16T16:38:46.823 INFO:teuthology.orchestra.run.smithi093.stdout:Mon IP `172.21.15.93` is in CIDR network `172.21.0.0/20` 2023-11-16T16:38:46.823 INFO:teuthology.orchestra.run.smithi093.stdout:Mon IP `172.21.15.93` is in CIDR network `172.21.0.0/20` 2023-11-16T16:38:46.823 INFO:teuthology.orchestra.run.smithi093.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-11-16T16:38:46.824 INFO:teuthology.orchestra.run.smithi093.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-11-16T16:38:46.825 INFO:teuthology.orchestra.run.smithi093.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207... 2023-11-16T16:38:47.499 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stdout d5b1f852d682af5926771a6e4760786b6147f99518b76fd91b07a1b9e99bfe99 2023-11-16T16:38:47.500 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207... 2023-11-16T16:38:47.500 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Getting image source signatures 2023-11-16T16:38:47.500 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Copying blob sha256:3a8f6e475435f9e3b5d92795b454ceb30085f6221d261249c26042723227dea7 2023-11-16T16:38:47.500 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Copying blob sha256:837e66e3ff89804ed1bb42786f80e6c23534ff65fe48526d5007449f4e1f79b0 2023-11-16T16:38:47.500 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Copying config sha256:d5b1f852d682af5926771a6e4760786b6147f99518b76fd91b07a1b9e99bfe99 2023-11-16T16:38:47.500 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Writing manifest to image destination 2023-11-16T16:38:47.500 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Storing signatures 2023-11-16T16:38:48.377 INFO:teuthology.orchestra.run.smithi093.stdout:ceph: stdout ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev) 2023-11-16T16:38:48.378 INFO:teuthology.orchestra.run.smithi093.stdout:Ceph version: ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev) 2023-11-16T16:38:48.378 INFO:teuthology.orchestra.run.smithi093.stdout:Extracting ceph user uid/gid from container image... 2023-11-16T16:38:49.156 INFO:teuthology.orchestra.run.smithi093.stdout:stat: stdout 167 167 2023-11-16T16:38:49.156 INFO:teuthology.orchestra.run.smithi093.stdout:Creating initial keys... 2023-11-16T16:38:49.931 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph-authtool: stdout AQCZRVZlXLGqIBAAALO/12pd3uAqFbfTq0jKPg== 2023-11-16T16:38:50.705 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph-authtool: stdout AQCaRVZlisy6ERAANBI+tzoCzg36rVKf3FoNYA== 2023-11-16T16:38:51.480 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph-authtool: stdout AQCbRVZlg9qxBhAADbmL5o7sXXa8Az9K7JT6fw== 2023-11-16T16:38:51.481 INFO:teuthology.orchestra.run.smithi093.stdout:Creating initial monmap... 2023-11-16T16:38:52.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-16T16:38:52.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-11-16T16:38:52.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:38:52.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-16T16:38:52.257 INFO:teuthology.orchestra.run.smithi093.stdout:monmaptool for a [v2:172.21.15.93:3300,v1:172.21.15.93:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-16T16:38:52.257 INFO:teuthology.orchestra.run.smithi093.stdout:setting min_mon_release = pacific 2023-11-16T16:38:52.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: set fsid to 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:38:52.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-16T16:38:52.257 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:38:52.257 INFO:teuthology.orchestra.run.smithi093.stdout:Creating mon... 2023-11-16T16:38:53.036 INFO:teuthology.orchestra.run.smithi093.stdout:create mon.a on 2023-11-16T16:38:53.248 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-11-16T16:38:53.445 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-11-16T16:38:53.652 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7.target → /etc/systemd/system/ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7.target. 2023-11-16T16:38:53.653 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7.target → /etc/systemd/system/ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7.target. 2023-11-16T16:38:53.946 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.a 2023-11-16T16:38:53.946 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.a.service: Unit ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.a.service not loaded. 2023-11-16T16:38:54.140 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7.target.wants/ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@.service. 2023-11-16T16:38:54.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:54 smithi093 systemd[1]: Starting Ceph mon.a for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:38:54.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:54 smithi093 podman[97322]: 2023-11-16T16:38:54.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:54 smithi093 bash[97322]: 2f413399dc4fe22f9c5c45b2b5ec8a2c42eff2bca88ede56a2390aa554eefae4 2023-11-16T16:38:54.948 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-16T16:38:54.948 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout disabled 2023-11-16T16:38:54.964 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-16T16:38:54.964 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-11-16T16:38:54.964 INFO:teuthology.orchestra.run.smithi093.stdout:firewalld.service is not enabled 2023-11-16T16:38:54.964 INFO:teuthology.orchestra.run.smithi093.stdout:Not possible to enable service . firewalld.service is not available 2023-11-16T16:38:54.965 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mon to start... 2023-11-16T16:38:54.965 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mon... 2023-11-16T16:38:55.262 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:54 smithi093 ceph-mon[97362]: mkfs 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:38:55.263 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:54 smithi093 ceph-mon[97362]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-16T16:38:55.263 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:54 smithi093 systemd[1]: Started Ceph mon.a for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:38:56.051 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout cluster: 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout id: 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout services: 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.69624s) 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout data: 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-11-16T16:38:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout pgs: 2023-11-16T16:38:56.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:38:56.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.516+0000 7fed04000700 1 Processor -- start 2023-11-16T16:38:56.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.516+0000 7fed04000700 1 -- start start 2023-11-16T16:38:56.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.516+0000 7fed04000700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 0x7fecfc07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.516+0000 7fed04000700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc07a820 con 0x7fecfc07bde0 2023-11-16T16:38:56.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.517+0000 7fed01d9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 0x7fecfc07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.517+0000 7fed01d9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 0x7fecfc07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47922/0 (socket says 172.21.15.93:47922) 2023-11-16T16:38:56.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.517+0000 7fed01d9c700 1 -- 172.21.15.93:0/3589720295 learned_addr learned my addr 172.21.15.93:0/3589720295 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:38:56.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.517+0000 7fed01d9c700 1 -- 172.21.15.93:0/3589720295 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc07a960 con 0x7fecfc07bde0 2023-11-16T16:38:56.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.518+0000 7fed01d9c700 1 --2- 172.21.15.93:0/3589720295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 0x7fecfc07a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fecf80098a0 tx=0x7fecf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c7bcbde95494ad server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:56.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.518+0000 7fed00d9a700 1 -- 172.21.15.93:0/3589720295 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecf8004160 con 0x7fecfc07bde0 2023-11-16T16:38:56.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.518+0000 7fed00d9a700 1 -- 172.21.15.93:0/3589720295 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf80042c0 con 0x7fecfc07bde0 2023-11-16T16:38:56.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.518+0000 7fed00d9a700 1 -- 172.21.15.93:0/3589720295 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecf80045a0 con 0x7fecfc07bde0 2023-11-16T16:38:56.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.518+0000 7fed04000700 1 -- 172.21.15.93:0/3589720295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 msgr2=0x7fecfc07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.518+0000 7fed04000700 1 --2- 172.21.15.93:0/3589720295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 0x7fecfc07a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fecf80098a0 tx=0x7fecf8009bb0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.518+0000 7fed04000700 1 -- 172.21.15.93:0/3589720295 shutdown_connections 2023-11-16T16:38:56.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.518+0000 7fed04000700 1 --2- 172.21.15.93:0/3589720295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 0x7fecfc07a2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.518+0000 7fed04000700 1 -- 172.21.15.93:0/3589720295 >> 172.21.15.93:0/3589720295 conn(0x7fecfc100fa0 msgr2=0x7fecfc1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:56.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.519+0000 7fed04000700 1 -- 172.21.15.93:0/3589720295 shutdown_connections 2023-11-16T16:38:56.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.519+0000 7fed04000700 1 -- 172.21.15.93:0/3589720295 wait complete. 2023-11-16T16:38:56.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.520+0000 7fed04000700 1 Processor -- start 2023-11-16T16:38:56.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.520+0000 7fed04000700 1 -- start start 2023-11-16T16:38:56.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.520+0000 7fed04000700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 0x7fecfc19afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.520+0000 7fed04000700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc19b520 con 0x7fecfc07bde0 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.520+0000 7fed01d9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 0x7fecfc19afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.521+0000 7fed01d9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 0x7fecfc19afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47930/0 (socket says 172.21.15.93:47930) 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.521+0000 7fed01d9c700 1 -- 172.21.15.93:0/2252027763 learned_addr learned my addr 172.21.15.93:0/2252027763 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.521+0000 7fed01d9c700 1 -- 172.21.15.93:0/2252027763 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf8009550 con 0x7fecfc07bde0 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.521+0000 7fed01d9c700 1 --2- 172.21.15.93:0/2252027763 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 0x7fecfc19afe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fecf8004000 tx=0x7fecf8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.521+0000 7fecf2ffd700 1 -- 172.21.15.93:0/2252027763 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecf8011630 con 0x7fecfc07bde0 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.521+0000 7fecf2ffd700 1 -- 172.21.15.93:0/2252027763 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf8011790 con 0x7fecfc07bde0 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.521+0000 7fed04000700 1 -- 172.21.15.93:0/2252027763 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecfc07ae60 con 0x7fecfc07bde0 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.521+0000 7fecf2ffd700 1 -- 172.21.15.93:0/2252027763 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecf8011a30 con 0x7fecfc07bde0 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.522+0000 7fed04000700 1 -- 172.21.15.93:0/2252027763 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecfc07acd0 con 0x7fecfc07bde0 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.522+0000 7fecf2ffd700 1 -- 172.21.15.93:0/2252027763 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fecf8022070 con 0x7fecfc07bde0 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.522+0000 7fecf2ffd700 1 -- 172.21.15.93:0/2252027763 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fecf801c450 con 0x7fecfc07bde0 2023-11-16T16:38:56.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.523+0000 7fed04000700 1 -- 172.21.15.93:0/2252027763 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecfc194e60 con 0x7fecfc07bde0 2023-11-16T16:38:56.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.526+0000 7fecf2ffd700 1 -- 172.21.15.93:0/2252027763 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fecf801b450 con 0x7fecfc07bde0 2023-11-16T16:38:56.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.597+0000 7fed04000700 1 -- 172.21.15.93:0/2252027763 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fecfc02cfe0 con 0x7fecfc07bde0 2023-11-16T16:38:56.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.597+0000 7fecf2ffd700 1 -- 172.21.15.93:0/2252027763 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+316 (secure 0 0 0) 0x7fecf801b5f0 con 0x7fecfc07bde0 2023-11-16T16:38:56.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.598+0000 7fed04000700 1 -- 172.21.15.93:0/2252027763 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 msgr2=0x7fecfc19afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.598+0000 7fed04000700 1 --2- 172.21.15.93:0/2252027763 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 0x7fecfc19afe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fecf8004000 tx=0x7fecf8004750 comp rx=0 tx=0).stop 2023-11-16T16:38:56.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.598+0000 7fed04000700 1 -- 172.21.15.93:0/2252027763 shutdown_connections 2023-11-16T16:38:56.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.598+0000 7fed04000700 1 --2- 172.21.15.93:0/2252027763 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecfc07bde0 0x7fecfc19afe0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.598+0000 7fed04000700 1 -- 172.21.15.93:0/2252027763 >> 172.21.15.93:0/2252027763 conn(0x7fecfc100fa0 msgr2=0x7fecfc102800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:56.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.598+0000 7fed04000700 1 -- 172.21.15.93:0/2252027763 shutdown_connections 2023-11-16T16:38:56.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:55.598+0000 7fed04000700 1 -- 172.21.15.93:0/2252027763 wait complete. 2023-11-16T16:38:56.056 INFO:teuthology.orchestra.run.smithi093.stdout:mon is available 2023-11-16T16:38:56.056 INFO:teuthology.orchestra.run.smithi093.stdout:Assimilating anything we can from ceph.conf... 2023-11-16T16:38:56.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:55 smithi093 ceph-mon[97362]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-16T16:38:56.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:55 smithi093 ceph-mon[97362]: monmap e1: 1 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0]} removed_ranks: {} 2023-11-16T16:38:56.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:55 smithi093 ceph-mon[97362]: fsmap 2023-11-16T16:38:56.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:55 smithi093 ceph-mon[97362]: osdmap e1: 0 total, 0 up, 0 in 2023-11-16T16:38:56.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:55 smithi093 ceph-mon[97362]: mgrmap e1: no daemons active 2023-11-16T16:38:56.317 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:55 smithi093 ceph-mon[97362]: from='client.? 172.21.15.93:0/2252027763' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-11-16T16:38:57.137 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:38:57.137 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [global] 2023-11-16T16:38:57.137 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout fsid = 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:38:57.137 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.93:3300,v1:172.21.15.93:6789] 2023-11-16T16:38:57.137 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-16T16:38:57.138 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-16T16:38:57.138 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-16T16:38:57.138 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-16T16:38:57.138 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:38:57.138 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-16T16:38:57.138 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-16T16:38:57.138 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:38:57.138 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [osd] 2023-11-16T16:38:57.138 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-16T16:38:57.138 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-16T16:38:57.138 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-16T16:38:57.138 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.563+0000 7f8f08273700 1 Processor -- start 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.564+0000 7f8f08273700 1 -- start start 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.564+0000 7f8f08273700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 0x7f8f0007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.564+0000 7f8f08273700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0007a820 con 0x7f8f0007bde0 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.565+0000 7f8f0600f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 0x7f8f0007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.565+0000 7f8f0600f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 0x7f8f0007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47938/0 (socket says 172.21.15.93:47938) 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.565+0000 7f8f0600f700 1 -- 172.21.15.93:0/1051579409 learned_addr learned my addr 172.21.15.93:0/1051579409 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.565+0000 7f8f0600f700 1 -- 172.21.15.93:0/1051579409 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0007a960 con 0x7f8f0007bde0 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.565+0000 7f8f0600f700 1 --2- 172.21.15.93:0/1051579409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 0x7f8f0007a2e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0098a0 tx=0x7f8efc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121972a3c10563a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.565+0000 7f8f0500d700 1 -- 172.21.15.93:0/1051579409 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8efc004160 con 0x7f8f0007bde0 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.565+0000 7f8f0500d700 1 -- 172.21.15.93:0/1051579409 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc0042c0 con 0x7f8f0007bde0 2023-11-16T16:38:57.139 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.565+0000 7f8f0500d700 1 -- 172.21.15.93:0/1051579409 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8efc0045a0 con 0x7f8f0007bde0 2023-11-16T16:38:57.140 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.566+0000 7f8f08273700 1 -- 172.21.15.93:0/1051579409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 msgr2=0x7f8f0007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.140 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.566+0000 7f8f08273700 1 --2- 172.21.15.93:0/1051579409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 0x7f8f0007a2e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0098a0 tx=0x7f8efc009bb0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.140 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.566+0000 7f8f08273700 1 -- 172.21.15.93:0/1051579409 shutdown_connections 2023-11-16T16:38:57.140 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.566+0000 7f8f08273700 1 --2- 172.21.15.93:0/1051579409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 0x7f8f0007a2e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.140 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.566+0000 7f8f08273700 1 -- 172.21.15.93:0/1051579409 >> 172.21.15.93:0/1051579409 conn(0x7f8f00100fa0 msgr2=0x7f8f001033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:57.140 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.566+0000 7f8f08273700 1 -- 172.21.15.93:0/1051579409 shutdown_connections 2023-11-16T16:38:57.140 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.566+0000 7f8f08273700 1 -- 172.21.15.93:0/1051579409 wait complete. 2023-11-16T16:38:57.140 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.567+0000 7f8f08273700 1 Processor -- start 2023-11-16T16:38:57.140 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.567+0000 7f8f08273700 1 -- start start 2023-11-16T16:38:57.140 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.568+0000 7f8f08273700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 0x7f8f0019afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:57.140 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.568+0000 7f8f08273700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0019b520 con 0x7f8f0007bde0 2023-11-16T16:38:57.140 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.568+0000 7f8f0600f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 0x7f8f0019afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.568+0000 7f8f0600f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 0x7f8f0019afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47942/0 (socket says 172.21.15.93:47942) 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.568+0000 7f8f0600f700 1 -- 172.21.15.93:0/2852370325 learned_addr learned my addr 172.21.15.93:0/2852370325 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.568+0000 7f8f0600f700 1 -- 172.21.15.93:0/2852370325 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc009550 con 0x7f8f0007bde0 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.568+0000 7f8f0600f700 1 --2- 172.21.15.93:0/2852370325 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 0x7f8f0019afe0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8efc004000 tx=0x7f8efc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.569+0000 7f8ef77fe700 1 -- 172.21.15.93:0/2852370325 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8efc011630 con 0x7f8f0007bde0 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.569+0000 7f8ef77fe700 1 -- 172.21.15.93:0/2852370325 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc011790 con 0x7f8f0007bde0 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.569+0000 7f8ef77fe700 1 -- 172.21.15.93:0/2852370325 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8efc011a30 con 0x7f8f0007bde0 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.569+0000 7f8f08273700 1 -- 172.21.15.93:0/2852370325 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0007ae60 con 0x7f8f0007bde0 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.569+0000 7f8f08273700 1 -- 172.21.15.93:0/2852370325 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0007acd0 con 0x7f8f0007bde0 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.569+0000 7f8ef77fe700 1 -- 172.21.15.93:0/2852370325 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8efc022070 con 0x7f8f0007bde0 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.569+0000 7f8ef77fe700 1 -- 172.21.15.93:0/2852370325 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8efc01c450 con 0x7f8f0007bde0 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.570+0000 7f8f08273700 1 -- 172.21.15.93:0/2852370325 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f00194e60 con 0x7f8f0007bde0 2023-11-16T16:38:57.141 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.572+0000 7f8ef77fe700 1 -- 172.21.15.93:0/2852370325 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f8efc01b450 con 0x7f8f0007bde0 2023-11-16T16:38:57.142 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.642+0000 7f8f08273700 1 -- 172.21.15.93:0/2852370325 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8f0002cfe0 con 0x7f8f0007bde0 2023-11-16T16:38:57.142 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.646+0000 7f8ef77fe700 1 -- 172.21.15.93:0/2852370325 <== mon.0 v2:172.21.15.93:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8efc01b5f0 con 0x7f8f0007bde0 2023-11-16T16:38:57.142 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.646+0000 7f8ef77fe700 1 -- 172.21.15.93:0/2852370325 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f8efc0376e0 con 0x7f8f0007bde0 2023-11-16T16:38:57.142 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.647+0000 7f8f08273700 1 -- 172.21.15.93:0/2852370325 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 msgr2=0x7f8f0019afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.142 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.647+0000 7f8f08273700 1 --2- 172.21.15.93:0/2852370325 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 0x7f8f0019afe0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8efc004000 tx=0x7f8efc004750 comp rx=0 tx=0).stop 2023-11-16T16:38:57.142 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.648+0000 7f8f08273700 1 -- 172.21.15.93:0/2852370325 shutdown_connections 2023-11-16T16:38:57.142 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.648+0000 7f8f08273700 1 --2- 172.21.15.93:0/2852370325 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0007bde0 0x7f8f0019afe0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.142 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.648+0000 7f8f08273700 1 -- 172.21.15.93:0/2852370325 >> 172.21.15.93:0/2852370325 conn(0x7f8f00100fa0 msgr2=0x7f8f00102800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:57.142 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.648+0000 7f8f08273700 1 -- 172.21.15.93:0/2852370325 shutdown_connections 2023-11-16T16:38:57.142 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:56.648+0000 7f8f08273700 1 -- 172.21.15.93:0/2852370325 wait complete. 2023-11-16T16:38:57.142 INFO:teuthology.orchestra.run.smithi093.stdout:Generating new minimal ceph.conf... 2023-11-16T16:38:57.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:56 smithi093 ceph-mon[97362]: from='client.? 172.21.15.93:0/2852370325' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-16T16:38:57.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:56 smithi093 ceph-mon[97362]: from='client.? 172.21.15.93:0/2852370325' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-11-16T16:38:58.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:57 smithi093 ceph-mon[97362]: from='client.? 172.21.15.93:0/1849305835' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:38:58.219 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.689+0000 7ff2126a1700 1 Processor -- start 2023-11-16T16:38:58.219 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.689+0000 7ff2126a1700 1 -- start start 2023-11-16T16:38:58.220 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.689+0000 7ff2126a1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 0x7ff20c105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.220 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.689+0000 7ff2126a1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c106260 con 0x7ff20c105910 2023-11-16T16:38:58.220 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.689+0000 7ff20bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 0x7ff20c105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.220 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.689+0000 7ff20bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 0x7ff20c105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59004/0 (socket says 172.21.15.93:59004) 2023-11-16T16:38:58.220 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.689+0000 7ff20bfff700 1 -- 172.21.15.93:0/584271256 learned_addr learned my addr 172.21.15.93:0/584271256 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:38:58.220 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.690+0000 7ff20bfff700 1 -- 172.21.15.93:0/584271256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c1063a0 con 0x7ff20c105910 2023-11-16T16:38:58.220 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.690+0000 7ff20bfff700 1 --2- 172.21.15.93:0/584271256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 0x7ff20c105d20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0098a0 tx=0x7ff1fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694e600c8bcfdd60 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:58.220 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.690+0000 7ff20affd700 1 -- 172.21.15.93:0/584271256 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1fc004160 con 0x7ff20c105910 2023-11-16T16:38:58.221 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.690+0000 7ff20affd700 1 -- 172.21.15.93:0/584271256 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1fc0042c0 con 0x7ff20c105910 2023-11-16T16:38:58.221 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.690+0000 7ff20affd700 1 -- 172.21.15.93:0/584271256 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1fc011a00 con 0x7ff20c105910 2023-11-16T16:38:58.221 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.691+0000 7ff2126a1700 1 -- 172.21.15.93:0/584271256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 msgr2=0x7ff20c105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.221 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.691+0000 7ff2126a1700 1 --2- 172.21.15.93:0/584271256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 0x7ff20c105d20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0098a0 tx=0x7ff1fc009bb0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.221 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.691+0000 7ff2126a1700 1 -- 172.21.15.93:0/584271256 shutdown_connections 2023-11-16T16:38:58.221 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.691+0000 7ff2126a1700 1 --2- 172.21.15.93:0/584271256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 0x7ff20c105d20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.221 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.691+0000 7ff2126a1700 1 -- 172.21.15.93:0/584271256 >> 172.21.15.93:0/584271256 conn(0x7ff20c100fa0 msgr2=0x7ff20c1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:58.221 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.691+0000 7ff2126a1700 1 -- 172.21.15.93:0/584271256 shutdown_connections 2023-11-16T16:38:58.221 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.691+0000 7ff2126a1700 1 -- 172.21.15.93:0/584271256 wait complete. 2023-11-16T16:38:58.221 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.692+0000 7ff2126a1700 1 Processor -- start 2023-11-16T16:38:58.222 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.692+0000 7ff2126a1700 1 -- start start 2023-11-16T16:38:58.222 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.692+0000 7ff2126a1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 0x7ff20c19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.222 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.692+0000 7ff2126a1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c19bca0 con 0x7ff20c105910 2023-11-16T16:38:58.222 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.693+0000 7ff20bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 0x7ff20c19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.222 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.693+0000 7ff20bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 0x7ff20c19b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59006/0 (socket says 172.21.15.93:59006) 2023-11-16T16:38:58.222 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.693+0000 7ff20bfff700 1 -- 172.21.15.93:0/1849305835 learned_addr learned my addr 172.21.15.93:0/1849305835 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:38:58.222 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.693+0000 7ff20bfff700 1 -- 172.21.15.93:0/1849305835 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc009550 con 0x7ff20c105910 2023-11-16T16:38:58.222 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.693+0000 7ff20bfff700 1 --2- 172.21.15.93:0/1849305835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 0x7ff20c19b760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc006af0 tx=0x7ff1fc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:58.222 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.693+0000 7ff2097fa700 1 -- 172.21.15.93:0/1849305835 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1fc003a40 con 0x7ff20c105910 2023-11-16T16:38:58.223 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.693+0000 7ff2097fa700 1 -- 172.21.15.93:0/1849305835 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1fc003ba0 con 0x7ff20c105910 2023-11-16T16:38:58.223 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.693+0000 7ff2126a1700 1 -- 172.21.15.93:0/1849305835 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20c107960 con 0x7ff20c105910 2023-11-16T16:38:58.223 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.694+0000 7ff2097fa700 1 -- 172.21.15.93:0/1849305835 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1fc01a6e0 con 0x7ff20c105910 2023-11-16T16:38:58.223 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.694+0000 7ff2126a1700 1 -- 172.21.15.93:0/1849305835 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20c1077d0 con 0x7ff20c105910 2023-11-16T16:38:58.223 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.694+0000 7ff2097fa700 1 -- 172.21.15.93:0/1849305835 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff1fc01ab60 con 0x7ff20c105910 2023-11-16T16:38:58.223 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.694+0000 7ff2097fa700 1 -- 172.21.15.93:0/1849305835 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff1fc019700 con 0x7ff20c105910 2023-11-16T16:38:58.223 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.695+0000 7ff2126a1700 1 -- 172.21.15.93:0/1849305835 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff20c062120 con 0x7ff20c105910 2023-11-16T16:38:58.223 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.698+0000 7ff2097fa700 1 -- 172.21.15.93:0/1849305835 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7ff1fc02dde0 con 0x7ff20c105910 2023-11-16T16:38:58.223 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.768+0000 7ff2126a1700 1 -- 172.21.15.93:0/1849305835 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff20c19c1e0 con 0x7ff20c105910 2023-11-16T16:38:58.224 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.768+0000 7ff2097fa700 1 -- 172.21.15.93:0/1849305835 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7ff1fc024a60 con 0x7ff20c105910 2023-11-16T16:38:58.224 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.769+0000 7ff2126a1700 1 -- 172.21.15.93:0/1849305835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 msgr2=0x7ff20c19b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.224 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.769+0000 7ff2126a1700 1 --2- 172.21.15.93:0/1849305835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 0x7ff20c19b760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc006af0 tx=0x7ff1fc011ad0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.224 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.769+0000 7ff2126a1700 1 -- 172.21.15.93:0/1849305835 shutdown_connections 2023-11-16T16:38:58.224 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.769+0000 7ff2126a1700 1 --2- 172.21.15.93:0/1849305835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff20c105910 0x7ff20c19b760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.224 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.769+0000 7ff2126a1700 1 -- 172.21.15.93:0/1849305835 >> 172.21.15.93:0/1849305835 conn(0x7ff20c100fa0 msgr2=0x7ff20c101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:58.224 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.769+0000 7ff2126a1700 1 -- 172.21.15.93:0/1849305835 shutdown_connections 2023-11-16T16:38:58.224 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:38:57.769+0000 7ff2126a1700 1 -- 172.21.15.93:0/1849305835 wait complete. 2023-11-16T16:38:58.224 INFO:teuthology.orchestra.run.smithi093.stdout:Restarting the monitor... 2023-11-16T16:38:58.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:58 smithi093 systemd[1]: Stopping Ceph mon.a for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:38:59.084 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:58 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-a[97338]: 2023-11-16T16:38:58.668+0000 7fe114fb5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:38:59.084 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:58 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-a[97338]: 2023-11-16T16:38:58.668+0000 7fe114fb5700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-11-16T16:38:59.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:59 smithi093 bash[97700]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-a 2023-11-16T16:38:59.889 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:59 smithi093 systemd[1]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.a.service: Succeeded. 2023-11-16T16:38:59.889 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:59 smithi093 systemd[1]: Stopped Ceph mon.a for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:38:59.889 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:59 smithi093 systemd[1]: Starting Ceph mon.a for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:39:00.119 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:38:59 smithi093 podman[97823]: 2023-11-16T16:39:00.119 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: set uid:gid to 167:167 (ceph:ceph) 2023-11-16T16:39:00.140 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev), process ceph-mon, pid 7 2023-11-16T16:39:00.140 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: pidfile_write: ignore empty --pid-file 2023-11-16T16:39:00.140 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 bash[97823]: c53a933466d7e529cc46c80d2e2ee033301ef2f767d114fe8a48bd90bb8a98ce 2023-11-16T16:39:00.140 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: load: jerasure load: lrc 2023-11-16T16:39:00.140 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: RocksDB version: 7.9.2 2023-11-16T16:39:00.140 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Git sha 0 2023-11-16T16:39:00.140 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Compile date 2023-11-15 23:53:54 2023-11-16T16:39:00.140 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: DB SUMMARY 2023-11-16T16:39:00.140 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: DB Session ID: WP0NTPDOI5DRC581DS1L 2023-11-16T16:39:00.140 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: CURRENT file: CURRENT 2023-11-16T16:39:00.140 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: IDENTITY file: IDENTITY 2023-11-16T16:39:00.141 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-11-16T16:39:00.141 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-11-16T16:39:00.141 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 97726 ; 2023-11-16T16:39:00.141 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.error_if_exists: 0 2023-11-16T16:39:00.141 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.create_if_missing: 0 2023-11-16T16:39:00.184 INFO:teuthology.orchestra.run.smithi093.stdout:Setting mon public_network to 172.21.0.0/20 2023-11-16T16:39:00.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.paranoid_checks: 1 2023-11-16T16:39:00.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-16T16:39:00.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-16T16:39:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-16T16:39:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.env: 0x55fc883cb720 2023-11-16T16:39:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.fs: PosixFileSystem 2023-11-16T16:39:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.info_log: 0x55fc8b6e53c0 2023-11-16T16:39:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_file_opening_threads: 16 2023-11-16T16:39:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.statistics: (nil) 2023-11-16T16:39:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.use_fsync: 0 2023-11-16T16:39:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_log_file_size: 0 2023-11-16T16:39:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-16T16:39:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-16T16:39:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.keep_log_file_num: 1000 2023-11-16T16:39:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.recycle_log_file_num: 0 2023-11-16T16:39:00.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.allow_fallocate: 1 2023-11-16T16:39:00.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.allow_mmap_reads: 0 2023-11-16T16:39:00.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.allow_mmap_writes: 0 2023-11-16T16:39:00.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.use_direct_reads: 0 2023-11-16T16:39:00.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-16T16:39:00.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.create_missing_column_families: 0 2023-11-16T16:39:00.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.db_log_dir: 2023-11-16T16:39:00.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.wal_dir: 2023-11-16T16:39:00.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-16T16:39:00.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-16T16:39:00.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-16T16:39:00.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-16T16:39:00.403 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-16T16:39:00.403 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-16T16:39:00.403 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.advise_random_on_open: 1 2023-11-16T16:39:00.403 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.db_write_buffer_size: 0 2023-11-16T16:39:00.403 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.write_buffer_manager: 0x55fc8a9563c0 2023-11-16T16:39:00.403 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-16T16:39:00.403 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-16T16:39:00.403 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-16T16:39:00.403 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.rate_limiter: (nil) 2023-11-16T16:39:00.403 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-16T16:39:00.403 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.wal_recovery_mode: 2 2023-11-16T16:39:00.404 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.enable_thread_tracking: 0 2023-11-16T16:39:00.404 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.enable_pipelined_write: 0 2023-11-16T16:39:00.405 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.unordered_write: 0 2023-11-16T16:39:00.405 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-16T16:39:00.405 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-16T16:39:00.405 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-16T16:39:00.405 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-16T16:39:00.405 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.row_cache: None 2023-11-16T16:39:00.405 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.wal_filter: None 2023-11-16T16:39:00.405 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-16T16:39:00.405 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.allow_ingest_behind: 0 2023-11-16T16:39:00.405 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.two_write_queues: 0 2023-11-16T16:39:00.405 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.manual_wal_flush: 0 2023-11-16T16:39:00.405 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.wal_compression: 0 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.atomic_flush: 0 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.log_readahead_size: 0 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.best_efforts_recovery: 0 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.allow_data_in_errors: 0 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.db_host_id: __hostname__ 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-16T16:39:00.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_background_jobs: 2 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_background_compactions: -1 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_subcompactions: 1 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_total_wal_size: 0 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_open_files: -1 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bytes_per_sync: 0 2023-11-16T16:39:00.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-16T16:39:00.408 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-16T16:39:00.408 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_readahead_size: 0 2023-11-16T16:39:00.408 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_background_flushes: -1 2023-11-16T16:39:00.408 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Compression algorithms supported: 2023-11-16T16:39:00.408 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-16T16:39:00.408 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: kZSTD supported: 0 2023-11-16T16:39:00.408 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: kXpressCompression supported: 0 2023-11-16T16:39:00.408 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: kLZ4HCCompression supported: 1 2023-11-16T16:39:00.408 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: kZlibCompression supported: 1 2023-11-16T16:39:00.408 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: kSnappyCompression supported: 1 2023-11-16T16:39:00.408 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: kLZ4Compression supported: 1 2023-11-16T16:39:00.408 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: kBZip2Compression supported: 0 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.merge_operator: 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_filter: None 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_filter_factory: None 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.sst_partitioner_factory: None 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fc8b6e54e0) 2023-11-16T16:39:00.409 INFO:journalctl@ceph.mon.a.smithi093.stdout: cache_index_and_filter_blocks: 1 2023-11-16T16:39:00.410 INFO:journalctl@ceph.mon.a.smithi093.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-16T16:39:00.410 INFO:journalctl@ceph.mon.a.smithi093.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-16T16:39:00.410 INFO:journalctl@ceph.mon.a.smithi093.stdout: pin_top_level_index_and_filter: 1 2023-11-16T16:39:00.410 INFO:journalctl@ceph.mon.a.smithi093.stdout: index_type: 0 2023-11-16T16:39:00.410 INFO:journalctl@ceph.mon.a.smithi093.stdout: data_block_index_type: 0 2023-11-16T16:39:00.410 INFO:journalctl@ceph.mon.a.smithi093.stdout: index_shortening: 1 2023-11-16T16:39:00.410 INFO:journalctl@ceph.mon.a.smithi093.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-16T16:39:00.410 INFO:journalctl@ceph.mon.a.smithi093.stdout: checksum: 4 2023-11-16T16:39:00.410 INFO:journalctl@ceph.mon.a.smithi093.stdout: no_block_cache: 0 2023-11-16T16:39:00.410 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_cache: 0x55fc8a9f71f0 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_cache_name: BinnedLRUCache 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_cache_options: 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: capacity : 536870912 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: num_shard_bits : 4 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: strict_capacity_limit : 0 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: high_pri_pool_ratio: 0.000 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_cache_compressed: (nil) 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: persistent_cache: (nil) 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_size: 4096 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_size_deviation: 10 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_restart_interval: 16 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: index_block_restart_interval: 1 2023-11-16T16:39:00.411 INFO:journalctl@ceph.mon.a.smithi093.stdout: metadata_block_size: 4096 2023-11-16T16:39:00.412 INFO:journalctl@ceph.mon.a.smithi093.stdout: partition_filters: 0 2023-11-16T16:39:00.412 INFO:journalctl@ceph.mon.a.smithi093.stdout: use_delta_encoding: 1 2023-11-16T16:39:00.412 INFO:journalctl@ceph.mon.a.smithi093.stdout: filter_policy: bloomfilter 2023-11-16T16:39:00.412 INFO:journalctl@ceph.mon.a.smithi093.stdout: whole_key_filtering: 1 2023-11-16T16:39:00.412 INFO:journalctl@ceph.mon.a.smithi093.stdout: verify_compression: 0 2023-11-16T16:39:00.412 INFO:journalctl@ceph.mon.a.smithi093.stdout: read_amp_bytes_per_bit: 0 2023-11-16T16:39:00.412 INFO:journalctl@ceph.mon.a.smithi093.stdout: format_version: 5 2023-11-16T16:39:00.412 INFO:journalctl@ceph.mon.a.smithi093.stdout: enable_index_compression: 1 2023-11-16T16:39:00.412 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_align: 0 2023-11-16T16:39:00.412 INFO:journalctl@ceph.mon.a.smithi093.stdout: max_auto_readahead_size: 262144 2023-11-16T16:39:00.412 INFO:journalctl@ceph.mon.a.smithi093.stdout: prepopulate_block_cache: 0 2023-11-16T16:39:00.412 INFO:journalctl@ceph.mon.a.smithi093.stdout: initial_auto_readahead_size: 8192 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout: num_file_reads_for_auto_readahead: 2 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.write_buffer_size: 33554432 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_write_buffer_number: 2 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compression: NoCompression 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bottommost_compression: Disabled 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.prefix_extractor: nullptr 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.num_levels: 7 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-16T16:39:00.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compression_opts.level: 32767 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compression_opts.strategy: 0 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-16T16:39:00.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compression_opts.enabled: false 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.target_file_size_base: 67108864 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-16T16:39:00.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.arena_block_size: 1048576 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.disable_auto_compactions: 0 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-16T16:39:00.416 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.table_properties_collectors: 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.inplace_update_support: 0 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-16T16:39:00.417 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.bloom_locality: 0 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.max_successive_merges: 0 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.paranoid_file_checks: 0 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.force_consistency_checks: 1 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.report_bg_io_stats: 0 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.ttl: 2592000 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.enable_blob_files: false 2023-11-16T16:39:00.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.min_blob_size: 0 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.blob_file_size: 268435456 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.blob_file_starting_level: 0 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 794f971f-d4fc-4009-b127-95276d47b41c 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152740131353, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-11-16T16:39:00.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-11-16T16:39:00.420 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152740133833, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93034, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 310, "table_properties": {"data_size": 91056, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13416, "raw_average_key_size": 49, "raw_value_size": 83748, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 269, "num_filter_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1700152740, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "794f971f-d4fc-4009-b127-95276d47b41c", "db_session_id": "WP0NTPDOI5DRC581DS1L", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-11-16T16:39:00.420 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152740134028, "job": 1, "event": "recovery_finished"} 2023-11-16T16:39:00.420 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-11-16T16:39:00.420 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-16T16:39:00.420 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fc8aa94000 2023-11-16T16:39:00.420 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: DB pointer 0x55fc8aa80000 2023-11-16T16:39:00.420 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-16T16:39:00.420 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-16T16:39:00.420 INFO:journalctl@ceph.mon.a.smithi093.stdout: ** DB Stats ** 2023-11-16T16:39:00.420 INFO:journalctl@ceph.mon.a.smithi093.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T16:39:00.420 INFO:journalctl@ceph.mon.a.smithi093.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-16T16:39:00.420 INFO:journalctl@ceph.mon.a.smithi093.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: ** Compaction Stats [default] ** 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: L0 2/0 92.69 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 61.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: Sum 2/0 92.69 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 61.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 61.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-11-16T16:39:00.421 INFO:journalctl@ceph.mon.a.smithi093.stdout: ** Compaction Stats [default] ** 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 61.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-16T16:39:00.422 INFO:journalctl@ceph.mon.a.smithi093.stdout: Cumulative compaction: 0.00 GB write, 9.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout: Interval compaction: 0.00 GB write, 9.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout: Block cache BinnedLRUCache@0x55fc8a9f71f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: starting mon.a rank 0 at public addrs [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] at bind addrs [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: mon.a@-1(???) e1 preinit fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: mon.a@-1(???).mds e1 new map 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: mon.a@-1(???).mds e1 print_map 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout: e1 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-16T16:39:00.423 INFO:journalctl@ceph.mon.a.smithi093.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout: legacy client fscid: -1 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout: No filesystems configured 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: expand_channel_meta expand map: {default=false} 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: expand_channel_meta from 'false' to 'false' 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: expand_channel_meta expanded map: {default=false} 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: expand_channel_meta expand map: {default=info} 2023-11-16T16:39:00.424 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: expand_channel_meta from 'info' to 'info' 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: expand_channel_meta expanded map: {default=info} 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: expand_channel_meta expand map: {default=daemon} 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: expand_channel_meta expanded map: {default=daemon} 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: expand_channel_meta expand map: {default=debug} 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: expand_channel_meta from 'debug' to 'debug' 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: expand_channel_meta expanded map: {default=debug} 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 systemd[1]: Started Ceph mon.a for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: monmap e1: 1 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0]} removed_ranks: {} 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: fsmap 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: osdmap e1: 0 total, 0 up, 0 in 2023-11-16T16:39:00.425 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:00 smithi093 ceph-mon[97863]: mgrmap e1: no daemons active 2023-11-16T16:39:01.266 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.694+0000 7f096eda4700 1 Processor -- start 2023-11-16T16:39:01.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.694+0000 7f096eda4700 1 -- start start 2023-11-16T16:39:01.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.694+0000 7f096eda4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 0x7f096807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:01.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.694+0000 7f096eda4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096807a820 con 0x7f096807bde0 2023-11-16T16:39:01.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.695+0000 7f096cb40700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 0x7f096807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:01.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.695+0000 7f096cb40700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 0x7f096807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59012/0 (socket says 172.21.15.93:59012) 2023-11-16T16:39:01.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.695+0000 7f096cb40700 1 -- 172.21.15.93:0/1239970022 learned_addr learned my addr 172.21.15.93:0/1239970022 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:01.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.695+0000 7f096cb40700 1 -- 172.21.15.93:0/1239970022 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096807a960 con 0x7f096807bde0 2023-11-16T16:39:01.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.695+0000 7f096cb40700 1 --2- 172.21.15.93:0/1239970022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 0x7f096807a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f095c0098a0 tx=0x7f095c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6d0429caa61892 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:01.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.696+0000 7f09677fe700 1 -- 172.21.15.93:0/1239970022 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095c004160 con 0x7f096807bde0 2023-11-16T16:39:01.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.696+0000 7f09677fe700 1 -- 172.21.15.93:0/1239970022 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f095c0042c0 con 0x7f096807bde0 2023-11-16T16:39:01.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.696+0000 7f09677fe700 1 -- 172.21.15.93:0/1239970022 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095c011a00 con 0x7f096807bde0 2023-11-16T16:39:01.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.696+0000 7f096eda4700 1 -- 172.21.15.93:0/1239970022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 msgr2=0x7f096807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:01.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.696+0000 7f096eda4700 1 --2- 172.21.15.93:0/1239970022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 0x7f096807a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f095c0098a0 tx=0x7f095c009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.696+0000 7f096eda4700 1 -- 172.21.15.93:0/1239970022 shutdown_connections 2023-11-16T16:39:01.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.696+0000 7f096eda4700 1 --2- 172.21.15.93:0/1239970022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 0x7f096807a2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.696+0000 7f096eda4700 1 -- 172.21.15.93:0/1239970022 >> 172.21.15.93:0/1239970022 conn(0x7f0968100fa0 msgr2=0x7f09681033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:01.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.696+0000 7f096eda4700 1 -- 172.21.15.93:0/1239970022 shutdown_connections 2023-11-16T16:39:01.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.696+0000 7f096eda4700 1 -- 172.21.15.93:0/1239970022 wait complete. 2023-11-16T16:39:01.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.697+0000 7f096eda4700 1 Processor -- start 2023-11-16T16:39:01.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.698+0000 7f096eda4700 1 -- start start 2023-11-16T16:39:01.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.698+0000 7f096eda4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 0x7f096819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:01.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.698+0000 7f096eda4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096819bcc0 con 0x7f096807bde0 2023-11-16T16:39:01.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.698+0000 7f096cb40700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 0x7f096819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:01.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.698+0000 7f096cb40700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 0x7f096819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59026/0 (socket says 172.21.15.93:59026) 2023-11-16T16:39:01.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.698+0000 7f096cb40700 1 -- 172.21.15.93:0/4076446390 learned_addr learned my addr 172.21.15.93:0/4076446390 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:01.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.698+0000 7f096cb40700 1 -- 172.21.15.93:0/4076446390 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c009550 con 0x7f096807bde0 2023-11-16T16:39:01.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.699+0000 7f096cb40700 1 --2- 172.21.15.93:0/4076446390 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 0x7f096819b780 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f095c009fb0 tx=0x7f095c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:01.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.699+0000 7f0965ffb700 1 -- 172.21.15.93:0/4076446390 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095c003930 con 0x7f096807bde0 2023-11-16T16:39:01.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.699+0000 7f096eda4700 1 -- 172.21.15.93:0/4076446390 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096810be00 con 0x7f096807bde0 2023-11-16T16:39:01.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.699+0000 7f0965ffb700 1 -- 172.21.15.93:0/4076446390 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f095c003a90 con 0x7f096807bde0 2023-11-16T16:39:01.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.699+0000 7f0965ffb700 1 -- 172.21.15.93:0/4076446390 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095c0195e0 con 0x7f096807bde0 2023-11-16T16:39:01.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.699+0000 7f096eda4700 1 -- 172.21.15.93:0/4076446390 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096810bc70 con 0x7f096807bde0 2023-11-16T16:39:01.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.699+0000 7f0965ffb700 1 -- 172.21.15.93:0/4076446390 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f095c019bc0 con 0x7f096807bde0 2023-11-16T16:39:01.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.699+0000 7f0965ffb700 1 -- 172.21.15.93:0/4076446390 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f095c0243f0 con 0x7f096807bde0 2023-11-16T16:39:01.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.700+0000 7f096eda4700 1 -- 172.21.15.93:0/4076446390 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0968062120 con 0x7f096807bde0 2023-11-16T16:39:01.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.703+0000 7f0965ffb700 1 -- 172.21.15.93:0/4076446390 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f095c003c00 con 0x7f096807bde0 2023-11-16T16:39:01.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.772+0000 7f096eda4700 1 -- 172.21.15.93:0/4076446390 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f096819c200 con 0x7f096807bde0 2023-11-16T16:39:01.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.775+0000 7f0965ffb700 1 -- 172.21.15.93:0/4076446390 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f095c01a9d0 con 0x7f096807bde0 2023-11-16T16:39:01.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.776+0000 7f096eda4700 1 -- 172.21.15.93:0/4076446390 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 msgr2=0x7f096819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:01.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.776+0000 7f096eda4700 1 --2- 172.21.15.93:0/4076446390 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 0x7f096819b780 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f095c009fb0 tx=0x7f095c011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.776+0000 7f096eda4700 1 -- 172.21.15.93:0/4076446390 shutdown_connections 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.776+0000 7f096eda4700 1 --2- 172.21.15.93:0/4076446390 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096807bde0 0x7f096819b780 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.776+0000 7f096eda4700 1 -- 172.21.15.93:0/4076446390 >> 172.21.15.93:0/4076446390 conn(0x7f0968100fa0 msgr2=0x7f0968101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.776+0000 7f096eda4700 1 -- 172.21.15.93:0/4076446390 shutdown_connections 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:00.776+0000 7f096eda4700 1 -- 172.21.15.93:0/4076446390 wait complete. 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi093.stdout:Wrote config to /etc/ceph/ceph.conf 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi093.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi093.stdout:Creating mgr... 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying port 9283 ... 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying port 8765 ... 2023-11-16T16:39:01.559 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.x 2023-11-16T16:39:01.559 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.x.service: Unit ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.x.service not loaded. 2023-11-16T16:39:01.757 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7.target.wants/ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@.service. 2023-11-16T16:39:01.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:01 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/4076446390' entity='client.admin' 2023-11-16T16:39:01.901 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:01 smithi093 systemd[1]: Starting Ceph mgr.x for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:39:02.516 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:02 smithi093 podman[98095]: 2023-11-16T16:39:02.574 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-16T16:39:02.574 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout disabled 2023-11-16T16:39:02.593 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-16T16:39:02.593 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-11-16T16:39:02.593 INFO:teuthology.orchestra.run.smithi093.stdout:firewalld.service is not enabled 2023-11-16T16:39:02.593 INFO:teuthology.orchestra.run.smithi093.stdout:Not possible to enable service . firewalld.service is not available 2023-11-16T16:39:02.615 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-16T16:39:02.616 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout disabled 2023-11-16T16:39:02.634 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-16T16:39:02.634 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-11-16T16:39:02.634 INFO:teuthology.orchestra.run.smithi093.stdout:firewalld.service is not enabled 2023-11-16T16:39:02.634 INFO:teuthology.orchestra.run.smithi093.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-11-16T16:39:02.634 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mgr to start... 2023-11-16T16:39:02.634 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mgr... 2023-11-16T16:39:02.820 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:02 smithi093 bash[98095]: 9644b5ae03cee4e8b5b18754aa81b9f6207b50b4aa4fed4116afc50e6fc87e71 2023-11-16T16:39:02.820 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:02 smithi093 systemd[1]: Started Ceph mgr.x for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:39:02.821 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:02 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:02.713+0000 7fb46fe91200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:39:03.080 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:02 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:02.818+0000 7fb46fe91200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:39:03.081 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:02 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:02.945+0000 7fb46fe91200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:39:03.081 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:03 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:03.078+0000 7fb46fe91200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:39:03.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:03 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/157775216' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-16T16:39:03.653 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:03 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:03.333+0000 7fb46fe91200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:39:03.832 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:39:03.832 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:39:03.832 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsid": "6a59c1a8-849e-11ee-8dbe-212e2dc638e7", 2023-11-16T16:39:03.832 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "health": { 2023-11-16T16:39:03.832 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-16T16:39:03.832 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-16T16:39:03.832 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-16T16:39:03.833 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:03.833 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-16T16:39:03.833 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-16T16:39:03.833 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 0 2023-11-16T16:39:03.833 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:39:03.833 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-16T16:39:03.833 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "a" 2023-11-16T16:39:03.833 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:39:03.833 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-11-16T16:39:03.834 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-16T16:39:03.834 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:03.834 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-16T16:39:03.834 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-16T16:39:03.834 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:03.834 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-16T16:39:03.834 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:03.834 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-16T16:39:03.834 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-16T16:39:03.835 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-16T16:39:03.835 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-16T16:39:03.835 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-16T16:39:03.835 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-16T16:39:03.835 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:03.835 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-16T16:39:03.835 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-16T16:39:03.835 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-16T16:39:03.835 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-16T16:39:03.835 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-16T16:39:03.836 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-16T16:39:03.836 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-16T16:39:03.836 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-16T16:39:03.836 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-16T16:39:03.836 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:03.836 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-16T16:39:03.836 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:03.836 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-16T16:39:03.836 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-16T16:39:03.837 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:03.837 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-16T16:39:03.837 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-16T16:39:03.837 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-16T16:39:03.837 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-16T16:39:03.837 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-16T16:39:03.837 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-16T16:39:03.837 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "restful" 2023-11-16T16:39:03.838 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:39:03.838 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:39:03.838 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:03.838 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-16T16:39:03.838 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:03.838 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modified": "2023-11-16T16:38:54.903744+0000", 2023-11-16T16:39:03.838 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:39:03.838 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:03.838 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-16T16:39:03.839 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:39:03.839 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.211+0000 7f7715c3d700 1 Processor -- start 2023-11-16T16:39:03.839 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.211+0000 7f7715c3d700 1 -- start start 2023-11-16T16:39:03.839 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.211+0000 7f7715c3d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 0x7f771007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:03.839 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.211+0000 7f7715c3d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771007a820 con 0x7f771007bde0 2023-11-16T16:39:03.839 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.212+0000 7f770f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 0x7f771007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:03.839 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.212+0000 7f770f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 0x7f771007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59056/0 (socket says 172.21.15.93:59056) 2023-11-16T16:39:03.839 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.212+0000 7f770f7fe700 1 -- 172.21.15.93:0/3495615365 learned_addr learned my addr 172.21.15.93:0/3495615365 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:03.839 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.212+0000 7f770f7fe700 1 -- 172.21.15.93:0/3495615365 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771007a960 con 0x7f771007bde0 2023-11-16T16:39:03.840 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.212+0000 7f770f7fe700 1 --2- 172.21.15.93:0/3495615365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 0x7f771007a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f77000098a0 tx=0x7f7700009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54baf6252a017334 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:03.840 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.213+0000 7f770e7fc700 1 -- 172.21.15.93:0/3495615365 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7700004160 con 0x7f771007bde0 2023-11-16T16:39:03.840 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.213+0000 7f770e7fc700 1 -- 172.21.15.93:0/3495615365 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f77000042c0 con 0x7f771007bde0 2023-11-16T16:39:03.840 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.213+0000 7f770e7fc700 1 -- 172.21.15.93:0/3495615365 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7700011a00 con 0x7f771007bde0 2023-11-16T16:39:03.840 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.213+0000 7f7715c3d700 1 -- 172.21.15.93:0/3495615365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 msgr2=0x7f771007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:03.840 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.213+0000 7f7715c3d700 1 --2- 172.21.15.93:0/3495615365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 0x7f771007a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f77000098a0 tx=0x7f7700009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:03.840 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.214+0000 7f7715c3d700 1 -- 172.21.15.93:0/3495615365 shutdown_connections 2023-11-16T16:39:03.840 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.214+0000 7f7715c3d700 1 --2- 172.21.15.93:0/3495615365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 0x7f771007a2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:03.840 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.214+0000 7f7715c3d700 1 -- 172.21.15.93:0/3495615365 >> 172.21.15.93:0/3495615365 conn(0x7f7710100fa0 msgr2=0x7f77101033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:03.841 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.214+0000 7f7715c3d700 1 -- 172.21.15.93:0/3495615365 shutdown_connections 2023-11-16T16:39:03.841 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.214+0000 7f7715c3d700 1 -- 172.21.15.93:0/3495615365 wait complete. 2023-11-16T16:39:03.841 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.215+0000 7f7715c3d700 1 Processor -- start 2023-11-16T16:39:03.841 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.215+0000 7f7715c3d700 1 -- start start 2023-11-16T16:39:03.841 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.215+0000 7f7715c3d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 0x7f771019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:03.841 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.215+0000 7f7715c3d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771019bcf0 con 0x7f771007bde0 2023-11-16T16:39:03.841 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.215+0000 7f770f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 0x7f771019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:03.841 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.215+0000 7f770f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 0x7f771019b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59066/0 (socket says 172.21.15.93:59066) 2023-11-16T16:39:03.841 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.215+0000 7f770f7fe700 1 -- 172.21.15.93:0/157775216 learned_addr learned my addr 172.21.15.93:0/157775216 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:03.841 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.216+0000 7f770f7fe700 1 -- 172.21.15.93:0/157775216 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7700009550 con 0x7f771007bde0 2023-11-16T16:39:03.842 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.216+0000 7f770f7fe700 1 --2- 172.21.15.93:0/157775216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 0x7f771019b7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7700006af0 tx=0x7f7700011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:03.842 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.216+0000 7f770cff9700 1 -- 172.21.15.93:0/157775216 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7700003a40 con 0x7f771007bde0 2023-11-16T16:39:03.842 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.216+0000 7f770cff9700 1 -- 172.21.15.93:0/157775216 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7700003ba0 con 0x7f771007bde0 2023-11-16T16:39:03.842 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.216+0000 7f770cff9700 1 -- 172.21.15.93:0/157775216 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f770001a700 con 0x7f771007bde0 2023-11-16T16:39:03.842 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.216+0000 7f7715c3d700 1 -- 172.21.15.93:0/157775216 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771010be00 con 0x7f771007bde0 2023-11-16T16:39:03.842 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.217+0000 7f7715c3d700 1 -- 172.21.15.93:0/157775216 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771010bc70 con 0x7f771007bde0 2023-11-16T16:39:03.842 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.217+0000 7f770cff9700 1 -- 172.21.15.93:0/157775216 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f770001ac30 con 0x7f771007bde0 2023-11-16T16:39:03.842 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.217+0000 7f770cff9700 1 -- 172.21.15.93:0/157775216 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7700019700 con 0x7f771007bde0 2023-11-16T16:39:03.842 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.218+0000 7f7715c3d700 1 -- 172.21.15.93:0/157775216 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771010bf90 con 0x7f771007bde0 2023-11-16T16:39:03.843 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.223+0000 7f770cff9700 1 -- 172.21.15.93:0/157775216 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f7700024ad0 con 0x7f771007bde0 2023-11-16T16:39:03.843 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.319+0000 7f7715c3d700 1 -- 172.21.15.93:0/157775216 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f771019c2f0 con 0x7f771007bde0 2023-11-16T16:39:03.843 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.320+0000 7f770cff9700 1 -- 172.21.15.93:0/157775216 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f7700024c70 con 0x7f771007bde0 2023-11-16T16:39:03.843 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.321+0000 7f7715c3d700 1 -- 172.21.15.93:0/157775216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 msgr2=0x7f771019b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:03.843 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.321+0000 7f7715c3d700 1 --2- 172.21.15.93:0/157775216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 0x7f771019b7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7700006af0 tx=0x7f7700011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:03.843 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.321+0000 7f7715c3d700 1 -- 172.21.15.93:0/157775216 shutdown_connections 2023-11-16T16:39:03.843 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.321+0000 7f7715c3d700 1 --2- 172.21.15.93:0/157775216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f771007bde0 0x7f771019b7b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:03.843 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.321+0000 7f7715c3d700 1 -- 172.21.15.93:0/157775216 >> 172.21.15.93:0/157775216 conn(0x7f7710100fa0 msgr2=0x7f7710101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:03.843 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.322+0000 7f7715c3d700 1 -- 172.21.15.93:0/157775216 shutdown_connections 2023-11-16T16:39:03.844 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:03.322+0000 7f7715c3d700 1 -- 172.21.15.93:0/157775216 wait complete. 2023-11-16T16:39:03.844 INFO:teuthology.orchestra.run.smithi093.stdout:mgr not available, waiting (1/15)... 2023-11-16T16:39:04.035 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:03 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:03.756+0000 7fb46fe91200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:39:04.035 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:03 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:03.873+0000 7fb46fe91200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:39:04.035 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:04 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:04.033+0000 7fb46fe91200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:39:04.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:04 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:04.154+0000 7fb46fe91200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:39:05.150 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:04 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:04.691+0000 7fb46fe91200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:39:05.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:05 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:05.463+0000 7fb46fe91200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:39:05.901 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:05 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:05.668+0000 7fb46fe91200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:39:06.782 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:06 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2360336086' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-16T16:39:06.782 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:06 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:06.661+0000 7fb46fe91200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:39:06.906 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:39:06.909 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:39:06.909 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsid": "6a59c1a8-849e-11ee-8dbe-212e2dc638e7", 2023-11-16T16:39:06.909 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "health": { 2023-11-16T16:39:06.909 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-16T16:39:06.909 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-16T16:39:06.909 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-16T16:39:06.909 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:06.909 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-16T16:39:06.909 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-16T16:39:06.909 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 0 2023-11-16T16:39:06.910 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:39:06.910 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-16T16:39:06.910 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "a" 2023-11-16T16:39:06.910 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:39:06.910 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-11-16T16:39:06.910 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-16T16:39:06.910 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:06.910 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-16T16:39:06.910 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-16T16:39:06.910 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:06.910 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-16T16:39:06.910 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:06.911 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-16T16:39:06.911 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-16T16:39:06.911 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-16T16:39:06.918 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-16T16:39:06.918 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-16T16:39:06.918 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-16T16:39:06.918 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:06.918 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-16T16:39:06.918 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-16T16:39:06.918 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-16T16:39:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-16T16:39:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-16T16:39:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-16T16:39:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-16T16:39:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-16T16:39:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-16T16:39:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-16T16:39:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-16T16:39:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-16T16:39:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "restful" 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modified": "2023-11-16T16:38:54.903744+0000", 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.332+0000 7f79ddbea700 1 Processor -- start 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.332+0000 7f79ddbea700 1 -- start start 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.332+0000 7f79ddbea700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 0x7f79d8105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.332+0000 7f79ddbea700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8106260 con 0x7f79d8105910 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.333+0000 7f79d77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 0x7f79d8105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.333+0000 7f79d77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 0x7f79d8105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59080/0 (socket says 172.21.15.93:59080) 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.333+0000 7f79d77fe700 1 -- 172.21.15.93:0/1794681199 learned_addr learned my addr 172.21.15.93:0/1794681199 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.333+0000 7f79d77fe700 1 -- 172.21.15.93:0/1794681199 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d81063a0 con 0x7f79d8105910 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.333+0000 7f79d77fe700 1 --2- 172.21.15.93:0/1794681199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 0x7f79d8105d20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79c80098a0 tx=0x7f79c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b43c15acdc2ad4 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:06.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.334+0000 7f79d67fc700 1 -- 172.21.15.93:0/1794681199 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79c8004160 con 0x7f79d8105910 2023-11-16T16:39:06.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.334+0000 7f79d67fc700 1 -- 172.21.15.93:0/1794681199 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79c80042c0 con 0x7f79d8105910 2023-11-16T16:39:06.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.334+0000 7f79d67fc700 1 -- 172.21.15.93:0/1794681199 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79c8011a00 con 0x7f79d8105910 2023-11-16T16:39:06.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.334+0000 7f79ddbea700 1 -- 172.21.15.93:0/1794681199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 msgr2=0x7f79d8105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:06.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.334+0000 7f79ddbea700 1 --2- 172.21.15.93:0/1794681199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 0x7f79d8105d20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79c80098a0 tx=0x7f79c8009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:06.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.334+0000 7f79ddbea700 1 -- 172.21.15.93:0/1794681199 shutdown_connections 2023-11-16T16:39:06.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.334+0000 7f79ddbea700 1 --2- 172.21.15.93:0/1794681199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 0x7f79d8105d20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:06.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.334+0000 7f79ddbea700 1 -- 172.21.15.93:0/1794681199 >> 172.21.15.93:0/1794681199 conn(0x7f79d8100fa0 msgr2=0x7f79d81033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:06.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.334+0000 7f79ddbea700 1 -- 172.21.15.93:0/1794681199 shutdown_connections 2023-11-16T16:39:06.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.334+0000 7f79ddbea700 1 -- 172.21.15.93:0/1794681199 wait complete. 2023-11-16T16:39:06.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.335+0000 7f79ddbea700 1 Processor -- start 2023-11-16T16:39:06.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.336+0000 7f79ddbea700 1 -- start start 2023-11-16T16:39:06.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.336+0000 7f79ddbea700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 0x7f79d807bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.336+0000 7f79ddbea700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d807c190 con 0x7f79d8105910 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.336+0000 7f79d77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 0x7f79d807bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.336+0000 7f79d77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 0x7f79d807bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59088/0 (socket says 172.21.15.93:59088) 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.336+0000 7f79d77fe700 1 -- 172.21.15.93:0/2360336086 learned_addr learned my addr 172.21.15.93:0/2360336086 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.336+0000 7f79d77fe700 1 -- 172.21.15.93:0/2360336086 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c8009550 con 0x7f79d8105910 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.337+0000 7f79d77fe700 1 --2- 172.21.15.93:0/2360336086 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 0x7f79d807bc50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79c8006af0 tx=0x7f79c8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.337+0000 7f79d4ff9700 1 -- 172.21.15.93:0/2360336086 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79c8003a40 con 0x7f79d8105910 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.337+0000 7f79d4ff9700 1 -- 172.21.15.93:0/2360336086 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79c8003ba0 con 0x7f79d8105910 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.337+0000 7f79ddbea700 1 -- 172.21.15.93:0/2360336086 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d8107960 con 0x7f79d8105910 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.337+0000 7f79d4ff9700 1 -- 172.21.15.93:0/2360336086 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79c801a6e0 con 0x7f79d8105910 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.337+0000 7f79ddbea700 1 -- 172.21.15.93:0/2360336086 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d81077d0 con 0x7f79d8105910 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.338+0000 7f79d4ff9700 1 -- 172.21.15.93:0/2360336086 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f79c8024430 con 0x7f79d8105910 2023-11-16T16:39:06.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.338+0000 7f79d4ff9700 1 -- 172.21.15.93:0/2360336086 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f79c8019700 con 0x7f79d8105910 2023-11-16T16:39:06.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.338+0000 7f79be7fc700 1 -- 172.21.15.93:0/2360336086 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d8062120 con 0x7f79d8105910 2023-11-16T16:39:06.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.341+0000 7f79d4ff9700 1 -- 172.21.15.93:0/2360336086 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f79c8043b10 con 0x7f79d8105910 2023-11-16T16:39:06.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.416+0000 7f79be7fc700 1 -- 172.21.15.93:0/2360336086 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f79d81a0d90 con 0x7f79d8105910 2023-11-16T16:39:06.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.417+0000 7f79d4ff9700 1 -- 172.21.15.93:0/2360336086 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f79c8032030 con 0x7f79d8105910 2023-11-16T16:39:06.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.418+0000 7f79be7fc700 1 -- 172.21.15.93:0/2360336086 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 msgr2=0x7f79d807bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:06.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.418+0000 7f79be7fc700 1 --2- 172.21.15.93:0/2360336086 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 0x7f79d807bc50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79c8006af0 tx=0x7f79c8011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:06.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.418+0000 7f79be7fc700 1 -- 172.21.15.93:0/2360336086 shutdown_connections 2023-11-16T16:39:06.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.418+0000 7f79be7fc700 1 --2- 172.21.15.93:0/2360336086 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79d8105910 0x7f79d807bc50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:06.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.418+0000 7f79be7fc700 1 -- 172.21.15.93:0/2360336086 >> 172.21.15.93:0/2360336086 conn(0x7f79d8100fa0 msgr2=0x7f79d8101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:06.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.418+0000 7f79be7fc700 1 -- 172.21.15.93:0/2360336086 shutdown_connections 2023-11-16T16:39:06.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:06.418+0000 7f79be7fc700 1 -- 172.21.15.93:0/2360336086 wait complete. 2023-11-16T16:39:06.924 INFO:teuthology.orchestra.run.smithi093.stdout:mgr not available, waiting (2/15)... 2023-11-16T16:39:07.150 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:06 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:06.780+0000 7fb46fe91200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:39:07.150 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:07 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:07.005+0000 7fb46fe91200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:39:07.650 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:07 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:07.368+0000 7fb46fe91200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:39:08.152 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:07 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:07.652+0000 7fb46fe91200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:39:08.152 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:07 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:07.844+0000 7fb46fe91200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:39:08.650 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:08 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:08.187+0000 7fb46fe91200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:39:09.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:09 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:09.115+0000 7fb46fe91200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:39:09.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:09 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:09.303+0000 7fb46fe91200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:39:09.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:09 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1075841379' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-16T16:39:09.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:09 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:09.564+0000 7fb46fe91200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:39:09.901 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:09 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:09.723+0000 7fb46fe91200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:39:09.993 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:39:10.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:39:10.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsid": "6a59c1a8-849e-11ee-8dbe-212e2dc638e7", 2023-11-16T16:39:10.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "health": { 2023-11-16T16:39:10.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-16T16:39:10.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-16T16:39:10.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-16T16:39:10.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 0 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "a" 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-16T16:39:10.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-16T16:39:10.003 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-16T16:39:10.004 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-16T16:39:10.005 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-16T16:39:10.005 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-16T16:39:10.005 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-16T16:39:10.005 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-16T16:39:10.005 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "restful" 2023-11-16T16:39:10.005 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:39:10.005 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:39:10.005 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:10.005 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-16T16:39:10.005 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:10.005 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modified": "2023-11-16T16:38:54.903744+0000", 2023-11-16T16:39:10.005 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:39:10.006 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:10.006 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-16T16:39:10.006 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:39:10.006 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.453+0000 7f3f24e34700 1 Processor -- start 2023-11-16T16:39:10.006 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.453+0000 7f3f24e34700 1 -- start start 2023-11-16T16:39:10.006 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.454+0000 7f3f24e34700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 0x7f3f20105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:10.006 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.454+0000 7f3f24e34700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f20106260 con 0x7f3f20105910 2023-11-16T16:39:10.006 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.454+0000 7f3f1e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 0x7f3f20105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:10.006 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.454+0000 7f3f1e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 0x7f3f20105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35962/0 (socket says 172.21.15.93:35962) 2023-11-16T16:39:10.006 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.454+0000 7f3f1e59c700 1 -- 172.21.15.93:0/3648452899 learned_addr learned my addr 172.21.15.93:0/3648452899 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:10.006 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.454+0000 7f3f1e59c700 1 -- 172.21.15.93:0/3648452899 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f201063a0 con 0x7f3f20105910 2023-11-16T16:39:10.006 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.455+0000 7f3f1e59c700 1 --2- 172.21.15.93:0/3648452899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 0x7f3f20105d20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f100098a0 tx=0x7f3f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2693bda1c0bf200d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:10.007 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.455+0000 7f3f1d59a700 1 -- 172.21.15.93:0/3648452899 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f10004160 con 0x7f3f20105910 2023-11-16T16:39:10.007 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.455+0000 7f3f1d59a700 1 -- 172.21.15.93:0/3648452899 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f100042c0 con 0x7f3f20105910 2023-11-16T16:39:10.007 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.455+0000 7f3f1d59a700 1 -- 172.21.15.93:0/3648452899 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f10011a00 con 0x7f3f20105910 2023-11-16T16:39:10.007 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.456+0000 7f3f24e34700 1 -- 172.21.15.93:0/3648452899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 msgr2=0x7f3f20105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:10.007 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.456+0000 7f3f24e34700 1 --2- 172.21.15.93:0/3648452899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 0x7f3f20105d20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f100098a0 tx=0x7f3f10009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.007 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.456+0000 7f3f24e34700 1 -- 172.21.15.93:0/3648452899 shutdown_connections 2023-11-16T16:39:10.007 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.456+0000 7f3f24e34700 1 --2- 172.21.15.93:0/3648452899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 0x7f3f20105d20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.007 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.456+0000 7f3f24e34700 1 -- 172.21.15.93:0/3648452899 >> 172.21.15.93:0/3648452899 conn(0x7f3f20100fa0 msgr2=0x7f3f201033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:10.007 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.456+0000 7f3f24e34700 1 -- 172.21.15.93:0/3648452899 shutdown_connections 2023-11-16T16:39:10.007 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.456+0000 7f3f24e34700 1 -- 172.21.15.93:0/3648452899 wait complete. 2023-11-16T16:39:10.007 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.457+0000 7f3f24e34700 1 Processor -- start 2023-11-16T16:39:10.008 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.458+0000 7f3f24e34700 1 -- start start 2023-11-16T16:39:10.008 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.458+0000 7f3f24e34700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 0x7f3f20199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:10.008 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.458+0000 7f3f24e34700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f20199a90 con 0x7f3f20105910 2023-11-16T16:39:10.008 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.458+0000 7f3f1e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 0x7f3f20199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:10.008 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.458+0000 7f3f1e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 0x7f3f20199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35970/0 (socket says 172.21.15.93:35970) 2023-11-16T16:39:10.008 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.458+0000 7f3f1e59c700 1 -- 172.21.15.93:0/1075841379 learned_addr learned my addr 172.21.15.93:0/1075841379 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:10.008 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.459+0000 7f3f1e59c700 1 -- 172.21.15.93:0/1075841379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f10009550 con 0x7f3f20105910 2023-11-16T16:39:10.008 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.459+0000 7f3f1e59c700 1 --2- 172.21.15.93:0/1075841379 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 0x7f3f20199550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3f10006af0 tx=0x7f3f10011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:10.008 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.459+0000 7f3f0f7fe700 1 -- 172.21.15.93:0/1075841379 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f10003a40 con 0x7f3f20105910 2023-11-16T16:39:10.008 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.459+0000 7f3f0f7fe700 1 -- 172.21.15.93:0/1075841379 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f10003ba0 con 0x7f3f20105910 2023-11-16T16:39:10.010 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.459+0000 7f3f0f7fe700 1 -- 172.21.15.93:0/1075841379 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f1001a6e0 con 0x7f3f20105910 2023-11-16T16:39:10.010 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.459+0000 7f3f24e34700 1 -- 172.21.15.93:0/1075841379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f20107960 con 0x7f3f20105910 2023-11-16T16:39:10.010 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.459+0000 7f3f24e34700 1 -- 172.21.15.93:0/1075841379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f201077d0 con 0x7f3f20105910 2023-11-16T16:39:10.010 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.460+0000 7f3f0f7fe700 1 -- 172.21.15.93:0/1075841379 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3f1001ab60 con 0x7f3f20105910 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.460+0000 7f3f0f7fe700 1 -- 172.21.15.93:0/1075841379 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3f10019700 con 0x7f3f20105910 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.460+0000 7f3f24e34700 1 -- 172.21.15.93:0/1075841379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f20062120 con 0x7f3f20105910 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.463+0000 7f3f0f7fe700 1 -- 172.21.15.93:0/1075841379 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f3f10024a60 con 0x7f3f20105910 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.540+0000 7f3f24e34700 1 -- 172.21.15.93:0/1075841379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3f20199fd0 con 0x7f3f20105910 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.540+0000 7f3f0f7fe700 1 -- 172.21.15.93:0/1075841379 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f3f10024c00 con 0x7f3f20105910 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.541+0000 7f3f24e34700 1 -- 172.21.15.93:0/1075841379 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 msgr2=0x7f3f20199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.541+0000 7f3f24e34700 1 --2- 172.21.15.93:0/1075841379 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 0x7f3f20199550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3f10006af0 tx=0x7f3f10011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.541+0000 7f3f24e34700 1 -- 172.21.15.93:0/1075841379 shutdown_connections 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.541+0000 7f3f24e34700 1 --2- 172.21.15.93:0/1075841379 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f20105910 0x7f3f20199550 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.541+0000 7f3f24e34700 1 -- 172.21.15.93:0/1075841379 >> 172.21.15.93:0/1075841379 conn(0x7f3f20100fa0 msgr2=0x7f3f20101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.541+0000 7f3f24e34700 1 -- 172.21.15.93:0/1075841379 shutdown_connections 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:09.541+0000 7f3f24e34700 1 -- 172.21.15.93:0/1075841379 wait complete. 2023-11-16T16:39:10.011 INFO:teuthology.orchestra.run.smithi093.stdout:mgr not available, waiting (3/15)... 2023-11-16T16:39:10.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:10 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:10.011+0000 7fb46fe91200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:39:11.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:11.064+0000 7fb46fe91200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:39:11.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: Activating manager daemon x 2023-11-16T16:39:11.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: mgrmap e2: x(active, starting, since 0.00411806s) 2023-11-16T16:39:11.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: from='mgr.14100 172.21.15.93:0/2962119083' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-16T16:39:11.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: from='mgr.14100 172.21.15.93:0/2962119083' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-16T16:39:11.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: from='mgr.14100 172.21.15.93:0/2962119083' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-16T16:39:11.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: from='mgr.14100 172.21.15.93:0/2962119083' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:39:11.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: from='mgr.14100 172.21.15.93:0/2962119083' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-16T16:39:11.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: Manager daemon x is now available 2023-11-16T16:39:11.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: from='mgr.14100 172.21.15.93:0/2962119083' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-16T16:39:11.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: from='mgr.14100 172.21.15.93:0/2962119083' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-16T16:39:11.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: from='mgr.14100 172.21.15.93:0/2962119083' entity='mgr.x' 2023-11-16T16:39:11.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: from='mgr.14100 172.21.15.93:0/2962119083' entity='mgr.x' 2023-11-16T16:39:11.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:11 smithi093 ceph-mon[97863]: from='mgr.14100 172.21.15.93:0/2962119083' entity='mgr.x' 2023-11-16T16:39:13.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:13 smithi093 ceph-mon[97863]: mgrmap e3: x(active, since 1.01s) 2023-11-16T16:39:13.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:13 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/624877438' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-16T16:39:13.293 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:39:13.293 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:39:13.293 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsid": "6a59c1a8-849e-11ee-8dbe-212e2dc638e7", 2023-11-16T16:39:13.293 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "health": { 2023-11-16T16:39:13.293 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-16T16:39:13.293 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-16T16:39:13.293 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-16T16:39:13.293 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:13.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-16T16:39:13.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-16T16:39:13.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 0 2023-11-16T16:39:13.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:39:13.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-16T16:39:13.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "a" 2023-11-16T16:39:13.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:39:13.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-11-16T16:39:13.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-16T16:39:13.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:13.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-16T16:39:13.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-16T16:39:13.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:13.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-16T16:39:13.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:13.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-16T16:39:13.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-16T16:39:13.307 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-16T16:39:13.307 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-16T16:39:13.307 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-16T16:39:13.307 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-16T16:39:13.307 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:13.307 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-16T16:39:13.307 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-16T16:39:13.308 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-16T16:39:13.308 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-16T16:39:13.308 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-16T16:39:13.308 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-16T16:39:13.308 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-16T16:39:13.308 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-16T16:39:13.308 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-16T16:39:13.308 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:13.308 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-16T16:39:13.309 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:13.309 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-16T16:39:13.309 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-16T16:39:13.309 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:13.309 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-16T16:39:13.309 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-16T16:39:13.309 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-16T16:39:13.309 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-16T16:39:13.309 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-16T16:39:13.310 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-16T16:39:13.310 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "restful" 2023-11-16T16:39:13.310 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:39:13.310 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:39:13.310 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:13.310 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-16T16:39:13.310 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:39:13.310 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modified": "2023-11-16T16:38:54.903744+0000", 2023-11-16T16:39:13.310 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:39:13.311 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:39:13.311 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-16T16:39:13.311 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:39:13.311 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.527+0000 7fb3c8483700 1 Processor -- start 2023-11-16T16:39:13.311 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.527+0000 7fb3c8483700 1 -- start start 2023-11-16T16:39:13.311 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.528+0000 7fb3c8483700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 0x7fb3c007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:13.311 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.528+0000 7fb3c8483700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c007a820 con 0x7fb3c007bde0 2023-11-16T16:39:13.311 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.528+0000 7fb3c621f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 0x7fb3c007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:13.311 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.528+0000 7fb3c621f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 0x7fb3c007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36048/0 (socket says 172.21.15.93:36048) 2023-11-16T16:39:13.311 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.528+0000 7fb3c621f700 1 -- 172.21.15.93:0/3316610484 learned_addr learned my addr 172.21.15.93:0/3316610484 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:13.312 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.528+0000 7fb3c621f700 1 -- 172.21.15.93:0/3316610484 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c007a960 con 0x7fb3c007bde0 2023-11-16T16:39:13.312 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.529+0000 7fb3c621f700 1 --2- 172.21.15.93:0/3316610484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 0x7fb3c007a2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0098a0 tx=0x7fb3bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321d839096dbaff9 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:13.312 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.529+0000 7fb3c521d700 1 -- 172.21.15.93:0/3316610484 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3bc004160 con 0x7fb3c007bde0 2023-11-16T16:39:13.312 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.529+0000 7fb3c521d700 1 -- 172.21.15.93:0/3316610484 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb3bc0042c0 con 0x7fb3c007bde0 2023-11-16T16:39:13.312 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.529+0000 7fb3c521d700 1 -- 172.21.15.93:0/3316610484 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3bc011a00 con 0x7fb3c007bde0 2023-11-16T16:39:13.312 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.529+0000 7fb3c8483700 1 -- 172.21.15.93:0/3316610484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 msgr2=0x7fb3c007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:13.312 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.529+0000 7fb3c8483700 1 --2- 172.21.15.93:0/3316610484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 0x7fb3c007a2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0098a0 tx=0x7fb3bc009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:13.312 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.530+0000 7fb3c8483700 1 -- 172.21.15.93:0/3316610484 shutdown_connections 2023-11-16T16:39:13.312 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.530+0000 7fb3c8483700 1 --2- 172.21.15.93:0/3316610484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 0x7fb3c007a2e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:13.313 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.530+0000 7fb3c8483700 1 -- 172.21.15.93:0/3316610484 >> 172.21.15.93:0/3316610484 conn(0x7fb3c0100fa0 msgr2=0x7fb3c01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:13.313 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.530+0000 7fb3c8483700 1 -- 172.21.15.93:0/3316610484 shutdown_connections 2023-11-16T16:39:13.313 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.530+0000 7fb3c8483700 1 -- 172.21.15.93:0/3316610484 wait complete. 2023-11-16T16:39:13.313 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.531+0000 7fb3c8483700 1 Processor -- start 2023-11-16T16:39:13.313 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.531+0000 7fb3c8483700 1 -- start start 2023-11-16T16:39:13.313 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.531+0000 7fb3c8483700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 0x7fb3c019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:13.313 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.531+0000 7fb3c8483700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c019bcf0 con 0x7fb3c007bde0 2023-11-16T16:39:13.313 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.531+0000 7fb3c621f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 0x7fb3c019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:13.313 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.531+0000 7fb3c621f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 0x7fb3c019b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36056/0 (socket says 172.21.15.93:36056) 2023-11-16T16:39:13.314 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.531+0000 7fb3c621f700 1 -- 172.21.15.93:0/624877438 learned_addr learned my addr 172.21.15.93:0/624877438 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:13.314 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.532+0000 7fb3c621f700 1 -- 172.21.15.93:0/624877438 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc009550 con 0x7fb3c007bde0 2023-11-16T16:39:13.314 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.532+0000 7fb3c621f700 1 --2- 172.21.15.93:0/624877438 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 0x7fb3c019b7b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009fb0 tx=0x7fb3bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:13.314 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.532+0000 7fb3b77fe700 1 -- 172.21.15.93:0/624877438 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3bc003930 con 0x7fb3c007bde0 2023-11-16T16:39:13.314 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.532+0000 7fb3b77fe700 1 -- 172.21.15.93:0/624877438 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb3bc003a90 con 0x7fb3c007bde0 2023-11-16T16:39:13.314 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.532+0000 7fb3b77fe700 1 -- 172.21.15.93:0/624877438 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3bc0195e0 con 0x7fb3c007bde0 2023-11-16T16:39:13.314 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.532+0000 7fb3c8483700 1 -- 172.21.15.93:0/624877438 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c010be00 con 0x7fb3c007bde0 2023-11-16T16:39:13.314 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.532+0000 7fb3c8483700 1 -- 172.21.15.93:0/624877438 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c010bc70 con 0x7fb3c007bde0 2023-11-16T16:39:13.314 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.533+0000 7fb3b77fe700 1 -- 172.21.15.93:0/624877438 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7fb3bc01a9d0 con 0x7fb3c007bde0 2023-11-16T16:39:13.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.533+0000 7fb3c8483700 1 -- 172.21.15.93:0/624877438 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c0062120 con 0x7fb3c007bde0 2023-11-16T16:39:13.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.534+0000 7fb3b77fe700 1 --2- 172.21.15.93:0/624877438 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fb3ac038660 0x7fb3ac03ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:13.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.534+0000 7fb3b77fe700 1 -- 172.21.15.93:0/624877438 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb3bc04c220 con 0x7fb3c007bde0 2023-11-16T16:39:13.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.534+0000 7fb3c5a1e700 1 --2- 172.21.15.93:0/624877438 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fb3ac038660 0x7fb3ac03ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:13.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.535+0000 7fb3c5a1e700 1 --2- 172.21.15.93:0/624877438 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fb3ac038660 0x7fb3ac03ab10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0006fd0 tx=0x7fb3b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:13.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.540+0000 7fb3b77fe700 1 -- 172.21.15.93:0/624877438 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb3bc003c00 con 0x7fb3c007bde0 2023-11-16T16:39:13.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.811+0000 7fb3c8483700 1 -- 172.21.15.93:0/624877438 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb3c019c480 con 0x7fb3c007bde0 2023-11-16T16:39:13.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.811+0000 7fb3b77fe700 1 -- 172.21.15.93:0/624877438 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fb3bc023b40 con 0x7fb3c007bde0 2023-11-16T16:39:13.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.813+0000 7fb3c8483700 1 -- 172.21.15.93:0/624877438 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fb3ac038660 msgr2=0x7fb3ac03ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:13.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.813+0000 7fb3c8483700 1 --2- 172.21.15.93:0/624877438 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fb3ac038660 0x7fb3ac03ab10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0006fd0 tx=0x7fb3b0006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:13.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.814+0000 7fb3c8483700 1 -- 172.21.15.93:0/624877438 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 msgr2=0x7fb3c019b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:13.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.814+0000 7fb3c8483700 1 --2- 172.21.15.93:0/624877438 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 0x7fb3c019b7b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009fb0 tx=0x7fb3bc011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:13.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.814+0000 7fb3c8483700 1 -- 172.21.15.93:0/624877438 shutdown_connections 2023-11-16T16:39:13.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.814+0000 7fb3c8483700 1 --2- 172.21.15.93:0/624877438 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fb3ac038660 0x7fb3ac03ab10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:13.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.814+0000 7fb3c8483700 1 --2- 172.21.15.93:0/624877438 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3c007bde0 0x7fb3c019b7b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:13.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.814+0000 7fb3c8483700 1 -- 172.21.15.93:0/624877438 >> 172.21.15.93:0/624877438 conn(0x7fb3c0100fa0 msgr2=0x7fb3c0101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:13.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.814+0000 7fb3c8483700 1 -- 172.21.15.93:0/624877438 shutdown_connections 2023-11-16T16:39:13.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:12.814+0000 7fb3c8483700 1 -- 172.21.15.93:0/624877438 wait complete. 2023-11-16T16:39:13.316 INFO:teuthology.orchestra.run.smithi093.stdout:mgr is available 2023-11-16T16:39:14.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:14 smithi093 ceph-mon[97863]: mgrmap e4: x(active, since 2s) 2023-11-16T16:39:14.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:14 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/706993138' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-16T16:39:14.486 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:39:14.486 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [global] 2023-11-16T16:39:14.487 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout fsid = 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:39:14.487 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-16T16:39:14.487 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-16T16:39:14.487 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-16T16:39:14.487 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-16T16:39:14.487 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:39:14.487 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-16T16:39:14.487 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-16T16:39:14.487 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-11-16T16:39:14.487 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [osd] 2023-11-16T16:39:14.488 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-16T16:39:14.488 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-16T16:39:14.488 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-16T16:39:14.488 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-16T16:39:14.488 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-16T16:39:14.488 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.836+0000 7fe364f21700 1 Processor -- start 2023-11-16T16:39:14.488 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.836+0000 7fe364f21700 1 -- start start 2023-11-16T16:39:14.488 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.837+0000 7fe364f21700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 0x7fe36007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:14.488 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.837+0000 7fe364f21700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36007a820 con 0x7fe36007bde0 2023-11-16T16:39:14.488 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.837+0000 7fe35e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 0x7fe36007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:14.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.837+0000 7fe35e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 0x7fe36007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36062/0 (socket says 172.21.15.93:36062) 2023-11-16T16:39:14.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.837+0000 7fe35e59c700 1 -- 172.21.15.93:0/956813573 learned_addr learned my addr 172.21.15.93:0/956813573 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:14.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.837+0000 7fe35e59c700 1 -- 172.21.15.93:0/956813573 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36007a960 con 0x7fe36007bde0 2023-11-16T16:39:14.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.838+0000 7fe35e59c700 1 --2- 172.21.15.93:0/956813573 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 0x7fe36007a2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe3500098a0 tx=0x7fe350009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82686ce52211b37 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:14.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.838+0000 7fe35d59a700 1 -- 172.21.15.93:0/956813573 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe350004160 con 0x7fe36007bde0 2023-11-16T16:39:14.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.838+0000 7fe35d59a700 1 -- 172.21.15.93:0/956813573 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe3500042c0 con 0x7fe36007bde0 2023-11-16T16:39:14.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.838+0000 7fe35d59a700 1 -- 172.21.15.93:0/956813573 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe350011a00 con 0x7fe36007bde0 2023-11-16T16:39:14.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.838+0000 7fe364f21700 1 -- 172.21.15.93:0/956813573 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 msgr2=0x7fe36007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:14.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.838+0000 7fe364f21700 1 --2- 172.21.15.93:0/956813573 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 0x7fe36007a2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe3500098a0 tx=0x7fe350009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:14.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.839+0000 7fe364f21700 1 -- 172.21.15.93:0/956813573 shutdown_connections 2023-11-16T16:39:14.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.839+0000 7fe364f21700 1 --2- 172.21.15.93:0/956813573 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 0x7fe36007a2e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:14.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.839+0000 7fe364f21700 1 -- 172.21.15.93:0/956813573 >> 172.21.15.93:0/956813573 conn(0x7fe360100fa0 msgr2=0x7fe3601033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:14.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.839+0000 7fe364f21700 1 -- 172.21.15.93:0/956813573 shutdown_connections 2023-11-16T16:39:14.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.839+0000 7fe364f21700 1 -- 172.21.15.93:0/956813573 wait complete. 2023-11-16T16:39:14.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.840+0000 7fe364f21700 1 Processor -- start 2023-11-16T16:39:14.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.840+0000 7fe364f21700 1 -- start start 2023-11-16T16:39:14.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.840+0000 7fe364f21700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 0x7fe36019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:14.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.840+0000 7fe364f21700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36019bc80 con 0x7fe36007bde0 2023-11-16T16:39:14.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.840+0000 7fe35e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 0x7fe36019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:14.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.840+0000 7fe35e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 0x7fe36019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36066/0 (socket says 172.21.15.93:36066) 2023-11-16T16:39:14.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.840+0000 7fe35e59c700 1 -- 172.21.15.93:0/706993138 learned_addr learned my addr 172.21.15.93:0/706993138 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:14.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.841+0000 7fe35e59c700 1 -- 172.21.15.93:0/706993138 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350009550 con 0x7fe36007bde0 2023-11-16T16:39:14.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.841+0000 7fe35e59c700 1 --2- 172.21.15.93:0/706993138 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 0x7fe36019b740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe350006af0 tx=0x7fe350011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:14.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.841+0000 7fe34f7fe700 1 -- 172.21.15.93:0/706993138 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe350003a40 con 0x7fe36007bde0 2023-11-16T16:39:14.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.841+0000 7fe34f7fe700 1 -- 172.21.15.93:0/706993138 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe350003ba0 con 0x7fe36007bde0 2023-11-16T16:39:14.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.841+0000 7fe364f21700 1 -- 172.21.15.93:0/706993138 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36010be00 con 0x7fe36007bde0 2023-11-16T16:39:14.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.841+0000 7fe34f7fe700 1 -- 172.21.15.93:0/706993138 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe35001a700 con 0x7fe36007bde0 2023-11-16T16:39:14.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.841+0000 7fe364f21700 1 -- 172.21.15.93:0/706993138 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36010bc70 con 0x7fe36007bde0 2023-11-16T16:39:14.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.842+0000 7fe34f7fe700 1 -- 172.21.15.93:0/706993138 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fe350024ad0 con 0x7fe36007bde0 2023-11-16T16:39:14.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.842+0000 7fe364f21700 1 -- 172.21.15.93:0/706993138 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe360062120 con 0x7fe36007bde0 2023-11-16T16:39:14.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.843+0000 7fe34f7fe700 1 --2- 172.21.15.93:0/706993138 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fe3480387d0 0x7fe34803ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:14.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.843+0000 7fe34f7fe700 1 -- 172.21.15.93:0/706993138 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe35004c630 con 0x7fe36007bde0 2023-11-16T16:39:14.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.843+0000 7fe35dd9b700 1 --2- 172.21.15.93:0/706993138 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fe3480387d0 0x7fe34803ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:14.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.844+0000 7fe35dd9b700 1 --2- 172.21.15.93:0/706993138 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fe3480387d0 0x7fe34803ac80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe354006fd0 tx=0x7fe354006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:14.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:13.848+0000 7fe34f7fe700 1 -- 172.21.15.93:0/706993138 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe350018bd0 con 0x7fe36007bde0 2023-11-16T16:39:14.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:14.043+0000 7fe364f21700 1 -- 172.21.15.93:0/706993138 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe36019c410 con 0x7fe36007bde0 2023-11-16T16:39:14.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:14.044+0000 7fe34f7fe700 1 -- 172.21.15.93:0/706993138 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fe3500187c0 con 0x7fe36007bde0 2023-11-16T16:39:14.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:14.046+0000 7fe364f21700 1 -- 172.21.15.93:0/706993138 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fe3480387d0 msgr2=0x7fe34803ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:14.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:14.046+0000 7fe364f21700 1 --2- 172.21.15.93:0/706993138 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fe3480387d0 0x7fe34803ac80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe354006fd0 tx=0x7fe354006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:14.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:14.046+0000 7fe364f21700 1 -- 172.21.15.93:0/706993138 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 msgr2=0x7fe36019b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:14.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:14.046+0000 7fe364f21700 1 --2- 172.21.15.93:0/706993138 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 0x7fe36019b740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe350006af0 tx=0x7fe350011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:14.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:14.046+0000 7fe364f21700 1 -- 172.21.15.93:0/706993138 shutdown_connections 2023-11-16T16:39:14.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:14.046+0000 7fe364f21700 1 --2- 172.21.15.93:0/706993138 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fe3480387d0 0x7fe34803ac80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:14.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:14.046+0000 7fe364f21700 1 --2- 172.21.15.93:0/706993138 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe36007bde0 0x7fe36019b740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:14.493 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:14.046+0000 7fe364f21700 1 -- 172.21.15.93:0/706993138 >> 172.21.15.93:0/706993138 conn(0x7fe360100fa0 msgr2=0x7fe360101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:14.493 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:14.046+0000 7fe364f21700 1 -- 172.21.15.93:0/706993138 shutdown_connections 2023-11-16T16:39:14.493 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:14.046+0000 7fe364f21700 1 -- 172.21.15.93:0/706993138 wait complete. 2023-11-16T16:39:14.493 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling cephadm module... 2023-11-16T16:39:16.423 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:16 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/803995018' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-11-16T16:39:16.423 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:16 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: ignoring --setuser ceph since I am not root 2023-11-16T16:39:16.423 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:16 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: ignoring --setgroup ceph since I am not root 2023-11-16T16:39:16.423 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:16 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:16.420+0000 7fab2864e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:39:16.758 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:16 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:16.523+0000 7fab2864e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:39:16.758 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:16 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:16.637+0000 7fab2864e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:39:16.758 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:16 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:16.756+0000 7fab2864e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:39:16.767 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.025+0000 7f6088fc8700 1 Processor -- start 2023-11-16T16:39:16.768 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.025+0000 7f6088fc8700 1 -- start start 2023-11-16T16:39:16.768 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.025+0000 7f6088fc8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 0x7f608407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:16.768 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.025+0000 7f6088fc8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608407a820 con 0x7f608407bde0 2023-11-16T16:39:16.768 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.025+0000 7f608259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 0x7f608407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:16.768 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.025+0000 7f608259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 0x7f608407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36080/0 (socket says 172.21.15.93:36080) 2023-11-16T16:39:16.768 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.025+0000 7f608259c700 1 -- 172.21.15.93:0/1305805969 learned_addr learned my addr 172.21.15.93:0/1305805969 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:16.769 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.026+0000 7f608259c700 1 -- 172.21.15.93:0/1305805969 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608407a960 con 0x7f608407bde0 2023-11-16T16:39:16.769 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.026+0000 7f608259c700 1 --2- 172.21.15.93:0/1305805969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 0x7f608407a2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f60740098a0 tx=0x7f6074009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb633a75719cda76 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:16.769 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.026+0000 7f608159a700 1 -- 172.21.15.93:0/1305805969 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6074004160 con 0x7f608407bde0 2023-11-16T16:39:16.769 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.026+0000 7f608159a700 1 -- 172.21.15.93:0/1305805969 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60740042c0 con 0x7f608407bde0 2023-11-16T16:39:16.769 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.026+0000 7f608159a700 1 -- 172.21.15.93:0/1305805969 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6074011a00 con 0x7f608407bde0 2023-11-16T16:39:16.769 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.027+0000 7f6088fc8700 1 -- 172.21.15.93:0/1305805969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 msgr2=0x7f608407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:16.770 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.027+0000 7f6088fc8700 1 --2- 172.21.15.93:0/1305805969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 0x7f608407a2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f60740098a0 tx=0x7f6074009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:16.770 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.027+0000 7f6088fc8700 1 -- 172.21.15.93:0/1305805969 shutdown_connections 2023-11-16T16:39:16.770 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.027+0000 7f6088fc8700 1 --2- 172.21.15.93:0/1305805969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 0x7f608407a2e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:16.770 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.027+0000 7f6088fc8700 1 -- 172.21.15.93:0/1305805969 >> 172.21.15.93:0/1305805969 conn(0x7f6084100fa0 msgr2=0x7f60841033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:16.770 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.027+0000 7f6088fc8700 1 -- 172.21.15.93:0/1305805969 shutdown_connections 2023-11-16T16:39:16.770 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.027+0000 7f6088fc8700 1 -- 172.21.15.93:0/1305805969 wait complete. 2023-11-16T16:39:16.771 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.028+0000 7f6088fc8700 1 Processor -- start 2023-11-16T16:39:16.771 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.028+0000 7f6088fc8700 1 -- start start 2023-11-16T16:39:16.771 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.029+0000 7f6088fc8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 0x7f6084108d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:16.771 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.029+0000 7f6088fc8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6084109280 con 0x7f608407bde0 2023-11-16T16:39:16.771 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.029+0000 7f608259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 0x7f6084108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:16.771 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.029+0000 7f608259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 0x7f6084108d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36090/0 (socket says 172.21.15.93:36090) 2023-11-16T16:39:16.772 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.029+0000 7f608259c700 1 -- 172.21.15.93:0/803995018 learned_addr learned my addr 172.21.15.93:0/803995018 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:16.772 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.029+0000 7f608259c700 1 -- 172.21.15.93:0/803995018 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6074009550 con 0x7f608407bde0 2023-11-16T16:39:16.772 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.029+0000 7f608259c700 1 --2- 172.21.15.93:0/803995018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 0x7f6084108d40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6074006af0 tx=0x7f6074011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:16.772 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.030+0000 7f60737fe700 1 -- 172.21.15.93:0/803995018 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6074003a40 con 0x7f608407bde0 2023-11-16T16:39:16.772 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.030+0000 7f60737fe700 1 -- 172.21.15.93:0/803995018 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6074003ba0 con 0x7f608407bde0 2023-11-16T16:39:16.772 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.030+0000 7f60737fe700 1 -- 172.21.15.93:0/803995018 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f607401a700 con 0x7f608407bde0 2023-11-16T16:39:16.772 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.030+0000 7f6088fc8700 1 -- 172.21.15.93:0/803995018 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608410be00 con 0x7f608407bde0 2023-11-16T16:39:16.773 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.030+0000 7f6088fc8700 1 -- 172.21.15.93:0/803995018 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608410bc70 con 0x7f608407bde0 2023-11-16T16:39:16.773 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.031+0000 7f60737fe700 1 -- 172.21.15.93:0/803995018 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f6074024430 con 0x7f608407bde0 2023-11-16T16:39:16.773 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.031+0000 7f6088fc8700 1 -- 172.21.15.93:0/803995018 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6084062120 con 0x7f608407bde0 2023-11-16T16:39:16.773 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.031+0000 7f60737fe700 1 --2- 172.21.15.93:0/803995018 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f606c0387d0 0x7f606c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:16.773 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.031+0000 7f60737fe700 1 -- 172.21.15.93:0/803995018 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f607404c620 con 0x7f608407bde0 2023-11-16T16:39:16.773 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.031+0000 7f6081d9b700 1 --2- 172.21.15.93:0/803995018 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f606c0387d0 0x7f606c03ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:16.774 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.032+0000 7f6081d9b700 1 --2- 172.21.15.93:0/803995018 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f606c0387d0 0x7f606c03ac80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6078006fd0 tx=0x7f6078006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:16.774 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.036+0000 7f60737fe700 1 -- 172.21.15.93:0/803995018 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6074024ad0 con 0x7f608407bde0 2023-11-16T16:39:16.774 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:15.285+0000 7f6088fc8700 1 -- 172.21.15.93:0/803995018 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f6084105bc0 con 0x7f608407bde0 2023-11-16T16:39:16.774 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:16.154+0000 7f60737fe700 1 -- 172.21.15.93:0/803995018 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f607402d3f0 con 0x7f608407bde0 2023-11-16T16:39:16.774 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:16.156+0000 7f60737fe700 1 -- 172.21.15.93:0/803995018 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f6074021da0 con 0x7f608407bde0 2023-11-16T16:39:16.774 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:16.159+0000 7f6088fc8700 1 -- 172.21.15.93:0/803995018 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f606c0387d0 msgr2=0x7f606c03ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:16.775 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:16.159+0000 7f6088fc8700 1 --2- 172.21.15.93:0/803995018 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f606c0387d0 0x7f606c03ac80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6078006fd0 tx=0x7f6078006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:16.775 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:16.159+0000 7f6088fc8700 1 -- 172.21.15.93:0/803995018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 msgr2=0x7f6084108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:16.775 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:16.159+0000 7f6088fc8700 1 --2- 172.21.15.93:0/803995018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 0x7f6084108d40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6074006af0 tx=0x7f6074011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:16.775 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:16.160+0000 7f6088fc8700 1 -- 172.21.15.93:0/803995018 shutdown_connections 2023-11-16T16:39:16.775 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:16.160+0000 7f6088fc8700 1 --2- 172.21.15.93:0/803995018 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f606c0387d0 0x7f606c03ac80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:16.775 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:16.160+0000 7f6088fc8700 1 --2- 172.21.15.93:0/803995018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f608407bde0 0x7f6084108d40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:16.775 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:16.160+0000 7f6088fc8700 1 -- 172.21.15.93:0/803995018 >> 172.21.15.93:0/803995018 conn(0x7f6084100fa0 msgr2=0x7f6084101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:16.776 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:16.160+0000 7f6088fc8700 1 -- 172.21.15.93:0/803995018 shutdown_connections 2023-11-16T16:39:16.776 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:16.160+0000 7f6088fc8700 1 -- 172.21.15.93:0/803995018 wait complete. 2023-11-16T16:39:17.093 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:16 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:16.985+0000 7fab2864e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:39:17.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:17 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/803995018' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-11-16T16:39:17.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:17 smithi093 ceph-mon[97863]: mgrmap e5: x(active, since 5s) 2023-11-16T16:39:17.664 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:17 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:17.400+0000 7fab2864e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:39:17.664 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:17 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:17.519+0000 7fab2864e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:39:18.062 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:17 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:17.689+0000 7fab2864e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:39:18.062 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:17 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:17.822+0000 7fab2864e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:39:18.152 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:39:18.152 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-11-16T16:39:18.152 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-16T16:39:18.152 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-16T16:39:18.152 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-16T16:39:18.152 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:39:18.153 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.368+0000 7fbaea440700 1 Processor -- start 2023-11-16T16:39:18.153 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.369+0000 7fbaea440700 1 -- start start 2023-11-16T16:39:18.153 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.369+0000 7fbaea440700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 0x7fbae407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:18.153 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.369+0000 7fbaea440700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae407a820 con 0x7fbae407bde0 2023-11-16T16:39:18.153 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.369+0000 7fbae3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 0x7fbae407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:18.154 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.369+0000 7fbae3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 0x7fbae407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37638/0 (socket says 172.21.15.93:37638) 2023-11-16T16:39:18.154 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.369+0000 7fbae3fff700 1 -- 172.21.15.93:0/1067230412 learned_addr learned my addr 172.21.15.93:0/1067230412 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:18.154 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.370+0000 7fbae3fff700 1 -- 172.21.15.93:0/1067230412 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae407a960 con 0x7fbae407bde0 2023-11-16T16:39:18.154 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.370+0000 7fbae3fff700 1 --2- 172.21.15.93:0/1067230412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 0x7fbae407a2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbad40098a0 tx=0x7fbad4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f0e75826074079 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:18.154 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.370+0000 7fbae2ffd700 1 -- 172.21.15.93:0/1067230412 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad4004160 con 0x7fbae407bde0 2023-11-16T16:39:18.154 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.370+0000 7fbae2ffd700 1 -- 172.21.15.93:0/1067230412 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbad40042c0 con 0x7fbae407bde0 2023-11-16T16:39:18.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.370+0000 7fbae2ffd700 1 -- 172.21.15.93:0/1067230412 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad4011a00 con 0x7fbae407bde0 2023-11-16T16:39:18.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.371+0000 7fbaea440700 1 -- 172.21.15.93:0/1067230412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 msgr2=0x7fbae407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:18.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.371+0000 7fbaea440700 1 --2- 172.21.15.93:0/1067230412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 0x7fbae407a2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbad40098a0 tx=0x7fbad4009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:18.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.371+0000 7fbaea440700 1 -- 172.21.15.93:0/1067230412 shutdown_connections 2023-11-16T16:39:18.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.371+0000 7fbaea440700 1 --2- 172.21.15.93:0/1067230412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 0x7fbae407a2e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:18.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.371+0000 7fbaea440700 1 -- 172.21.15.93:0/1067230412 >> 172.21.15.93:0/1067230412 conn(0x7fbae4100fa0 msgr2=0x7fbae41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:18.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.371+0000 7fbaea440700 1 -- 172.21.15.93:0/1067230412 shutdown_connections 2023-11-16T16:39:18.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.371+0000 7fbaea440700 1 -- 172.21.15.93:0/1067230412 wait complete. 2023-11-16T16:39:18.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.372+0000 7fbaea440700 1 Processor -- start 2023-11-16T16:39:18.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.372+0000 7fbaea440700 1 -- start start 2023-11-16T16:39:18.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.373+0000 7fbaea440700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 0x7fbae419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:18.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.373+0000 7fbaea440700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae419bcf0 con 0x7fbae407bde0 2023-11-16T16:39:18.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.373+0000 7fbae3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 0x7fbae419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:18.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.373+0000 7fbae3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 0x7fbae419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37654/0 (socket says 172.21.15.93:37654) 2023-11-16T16:39:18.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.373+0000 7fbae3fff700 1 -- 172.21.15.93:0/1202575248 learned_addr learned my addr 172.21.15.93:0/1202575248 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:18.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.373+0000 7fbae3fff700 1 -- 172.21.15.93:0/1202575248 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad4009550 con 0x7fbae407bde0 2023-11-16T16:39:18.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.373+0000 7fbae3fff700 1 --2- 172.21.15.93:0/1202575248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 0x7fbae419b7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009fb0 tx=0x7fbad4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:18.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.374+0000 7fbae17fa700 1 -- 172.21.15.93:0/1202575248 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad4003930 con 0x7fbae407bde0 2023-11-16T16:39:18.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.374+0000 7fbae17fa700 1 -- 172.21.15.93:0/1202575248 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbad4003a90 con 0x7fbae407bde0 2023-11-16T16:39:18.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.374+0000 7fbae17fa700 1 -- 172.21.15.93:0/1202575248 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad40195e0 con 0x7fbae407bde0 2023-11-16T16:39:18.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.374+0000 7fbaea440700 1 -- 172.21.15.93:0/1202575248 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae410be00 con 0x7fbae407bde0 2023-11-16T16:39:18.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.374+0000 7fbaea440700 1 -- 172.21.15.93:0/1202575248 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae410bc70 con 0x7fbae407bde0 2023-11-16T16:39:18.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.375+0000 7fbae17fa700 1 -- 172.21.15.93:0/1202575248 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fbad401a9d0 con 0x7fbae407bde0 2023-11-16T16:39:18.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.375+0000 7fbaea440700 1 -- 172.21.15.93:0/1202575248 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae4062120 con 0x7fbae407bde0 2023-11-16T16:39:18.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.375+0000 7fbae17fa700 1 --2- 172.21.15.93:0/1202575248 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fbacc038780 0x7fbacc03ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:18.159 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.375+0000 7fbae17fa700 1 -- 172.21.15.93:0/1202575248 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbad404c4e0 con 0x7fbae407bde0 2023-11-16T16:39:18.159 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.376+0000 7fbae37fe700 1 -- 172.21.15.93:0/1202575248 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fbacc038780 msgr2=0x7fbacc03ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/772471105 2023-11-16T16:39:18.159 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.376+0000 7fbae37fe700 1 --2- 172.21.15.93:0/1202575248 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fbacc038780 0x7fbacc03ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:39:18.159 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.380+0000 7fbae17fa700 1 -- 172.21.15.93:0/1202575248 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbad4023b40 con 0x7fbae407bde0 2023-11-16T16:39:18.159 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.577+0000 7fbae37fe700 1 -- 172.21.15.93:0/1202575248 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fbacc038780 msgr2=0x7fbacc03ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/772471105 2023-11-16T16:39:18.160 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.577+0000 7fbae37fe700 1 --2- 172.21.15.93:0/1202575248 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fbacc038780 0x7fbacc03ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T16:39:18.160 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.638+0000 7fbaea440700 1 -- 172.21.15.93:0/1202575248 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fbae419c230 con 0x7fbae407bde0 2023-11-16T16:39:18.160 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.638+0000 7fbae17fa700 1 -- 172.21.15.93:0/1202575248 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fbad4023b40 con 0x7fbae407bde0 2023-11-16T16:39:18.160 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.640+0000 7fbaea440700 1 -- 172.21.15.93:0/1202575248 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fbacc038780 msgr2=0x7fbacc03ac30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:39:18.160 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.640+0000 7fbaea440700 1 --2- 172.21.15.93:0/1202575248 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fbacc038780 0x7fbacc03ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:18.160 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.640+0000 7fbaea440700 1 -- 172.21.15.93:0/1202575248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 msgr2=0x7fbae419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:18.161 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.640+0000 7fbaea440700 1 --2- 172.21.15.93:0/1202575248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 0x7fbae419b7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009fb0 tx=0x7fbad4011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:18.161 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.640+0000 7fbaea440700 1 -- 172.21.15.93:0/1202575248 shutdown_connections 2023-11-16T16:39:18.161 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.640+0000 7fbaea440700 1 --2- 172.21.15.93:0/1202575248 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7fbacc038780 0x7fbacc03ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:18.161 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.640+0000 7fbaea440700 1 --2- 172.21.15.93:0/1202575248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbae407bde0 0x7fbae419b7b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:18.161 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.640+0000 7fbaea440700 1 -- 172.21.15.93:0/1202575248 >> 172.21.15.93:0/1202575248 conn(0x7fbae4100fa0 msgr2=0x7fbae4101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:18.161 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.641+0000 7fbaea440700 1 -- 172.21.15.93:0/1202575248 shutdown_connections 2023-11-16T16:39:18.161 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:17.641+0000 7fbaea440700 1 -- 172.21.15.93:0/1202575248 wait complete. 2023-11-16T16:39:18.162 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for the mgr to restart... 2023-11-16T16:39:18.162 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mgr epoch 5... 2023-11-16T16:39:18.375 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:18 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1202575248' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-16T16:39:18.650 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:18 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:18.373+0000 7fab2864e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:39:19.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:19 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:19.141+0000 7fab2864e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:39:19.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:19 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:19.348+0000 7fab2864e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:39:20.614 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:20 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:20.276+0000 7fab2864e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:39:20.615 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:20 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:20.395+0000 7fab2864e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:39:21.245 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:20 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:20.612+0000 7fab2864e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:39:21.245 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:20 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:20.969+0000 7fab2864e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:39:21.650 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:21 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:21.243+0000 7fab2864e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:39:21.650 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:21 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:21.433+0000 7fab2864e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:39:22.150 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:21 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:21.771+0000 7fab2864e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:39:23.096 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:22 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:22.684+0000 7fab2864e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:39:23.096 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:22 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:22.853+0000 7fab2864e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:39:23.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:23 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:23.094+0000 7fab2864e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:39:23.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:23 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:23.243+0000 7fab2864e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:39:23.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:23 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:23.517+0000 7fab2864e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:39:24.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:24.566+0000 7fab2864e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: Active manager daemon x restarted 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: Activating manager daemon x 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: osdmap e2: 0 total, 0 up, 0 in 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: mgrmap e6: x(active, starting, since 0.00367188s) 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: Manager daemon x is now available 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:24.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:24 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:25.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:25 smithi093 ceph-mon[97863]: Found migration_current of "None". Setting to last migration. 2023-11-16T16:39:25.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:25 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:25.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:25 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-16T16:39:25.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:25 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-16T16:39:25.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:25 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:25.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:25 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:25.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:25 smithi093 ceph-mon[97863]: mgrmap e7: x(active, since 1.00586s) 2023-11-16T16:39:26.050 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:39:26.050 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-11-16T16:39:26.050 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-16T16:39:26.050 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:39:26.050 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.724+0000 7f4047c96700 1 Processor -- start 2023-11-16T16:39:26.051 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.724+0000 7f4047c96700 1 -- start start 2023-11-16T16:39:26.051 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.725+0000 7f4047c96700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 0x7f404007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:26.051 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.725+0000 7f4047c96700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404007a840 con 0x7f404007be00 2023-11-16T16:39:26.051 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.725+0000 7f4045a32700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 0x7f404007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:26.051 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.725+0000 7f4045a32700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 0x7f404007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37664/0 (socket says 172.21.15.93:37664) 2023-11-16T16:39:26.051 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.725+0000 7f4045a32700 1 -- 172.21.15.93:0/3242024222 learned_addr learned my addr 172.21.15.93:0/3242024222 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:26.051 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.725+0000 7f4045a32700 1 -- 172.21.15.93:0/3242024222 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404007a980 con 0x7f404007be00 2023-11-16T16:39:26.051 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.726+0000 7f4045a32700 1 --2- 172.21.15.93:0/3242024222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 0x7f404007a300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f403c0098a0 tx=0x7f403c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b40792b5ea4a6ae server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:26.051 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.726+0000 7f4044a30700 1 -- 172.21.15.93:0/3242024222 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f403c004160 con 0x7f404007be00 2023-11-16T16:39:26.051 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.726+0000 7f4044a30700 1 -- 172.21.15.93:0/3242024222 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f403c0042c0 con 0x7f404007be00 2023-11-16T16:39:26.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.726+0000 7f4044a30700 1 -- 172.21.15.93:0/3242024222 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f403c011a00 con 0x7f404007be00 2023-11-16T16:39:26.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.726+0000 7f4047c96700 1 -- 172.21.15.93:0/3242024222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 msgr2=0x7f404007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:26.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.726+0000 7f4047c96700 1 --2- 172.21.15.93:0/3242024222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 0x7f404007a300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f403c0098a0 tx=0x7f403c009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:26.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.726+0000 7f4047c96700 1 -- 172.21.15.93:0/3242024222 shutdown_connections 2023-11-16T16:39:26.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.726+0000 7f4047c96700 1 --2- 172.21.15.93:0/3242024222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 0x7f404007a300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:26.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.726+0000 7f4047c96700 1 -- 172.21.15.93:0/3242024222 >> 172.21.15.93:0/3242024222 conn(0x7f4040100fa0 msgr2=0x7f40401033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:26.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.727+0000 7f4047c96700 1 -- 172.21.15.93:0/3242024222 shutdown_connections 2023-11-16T16:39:26.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.727+0000 7f4047c96700 1 -- 172.21.15.93:0/3242024222 wait complete. 2023-11-16T16:39:26.052 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.727+0000 7f4047c96700 1 Processor -- start 2023-11-16T16:39:26.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.728+0000 7f4047c96700 1 -- start start 2023-11-16T16:39:26.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.728+0000 7f4047c96700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 0x7f404019b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:26.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.728+0000 7f4047c96700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404019bd00 con 0x7f404007be00 2023-11-16T16:39:26.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.728+0000 7f4045a32700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 0x7f404019b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:26.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.728+0000 7f4045a32700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 0x7f404019b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37668/0 (socket says 172.21.15.93:37668) 2023-11-16T16:39:26.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.728+0000 7f4045a32700 1 -- 172.21.15.93:0/2978063012 learned_addr learned my addr 172.21.15.93:0/2978063012 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:26.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.729+0000 7f4045a32700 1 -- 172.21.15.93:0/2978063012 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c009550 con 0x7f404007be00 2023-11-16T16:39:26.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.729+0000 7f4045a32700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 0x7f404019b7c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f403c006b20 tx=0x7f403c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:26.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.729+0000 7f4036ffd700 1 -- 172.21.15.93:0/2978063012 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f403c003b70 con 0x7f404007be00 2023-11-16T16:39:26.053 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.729+0000 7f4036ffd700 1 -- 172.21.15.93:0/2978063012 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f403c003cd0 con 0x7f404007be00 2023-11-16T16:39:26.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.729+0000 7f4047c96700 1 -- 172.21.15.93:0/2978063012 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404010bde0 con 0x7f404007be00 2023-11-16T16:39:26.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.729+0000 7f4036ffd700 1 -- 172.21.15.93:0/2978063012 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f403c0247c0 con 0x7f404007be00 2023-11-16T16:39:26.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.729+0000 7f4047c96700 1 -- 172.21.15.93:0/2978063012 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404010bc50 con 0x7f404007be00 2023-11-16T16:39:26.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.730+0000 7f4036ffd700 1 -- 172.21.15.93:0/2978063012 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f403c024a30 con 0x7f404007be00 2023-11-16T16:39:26.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.731+0000 7f4036ffd700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 0x7f402c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:26.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.731+0000 7f4036ffd700 1 -- 172.21.15.93:0/2978063012 --> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f402c03b390 con 0x7f402c0387d0 2023-11-16T16:39:26.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.731+0000 7f4036ffd700 1 -- 172.21.15.93:0/2978063012 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f403c04c850 con 0x7f404007be00 2023-11-16T16:39:26.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.731+0000 7f4045231700 1 -- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 msgr2=0x7f402c03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/772471105 2023-11-16T16:39:26.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.731+0000 7f4045231700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 0x7f402c03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:39:26.054 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.932+0000 7f4045231700 1 -- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 msgr2=0x7f402c03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/772471105 2023-11-16T16:39:26.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:18.932+0000 7f4045231700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 0x7f402c03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T16:39:26.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:19.333+0000 7f4045231700 1 -- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 msgr2=0x7f402c03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/772471105 2023-11-16T16:39:26.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:19.333+0000 7f4045231700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 0x7f402c03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-16T16:39:26.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:20.134+0000 7f4045231700 1 -- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 msgr2=0x7f402c03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/772471105 2023-11-16T16:39:26.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:20.134+0000 7f4045231700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 0x7f402c03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-16T16:39:26.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:21.735+0000 7f4045231700 1 -- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 msgr2=0x7f402c03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/772471105 2023-11-16T16:39:26.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:21.735+0000 7f4045231700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 0x7f402c03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-16T16:39:26.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:24.570+0000 7f4036ffd700 1 -- 172.21.15.93:0/2978063012 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f403c02d3f0 con 0x7f404007be00 2023-11-16T16:39:26.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:24.571+0000 7f4036ffd700 1 -- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 msgr2=0x7f402c03ac80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:39:26.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:24.571+0000 7f4036ffd700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 0x7f402c03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:26.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.572+0000 7f4036ffd700 1 -- 172.21.15.93:0/2978063012 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f403c04e620 con 0x7f404007be00 2023-11-16T16:39:26.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.573+0000 7f4036ffd700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f402c03bfb0 0x7f402c03e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:26.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.573+0000 7f4036ffd700 1 -- 172.21.15.93:0/2978063012 --> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f402c03b390 con 0x7f402c03bfb0 2023-11-16T16:39:26.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.573+0000 7f4045231700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f402c03bfb0 0x7f402c03e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:26.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.575+0000 7f4045231700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f402c03bfb0 0x7f402c03e390 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4030003a10 tx=0x7f4030009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:26.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.575+0000 7f4036ffd700 1 -- 172.21.15.93:0/2978063012 <== mgr.14120 v2:172.21.15.93:6800/2198359920 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f402c03b390 con 0x7f402c03bfb0 2023-11-16T16:39:26.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.582+0000 7f4047c96700 1 -- 172.21.15.93:0/2978063012 --> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f404019cb00 con 0x7f402c03bfb0 2023-11-16T16:39:26.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.582+0000 7f4036ffd700 1 -- 172.21.15.93:0/2978063012 <== mgr.14120 v2:172.21.15.93:6800/2198359920 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f404019cb00 con 0x7f402c03bfb0 2023-11-16T16:39:26.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.582+0000 7f4047c96700 1 -- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f402c03bfb0 msgr2=0x7f402c03e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:26.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.582+0000 7f4047c96700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f402c03bfb0 0x7f402c03e390 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4030003a10 tx=0x7f4030009290 comp rx=0 tx=0).stop 2023-11-16T16:39:26.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.582+0000 7f4047c96700 1 -- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 msgr2=0x7f404019b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:26.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.582+0000 7f4047c96700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 0x7f404019b7c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f403c006b20 tx=0x7f403c011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:26.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.582+0000 7f4047c96700 1 -- 172.21.15.93:0/2978063012 shutdown_connections 2023-11-16T16:39:26.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.582+0000 7f4047c96700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f402c03bfb0 0x7f402c03e390 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:26.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.582+0000 7f4047c96700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f404007be00 0x7f404019b7c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:26.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.582+0000 7f4047c96700 1 --2- 172.21.15.93:0/2978063012 >> [v2:172.21.15.93:6800/772471105,v1:172.21.15.93:6801/772471105] conn(0x7f402c0387d0 0x7f402c03ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:26.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.582+0000 7f4047c96700 1 -- 172.21.15.93:0/2978063012 >> 172.21.15.93:0/2978063012 conn(0x7f4040100fa0 msgr2=0x7f4040101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:26.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.583+0000 7f4047c96700 1 -- 172.21.15.93:0/2978063012 shutdown_connections 2023-11-16T16:39:26.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:25.583+0000 7f4047c96700 1 -- 172.21.15.93:0/2978063012 wait complete. 2023-11-16T16:39:26.057 INFO:teuthology.orchestra.run.smithi093.stdout:mgr epoch 5 is available 2023-11-16T16:39:26.057 INFO:teuthology.orchestra.run.smithi093.stdout:Setting orchestrator backend to cephadm... 2023-11-16T16:39:26.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:26 smithi093 ceph-mon[97863]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-11-16T16:39:26.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:26 smithi093 ceph-mon[97863]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-11-16T16:39:26.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:26 smithi093 ceph-mon[97863]: [16/Nov/2023:16:39:25] ENGINE Bus STARTING 2023-11-16T16:39:26.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:26 smithi093 ceph-mon[97863]: [16/Nov/2023:16:39:25] ENGINE Serving on http://172.21.15.93:8765 2023-11-16T16:39:26.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:26 smithi093 ceph-mon[97863]: [16/Nov/2023:16:39:26] ENGINE Serving on https://172.21.15.93:7150 2023-11-16T16:39:26.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:26 smithi093 ceph-mon[97863]: [16/Nov/2023:16:39:26] ENGINE Bus STARTED 2023-11-16T16:39:26.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:26 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:27.234 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.554+0000 7f35fbc6e700 1 Processor -- start 2023-11-16T16:39:27.234 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.554+0000 7f35fbc6e700 1 -- start start 2023-11-16T16:39:27.234 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.555+0000 7f35fbc6e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 0x7f35f407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:27.234 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.555+0000 7f35fbc6e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f407a840 con 0x7f35f407be00 2023-11-16T16:39:27.235 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.555+0000 7f35f9a0a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 0x7f35f407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:27.235 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.555+0000 7f35f9a0a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 0x7f35f407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37754/0 (socket says 172.21.15.93:37754) 2023-11-16T16:39:27.235 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.555+0000 7f35f9a0a700 1 -- 172.21.15.93:0/3273121272 learned_addr learned my addr 172.21.15.93:0/3273121272 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:27.235 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.555+0000 7f35f9a0a700 1 -- 172.21.15.93:0/3273121272 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f407a980 con 0x7f35f407be00 2023-11-16T16:39:27.235 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.555+0000 7f35f9a0a700 1 --2- 172.21.15.93:0/3273121272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 0x7f35f407a300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f35f00098a0 tx=0x7f35f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971637f20b0f6879 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:27.235 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.556+0000 7f35f8a08700 1 -- 172.21.15.93:0/3273121272 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35f0004160 con 0x7f35f407be00 2023-11-16T16:39:27.235 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.556+0000 7f35f8a08700 1 -- 172.21.15.93:0/3273121272 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f35f00042c0 con 0x7f35f407be00 2023-11-16T16:39:27.235 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.556+0000 7f35f8a08700 1 -- 172.21.15.93:0/3273121272 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35f0011a00 con 0x7f35f407be00 2023-11-16T16:39:27.235 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.556+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3273121272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 msgr2=0x7f35f407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:27.236 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.556+0000 7f35fbc6e700 1 --2- 172.21.15.93:0/3273121272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 0x7f35f407a300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f35f00098a0 tx=0x7f35f0009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:27.236 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.556+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3273121272 shutdown_connections 2023-11-16T16:39:27.236 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.556+0000 7f35fbc6e700 1 --2- 172.21.15.93:0/3273121272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 0x7f35f407a300 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:27.236 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.556+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3273121272 >> 172.21.15.93:0/3273121272 conn(0x7f35f4100fa0 msgr2=0x7f35f41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:27.236 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.556+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3273121272 shutdown_connections 2023-11-16T16:39:27.236 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.556+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3273121272 wait complete. 2023-11-16T16:39:27.236 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.557+0000 7f35fbc6e700 1 Processor -- start 2023-11-16T16:39:27.236 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.558+0000 7f35fbc6e700 1 -- start start 2023-11-16T16:39:27.236 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.558+0000 7f35fbc6e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 0x7f35f419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:27.237 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.558+0000 7f35fbc6e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f419bce0 con 0x7f35f407be00 2023-11-16T16:39:27.237 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.558+0000 7f35f9a0a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 0x7f35f419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:27.237 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.558+0000 7f35f9a0a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 0x7f35f419b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37758/0 (socket says 172.21.15.93:37758) 2023-11-16T16:39:27.237 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.558+0000 7f35f9a0a700 1 -- 172.21.15.93:0/3869263771 learned_addr learned my addr 172.21.15.93:0/3869263771 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:27.237 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.558+0000 7f35f9a0a700 1 -- 172.21.15.93:0/3869263771 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f0009550 con 0x7f35f407be00 2023-11-16T16:39:27.237 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.559+0000 7f35f9a0a700 1 --2- 172.21.15.93:0/3869263771 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 0x7f35f419b7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f35f0006b20 tx=0x7f35f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:27.237 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.559+0000 7f35eaffd700 1 -- 172.21.15.93:0/3869263771 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35f0003b70 con 0x7f35f407be00 2023-11-16T16:39:27.237 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.559+0000 7f35eaffd700 1 -- 172.21.15.93:0/3869263771 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f35f0003cd0 con 0x7f35f407be00 2023-11-16T16:39:27.237 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.559+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3869263771 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f410bde0 con 0x7f35f407be00 2023-11-16T16:39:27.238 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.559+0000 7f35eaffd700 1 -- 172.21.15.93:0/3869263771 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35f00247c0 con 0x7f35f407be00 2023-11-16T16:39:27.238 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.559+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3869263771 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f410bc50 con 0x7f35f407be00 2023-11-16T16:39:27.238 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.560+0000 7f35eaffd700 1 -- 172.21.15.93:0/3869263771 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f35f001ab40 con 0x7f35f407be00 2023-11-16T16:39:27.238 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.560+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3869263771 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f4062120 con 0x7f35f407be00 2023-11-16T16:39:27.238 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.560+0000 7f35eaffd700 1 --2- 172.21.15.93:0/3869263771 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f35e00386f0 0x7f35e003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:27.238 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.561+0000 7f35eaffd700 1 -- 172.21.15.93:0/3869263771 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f35f004c7c0 con 0x7f35f407be00 2023-11-16T16:39:27.238 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.561+0000 7f35f9209700 1 --2- 172.21.15.93:0/3869263771 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f35e00386f0 0x7f35e003aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:27.238 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.561+0000 7f35f9209700 1 --2- 172.21.15.93:0/3869263771 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f35e00386f0 0x7f35e003aba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f35e4006fd0 tx=0x7f35e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:27.239 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.566+0000 7f35eaffd700 1 -- 172.21.15.93:0/3869263771 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f35f001a950 con 0x7f35f407be00 2023-11-16T16:39:27.239 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.624+0000 7f35eaffd700 1 -- 172.21.15.93:0/3869263771 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f35f001a430 con 0x7f35f407be00 2023-11-16T16:39:27.239 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.779+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3869263771 --> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f35f4102380 con 0x7f35e00386f0 2023-11-16T16:39:27.239 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.787+0000 7f35eaffd700 1 -- 172.21.15.93:0/3869263771 <== mgr.14120 v2:172.21.15.93:6800/2198359920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f35f4102380 con 0x7f35e00386f0 2023-11-16T16:39:27.239 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.789+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3869263771 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f35e00386f0 msgr2=0x7f35e003aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:27.239 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.789+0000 7f35fbc6e700 1 --2- 172.21.15.93:0/3869263771 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f35e00386f0 0x7f35e003aba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f35e4006fd0 tx=0x7f35e4006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:27.239 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.789+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3869263771 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 msgr2=0x7f35f419b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:27.239 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.789+0000 7f35fbc6e700 1 --2- 172.21.15.93:0/3869263771 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 0x7f35f419b7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f35f0006b20 tx=0x7f35f0011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:27.240 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.789+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3869263771 shutdown_connections 2023-11-16T16:39:27.240 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.789+0000 7f35fbc6e700 1 --2- 172.21.15.93:0/3869263771 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f35e00386f0 0x7f35e003aba0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:27.240 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.789+0000 7f35fbc6e700 1 --2- 172.21.15.93:0/3869263771 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35f407be00 0x7f35f419b7a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:27.240 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.789+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3869263771 >> 172.21.15.93:0/3869263771 conn(0x7f35f4100fa0 msgr2=0x7f35f4101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:27.240 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.789+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3869263771 shutdown_connections 2023-11-16T16:39:27.240 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:26.790+0000 7f35fbc6e700 1 -- 172.21.15.93:0/3869263771 wait complete. 2023-11-16T16:39:27.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:27 smithi093 ceph-mon[97863]: mgrmap e8: x(active, since 2s) 2023-11-16T16:39:27.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:27 smithi093 ceph-mon[97863]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:27.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:27 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:27.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:27 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:28.514 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout value unchanged 2023-11-16T16:39:28.514 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.769+0000 7f371a350700 1 Processor -- start 2023-11-16T16:39:28.514 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.769+0000 7f371a350700 1 -- start start 2023-11-16T16:39:28.514 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.770+0000 7f371a350700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 0x7f371407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:28.515 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.770+0000 7f371a350700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371407a820 con 0x7f371407bde0 2023-11-16T16:39:28.515 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.770+0000 7f3713fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 0x7f371407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:28.515 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.770+0000 7f3713fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 0x7f371407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36364/0 (socket says 172.21.15.93:36364) 2023-11-16T16:39:28.515 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.770+0000 7f3713fff700 1 -- 172.21.15.93:0/856726990 learned_addr learned my addr 172.21.15.93:0/856726990 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:28.515 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.771+0000 7f3713fff700 1 -- 172.21.15.93:0/856726990 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371407a960 con 0x7f371407bde0 2023-11-16T16:39:28.515 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.771+0000 7f3713fff700 1 --2- 172.21.15.93:0/856726990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 0x7f371407a2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f37040098a0 tx=0x7f3704009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c98d7b77aa6a7c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:28.515 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.771+0000 7f3712ffd700 1 -- 172.21.15.93:0/856726990 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3704004160 con 0x7f371407bde0 2023-11-16T16:39:28.515 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.771+0000 7f3712ffd700 1 -- 172.21.15.93:0/856726990 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f37040042c0 con 0x7f371407bde0 2023-11-16T16:39:28.515 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.771+0000 7f3712ffd700 1 -- 172.21.15.93:0/856726990 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3704011a00 con 0x7f371407bde0 2023-11-16T16:39:28.516 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.772+0000 7f371a350700 1 -- 172.21.15.93:0/856726990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 msgr2=0x7f371407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:28.516 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.772+0000 7f371a350700 1 --2- 172.21.15.93:0/856726990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 0x7f371407a2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f37040098a0 tx=0x7f3704009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:28.516 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.772+0000 7f371a350700 1 -- 172.21.15.93:0/856726990 shutdown_connections 2023-11-16T16:39:28.516 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.772+0000 7f371a350700 1 --2- 172.21.15.93:0/856726990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 0x7f371407a2e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:28.516 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.772+0000 7f371a350700 1 -- 172.21.15.93:0/856726990 >> 172.21.15.93:0/856726990 conn(0x7f3714100fa0 msgr2=0x7f37141033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:28.516 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.772+0000 7f371a350700 1 -- 172.21.15.93:0/856726990 shutdown_connections 2023-11-16T16:39:28.516 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.772+0000 7f371a350700 1 -- 172.21.15.93:0/856726990 wait complete. 2023-11-16T16:39:28.516 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.773+0000 7f371a350700 1 Processor -- start 2023-11-16T16:39:28.516 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.773+0000 7f371a350700 1 -- start start 2023-11-16T16:39:28.516 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.774+0000 7f371a350700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 0x7f371419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:28.517 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.774+0000 7f371a350700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371419bcf0 con 0x7f371407bde0 2023-11-16T16:39:28.517 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.774+0000 7f3713fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 0x7f371419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:28.517 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.774+0000 7f3713fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 0x7f371419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36376/0 (socket says 172.21.15.93:36376) 2023-11-16T16:39:28.517 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.774+0000 7f3713fff700 1 -- 172.21.15.93:0/3246314768 learned_addr learned my addr 172.21.15.93:0/3246314768 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:28.517 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.774+0000 7f3713fff700 1 -- 172.21.15.93:0/3246314768 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3704009550 con 0x7f371407bde0 2023-11-16T16:39:28.517 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.775+0000 7f3713fff700 1 --2- 172.21.15.93:0/3246314768 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 0x7f371419b7b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3704006af0 tx=0x7f3704011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:28.517 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.775+0000 7f37117fa700 1 -- 172.21.15.93:0/3246314768 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3704003a40 con 0x7f371407bde0 2023-11-16T16:39:28.517 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.775+0000 7f37117fa700 1 -- 172.21.15.93:0/3246314768 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3704003ba0 con 0x7f371407bde0 2023-11-16T16:39:28.517 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.775+0000 7f37117fa700 1 -- 172.21.15.93:0/3246314768 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f370401a700 con 0x7f371407bde0 2023-11-16T16:39:28.517 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.775+0000 7f371a350700 1 -- 172.21.15.93:0/3246314768 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371410be00 con 0x7f371407bde0 2023-11-16T16:39:28.518 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.775+0000 7f371a350700 1 -- 172.21.15.93:0/3246314768 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371410bc70 con 0x7f371407bde0 2023-11-16T16:39:28.518 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.776+0000 7f37117fa700 1 -- 172.21.15.93:0/3246314768 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f3704024ad0 con 0x7f371407bde0 2023-11-16T16:39:28.518 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.776+0000 7f371a350700 1 -- 172.21.15.93:0/3246314768 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3714062120 con 0x7f371407bde0 2023-11-16T16:39:28.518 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.776+0000 7f37117fa700 1 --2- 172.21.15.93:0/3246314768 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f36fc038820 0x7f36fc03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:28.518 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.776+0000 7f37117fa700 1 -- 172.21.15.93:0/3246314768 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f370404c780 con 0x7f371407bde0 2023-11-16T16:39:28.518 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.777+0000 7f37137fe700 1 --2- 172.21.15.93:0/3246314768 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f36fc038820 0x7f36fc03acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:28.518 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.777+0000 7f37137fe700 1 --2- 172.21.15.93:0/3246314768 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f36fc038820 0x7f36fc03acd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3708006fd0 tx=0x7f3708006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:28.518 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.781+0000 7f37117fa700 1 -- 172.21.15.93:0/3246314768 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3704018b80 con 0x7f371407bde0 2023-11-16T16:39:28.518 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.979+0000 7f371a350700 1 -- 172.21.15.93:0/3246314768 --> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f37141023a0 con 0x7f36fc038820 2023-11-16T16:39:28.519 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.980+0000 7f37117fa700 1 -- 172.21.15.93:0/3246314768 <== mgr.14120 v2:172.21.15.93:6800/2198359920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f37141023a0 con 0x7f36fc038820 2023-11-16T16:39:28.519 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.982+0000 7f371a350700 1 -- 172.21.15.93:0/3246314768 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f36fc038820 msgr2=0x7f36fc03acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:28.519 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.982+0000 7f371a350700 1 --2- 172.21.15.93:0/3246314768 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f36fc038820 0x7f36fc03acd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3708006fd0 tx=0x7f3708006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:28.519 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.982+0000 7f371a350700 1 -- 172.21.15.93:0/3246314768 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 msgr2=0x7f371419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:28.519 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.982+0000 7f371a350700 1 --2- 172.21.15.93:0/3246314768 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 0x7f371419b7b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3704006af0 tx=0x7f3704011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:28.519 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.983+0000 7f371a350700 1 -- 172.21.15.93:0/3246314768 shutdown_connections 2023-11-16T16:39:28.519 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.983+0000 7f371a350700 1 --2- 172.21.15.93:0/3246314768 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f36fc038820 0x7f36fc03acd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:28.519 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.983+0000 7f371a350700 1 --2- 172.21.15.93:0/3246314768 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f371407bde0 0x7f371419b7b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:28.519 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.983+0000 7f371a350700 1 -- 172.21.15.93:0/3246314768 >> 172.21.15.93:0/3246314768 conn(0x7f3714100fa0 msgr2=0x7f3714101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:28.520 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.983+0000 7f371a350700 1 -- 172.21.15.93:0/3246314768 shutdown_connections 2023-11-16T16:39:28.520 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:27.983+0000 7f371a350700 1 -- 172.21.15.93:0/3246314768 wait complete. 2023-11-16T16:39:28.520 INFO:teuthology.orchestra.run.smithi093.stdout:Generating ssh key... 2023-11-16T16:39:28.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:28 smithi093 ceph-mon[97863]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:29.651 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: Generating public/private rsa key pair. 2023-11-16T16:39:29.651 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: Your identification has been saved in /tmp/tmp185p0ylg/key. 2023-11-16T16:39:29.651 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: Your public key has been saved in /tmp/tmp185p0ylg/key.pub. 2023-11-16T16:39:29.651 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: The key fingerprint is: 2023-11-16T16:39:29.651 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: SHA256:ggiCsjKLdz0Le/scYg2cg5fjWuAaaMFnjfCoXC5iKG8 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:39:29.651 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: The key's randomart image is: 2023-11-16T16:39:29.651 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: +---[RSA 3072]----+ 2023-11-16T16:39:29.652 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: | | 2023-11-16T16:39:29.652 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: |. | 2023-11-16T16:39:29.652 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: |=. | 2023-11-16T16:39:29.652 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: |+o+.o+ o | 2023-11-16T16:39:29.652 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: |++.*+.X S | 2023-11-16T16:39:29.652 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: |*+*. = * | 2023-11-16T16:39:29.652 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: |O=o.+ B o | 2023-11-16T16:39:29.652 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: |=oE+ *.= . | 2023-11-16T16:39:29.652 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: | .o o.ooo | 2023-11-16T16:39:29.652 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: +----[SHA256]-----+ 2023-11-16T16:39:29.794 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.052+0000 7f80aa1c2700 1 Processor -- start 2023-11-16T16:39:29.794 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.052+0000 7f80aa1c2700 1 -- start start 2023-11-16T16:39:29.794 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.053+0000 7f80aa1c2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 0x7f80a4105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:29.794 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.053+0000 7f80aa1c2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a4106260 con 0x7f80a4105910 2023-11-16T16:39:29.794 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.053+0000 7f80a37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 0x7f80a4105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:29.794 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.053+0000 7f80a37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 0x7f80a4105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36384/0 (socket says 172.21.15.93:36384) 2023-11-16T16:39:29.795 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.053+0000 7f80a37fe700 1 -- 172.21.15.93:0/1728749926 learned_addr learned my addr 172.21.15.93:0/1728749926 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:29.795 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.053+0000 7f80a37fe700 1 -- 172.21.15.93:0/1728749926 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a41063a0 con 0x7f80a4105910 2023-11-16T16:39:29.795 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.054+0000 7f80a37fe700 1 --2- 172.21.15.93:0/1728749926 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 0x7f80a4105d20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f80940098a0 tx=0x7f8094009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b55fe18c81e8d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:29.795 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.054+0000 7f80a27fc700 1 -- 172.21.15.93:0/1728749926 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8094004160 con 0x7f80a4105910 2023-11-16T16:39:29.795 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.054+0000 7f80a27fc700 1 -- 172.21.15.93:0/1728749926 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f80940042c0 con 0x7f80a4105910 2023-11-16T16:39:29.795 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.054+0000 7f80a27fc700 1 -- 172.21.15.93:0/1728749926 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8094011a00 con 0x7f80a4105910 2023-11-16T16:39:29.795 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.054+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1728749926 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 msgr2=0x7f80a4105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:29.795 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.054+0000 7f80aa1c2700 1 --2- 172.21.15.93:0/1728749926 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 0x7f80a4105d20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f80940098a0 tx=0x7f8094009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:29.795 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.055+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1728749926 shutdown_connections 2023-11-16T16:39:29.796 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.055+0000 7f80aa1c2700 1 --2- 172.21.15.93:0/1728749926 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 0x7f80a4105d20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:29.796 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.055+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1728749926 >> 172.21.15.93:0/1728749926 conn(0x7f80a4100fa0 msgr2=0x7f80a41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:29.796 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.055+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1728749926 shutdown_connections 2023-11-16T16:39:29.796 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.055+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1728749926 wait complete. 2023-11-16T16:39:29.796 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.056+0000 7f80aa1c2700 1 Processor -- start 2023-11-16T16:39:29.796 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.056+0000 7f80aa1c2700 1 -- start start 2023-11-16T16:39:29.796 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.056+0000 7f80aa1c2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 0x7f80a419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:29.796 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.056+0000 7f80aa1c2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a419bca0 con 0x7f80a4105910 2023-11-16T16:39:29.796 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.056+0000 7f80a37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 0x7f80a419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:29.797 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.056+0000 7f80a37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 0x7f80a419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36394/0 (socket says 172.21.15.93:36394) 2023-11-16T16:39:29.797 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.056+0000 7f80a37fe700 1 -- 172.21.15.93:0/1209149014 learned_addr learned my addr 172.21.15.93:0/1209149014 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:29.797 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.057+0000 7f80a37fe700 1 -- 172.21.15.93:0/1209149014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094009550 con 0x7f80a4105910 2023-11-16T16:39:29.797 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.057+0000 7f80a37fe700 1 --2- 172.21.15.93:0/1209149014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 0x7f80a419b760 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8094009fb0 tx=0x7f8094011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:29.797 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.057+0000 7f80a0ff9700 1 -- 172.21.15.93:0/1209149014 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8094003930 con 0x7f80a4105910 2023-11-16T16:39:29.797 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.057+0000 7f80a0ff9700 1 -- 172.21.15.93:0/1209149014 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8094003a90 con 0x7f80a4105910 2023-11-16T16:39:29.797 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.057+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1209149014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a4107960 con 0x7f80a4105910 2023-11-16T16:39:29.797 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.057+0000 7f80a0ff9700 1 -- 172.21.15.93:0/1209149014 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8094019610 con 0x7f80a4105910 2023-11-16T16:39:29.797 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.057+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1209149014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a41077d0 con 0x7f80a4105910 2023-11-16T16:39:29.797 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.058+0000 7f80a0ff9700 1 -- 172.21.15.93:0/1209149014 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f809401a430 con 0x7f80a4105910 2023-11-16T16:39:29.798 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.058+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1209149014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a4109de0 con 0x7f80a4105910 2023-11-16T16:39:29.798 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.059+0000 7f80a0ff9700 1 --2- 172.21.15.93:0/1209149014 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f808c0387d0 0x7f808c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:29.798 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.059+0000 7f80a0ff9700 1 -- 172.21.15.93:0/1209149014 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f809404c880 con 0x7f80a4105910 2023-11-16T16:39:29.798 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.059+0000 7f80a2ffd700 1 --2- 172.21.15.93:0/1209149014 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f808c0387d0 0x7f808c03ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:29.798 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.060+0000 7f80a2ffd700 1 --2- 172.21.15.93:0/1209149014 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f808c0387d0 0x7f808c03ac80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8098006fd0 tx=0x7f8098006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:29.798 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.064+0000 7f80a0ff9700 1 -- 172.21.15.93:0/1209149014 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f80940239e0 con 0x7f80a4105910 2023-11-16T16:39:29.798 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.261+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1209149014 --> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f80a41022e0 con 0x7f808c0387d0 2023-11-16T16:39:29.798 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.362+0000 7f80a0ff9700 1 -- 172.21.15.93:0/1209149014 <== mgr.14120 v2:172.21.15.93:6800/2198359920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f80a41022e0 con 0x7f808c0387d0 2023-11-16T16:39:29.799 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.364+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1209149014 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f808c0387d0 msgr2=0x7f808c03ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:29.799 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.364+0000 7f80aa1c2700 1 --2- 172.21.15.93:0/1209149014 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f808c0387d0 0x7f808c03ac80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8098006fd0 tx=0x7f8098006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:29.799 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.364+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1209149014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 msgr2=0x7f80a419b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:29.799 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.364+0000 7f80aa1c2700 1 --2- 172.21.15.93:0/1209149014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 0x7f80a419b760 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8094009fb0 tx=0x7f8094011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:29.799 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.365+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1209149014 shutdown_connections 2023-11-16T16:39:29.799 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.365+0000 7f80aa1c2700 1 --2- 172.21.15.93:0/1209149014 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f808c0387d0 0x7f808c03ac80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:29.799 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.365+0000 7f80aa1c2700 1 --2- 172.21.15.93:0/1209149014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80a4105910 0x7f80a419b760 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:29.799 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.365+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1209149014 >> 172.21.15.93:0/1209149014 conn(0x7f80a4100fa0 msgr2=0x7f80a4101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:29.799 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.365+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1209149014 shutdown_connections 2023-11-16T16:39:29.800 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:29.365+0000 7f80aa1c2700 1 -- 172.21.15.93:0/1209149014 wait complete. 2023-11-16T16:39:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:30 smithi093 ceph-mon[97863]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:30 smithi093 ceph-mon[97863]: Generating ssh key... 2023-11-16T16:39:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:30 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:30.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:30 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:30.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZspfJ+QMr9tkiR3JeSLbtDVZX5wMU7OwNvd099622r0TeEJpEBUjFdIojO6XfKPpswRL+9JMUiua0DHDiUMWCt6y+70bn6FOygTNs/Xfjbn/LI6DOPbx0u3JkUO0MbX6xJa+xorvK5yEnmC154x0Rsw3B2WBMP1+bf4DkOXc0Jz1fX/RvfgbBff4Ejt/WKDVBgAhcRV1it6EdD8/vQdsd+hjBY+PJV5+epVuZx/OgIM4tv6vy8scg5I94KIRjCS/sBIUNpaD+ionvNdWlHaC6JmPh9dmDfBuoZC/uiD1kIU+b0jb/H409zAL1FgDeT5c+SeLT96whTnSB49VAgf9OhDqIoIGk2hyxlBDFWJZF+OA/fM0mbhc6ywUEyLTe3O6EsZdhVIDYbMgHdL3Ipdr/VA9UaNiBRfGlxEZwpK9YwY3OzbmOIh0mfLAodC1vnjaahr9BBXKFXcsKu4/7t+g6Nte3ycuTUxTPMSsdMw1vKwfomfIcjljeWAqMvmCKT/E= ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:39:30.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.352+0000 7f47a768e700 1 Processor -- start 2023-11-16T16:39:30.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.353+0000 7f47a768e700 1 -- start start 2023-11-16T16:39:30.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.353+0000 7f47a768e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 0x7f47a007a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.353+0000 7f47a768e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a007a800 con 0x7f47a007bdc0 2023-11-16T16:39:30.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.353+0000 7f47a542a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 0x7f47a007a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.353+0000 7f47a542a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 0x7f47a007a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36400/0 (socket says 172.21.15.93:36400) 2023-11-16T16:39:30.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.353+0000 7f47a542a700 1 -- 172.21.15.93:0/541182056 learned_addr learned my addr 172.21.15.93:0/541182056 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:30.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.354+0000 7f47a542a700 1 -- 172.21.15.93:0/541182056 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a007a940 con 0x7f47a007bdc0 2023-11-16T16:39:30.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.354+0000 7f47a542a700 1 --2- 172.21.15.93:0/541182056 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 0x7f47a007a2c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f479c0098a0 tx=0x7f479c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddebba88469b3915 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:30.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.354+0000 7f4797fff700 1 -- 172.21.15.93:0/541182056 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f479c004160 con 0x7f47a007bdc0 2023-11-16T16:39:30.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.354+0000 7f4797fff700 1 -- 172.21.15.93:0/541182056 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f479c0042c0 con 0x7f47a007bdc0 2023-11-16T16:39:30.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.354+0000 7f4797fff700 1 -- 172.21.15.93:0/541182056 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f479c011a00 con 0x7f47a007bdc0 2023-11-16T16:39:30.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.355+0000 7f47a768e700 1 -- 172.21.15.93:0/541182056 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 msgr2=0x7f47a007a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.355+0000 7f47a768e700 1 --2- 172.21.15.93:0/541182056 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 0x7f47a007a2c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f479c0098a0 tx=0x7f479c009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.355+0000 7f47a768e700 1 -- 172.21.15.93:0/541182056 shutdown_connections 2023-11-16T16:39:30.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.355+0000 7f47a768e700 1 --2- 172.21.15.93:0/541182056 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 0x7f47a007a2c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.355+0000 7f47a768e700 1 -- 172.21.15.93:0/541182056 >> 172.21.15.93:0/541182056 conn(0x7f47a0100fa0 msgr2=0x7f47a01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:30.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.355+0000 7f47a768e700 1 -- 172.21.15.93:0/541182056 shutdown_connections 2023-11-16T16:39:30.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.355+0000 7f47a768e700 1 -- 172.21.15.93:0/541182056 wait complete. 2023-11-16T16:39:30.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.356+0000 7f47a768e700 1 Processor -- start 2023-11-16T16:39:30.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.356+0000 7f47a768e700 1 -- start start 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.356+0000 7f47a768e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 0x7f47a019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.356+0000 7f47a768e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a019bcf0 con 0x7f47a007bdc0 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.357+0000 7f47a542a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 0x7f47a019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.357+0000 7f47a542a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 0x7f47a019b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36406/0 (socket says 172.21.15.93:36406) 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.357+0000 7f47a542a700 1 -- 172.21.15.93:0/2125155448 learned_addr learned my addr 172.21.15.93:0/2125155448 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.357+0000 7f47a542a700 1 -- 172.21.15.93:0/2125155448 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c009550 con 0x7f47a007bdc0 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.357+0000 7f47a542a700 1 --2- 172.21.15.93:0/2125155448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 0x7f47a019b7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f479c009fb0 tx=0x7f479c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.360+0000 7f47967fc700 1 -- 172.21.15.93:0/2125155448 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f479c003930 con 0x7f47a007bdc0 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.360+0000 7f47967fc700 1 -- 172.21.15.93:0/2125155448 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f479c003a90 con 0x7f47a007bdc0 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.360+0000 7f47a768e700 1 -- 172.21.15.93:0/2125155448 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a010bdc0 con 0x7f47a007bdc0 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.360+0000 7f47967fc700 1 -- 172.21.15.93:0/2125155448 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f479c0195e0 con 0x7f47a007bdc0 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.360+0000 7f47a768e700 1 -- 172.21.15.93:0/2125155448 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a010bc30 con 0x7f47a007bdc0 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.361+0000 7f47967fc700 1 -- 172.21.15.93:0/2125155448 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f479c019850 con 0x7f47a007bdc0 2023-11-16T16:39:30.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.361+0000 7f47a768e700 1 -- 172.21.15.93:0/2125155448 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a0062120 con 0x7f47a007bdc0 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.361+0000 7f47967fc700 1 --2- 172.21.15.93:0/2125155448 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f478c0387d0 0x7f478c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.361+0000 7f47967fc700 1 -- 172.21.15.93:0/2125155448 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f479c04ca60 con 0x7f47a007bdc0 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.362+0000 7f47a4c29700 1 --2- 172.21.15.93:0/2125155448 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f478c0387d0 0x7f478c03ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.362+0000 7f47a4c29700 1 --2- 172.21.15.93:0/2125155448 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f478c0387d0 0x7f478c03ac80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4790006fd0 tx=0x7f4790006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.367+0000 7f47967fc700 1 -- 172.21.15.93:0/2125155448 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f479c02b340 con 0x7f47a007bdc0 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.563+0000 7f47a768e700 1 -- 172.21.15.93:0/2125155448 --> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f47a01023a0 con 0x7f478c0387d0 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.564+0000 7f47967fc700 1 -- 172.21.15.93:0/2125155448 <== mgr.14120 v2:172.21.15.93:6800/2198359920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f47a01023a0 con 0x7f478c0387d0 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.566+0000 7f47a768e700 1 -- 172.21.15.93:0/2125155448 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f478c0387d0 msgr2=0x7f478c03ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.566+0000 7f47a768e700 1 --2- 172.21.15.93:0/2125155448 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f478c0387d0 0x7f478c03ac80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4790006fd0 tx=0x7f4790006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.566+0000 7f47a768e700 1 -- 172.21.15.93:0/2125155448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 msgr2=0x7f47a019b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.566+0000 7f47a768e700 1 --2- 172.21.15.93:0/2125155448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 0x7f47a019b7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f479c009fb0 tx=0x7f479c011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.566+0000 7f47a768e700 1 -- 172.21.15.93:0/2125155448 shutdown_connections 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.566+0000 7f47a768e700 1 --2- 172.21.15.93:0/2125155448 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f478c0387d0 0x7f478c03ac80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.566+0000 7f47a768e700 1 --2- 172.21.15.93:0/2125155448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47a007bdc0 0x7f47a019b7b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.566+0000 7f47a768e700 1 -- 172.21.15.93:0/2125155448 >> 172.21.15.93:0/2125155448 conn(0x7f47a0100fa0 msgr2=0x7f47a0101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:30.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.566+0000 7f47a768e700 1 -- 172.21.15.93:0/2125155448 shutdown_connections 2023-11-16T16:39:30.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:30.566+0000 7f47a768e700 1 -- 172.21.15.93:0/2125155448 wait complete. 2023-11-16T16:39:30.977 INFO:teuthology.orchestra.run.smithi093.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-11-16T16:39:30.977 INFO:teuthology.orchestra.run.smithi093.stdout:Adding key to root@localhost authorized_keys... 2023-11-16T16:39:30.977 INFO:teuthology.orchestra.run.smithi093.stdout:Adding host smithi093... 2023-11-16T16:39:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:31 smithi093 ceph-mon[97863]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:32.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:32 smithi093 ceph-mon[97863]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi093", "addr": "172.21.15.93", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:33.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:33 smithi093 ceph-mon[97863]: Deploying cephadm binary to smithi093 2023-11-16T16:39:35.166 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout Added host 'smithi093' with addr '172.21.15.93' 2023-11-16T16:39:35.166 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.498+0000 7f01e2c16700 1 Processor -- start 2023-11-16T16:39:35.166 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.499+0000 7f01e2c16700 1 -- start start 2023-11-16T16:39:35.166 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.499+0000 7f01e2c16700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 0x7f01dc07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:35.167 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.499+0000 7f01e2c16700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc07a820 con 0x7f01dc07bde0 2023-11-16T16:39:35.167 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.499+0000 7f01e09b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 0x7f01dc07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:35.167 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.499+0000 7f01e09b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 0x7f01dc07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36416/0 (socket says 172.21.15.93:36416) 2023-11-16T16:39:35.167 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.499+0000 7f01e09b2700 1 -- 172.21.15.93:0/1724666749 learned_addr learned my addr 172.21.15.93:0/1724666749 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:35.167 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.499+0000 7f01e09b2700 1 -- 172.21.15.93:0/1724666749 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc07a960 con 0x7f01dc07bde0 2023-11-16T16:39:35.167 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.500+0000 7f01e09b2700 1 --2- 172.21.15.93:0/1724666749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 0x7f01dc07a2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f01d00098a0 tx=0x7f01d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae20beb3173c176 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:35.167 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.500+0000 7f01db7fe700 1 -- 172.21.15.93:0/1724666749 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01d0004160 con 0x7f01dc07bde0 2023-11-16T16:39:35.168 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.500+0000 7f01db7fe700 1 -- 172.21.15.93:0/1724666749 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f01d00042c0 con 0x7f01dc07bde0 2023-11-16T16:39:35.168 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.500+0000 7f01db7fe700 1 -- 172.21.15.93:0/1724666749 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01d0011a00 con 0x7f01dc07bde0 2023-11-16T16:39:35.168 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.500+0000 7f01e2c16700 1 -- 172.21.15.93:0/1724666749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 msgr2=0x7f01dc07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:35.168 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.500+0000 7f01e2c16700 1 --2- 172.21.15.93:0/1724666749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 0x7f01dc07a2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f01d00098a0 tx=0x7f01d0009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:35.168 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.501+0000 7f01e2c16700 1 -- 172.21.15.93:0/1724666749 shutdown_connections 2023-11-16T16:39:35.168 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.501+0000 7f01e2c16700 1 --2- 172.21.15.93:0/1724666749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 0x7f01dc07a2e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:35.168 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.501+0000 7f01e2c16700 1 -- 172.21.15.93:0/1724666749 >> 172.21.15.93:0/1724666749 conn(0x7f01dc100fa0 msgr2=0x7f01dc1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:35.168 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.501+0000 7f01e2c16700 1 -- 172.21.15.93:0/1724666749 shutdown_connections 2023-11-16T16:39:35.169 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.501+0000 7f01e2c16700 1 -- 172.21.15.93:0/1724666749 wait complete. 2023-11-16T16:39:35.169 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.502+0000 7f01e2c16700 1 Processor -- start 2023-11-16T16:39:35.169 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.502+0000 7f01e2c16700 1 -- start start 2023-11-16T16:39:35.169 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.502+0000 7f01e2c16700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 0x7f01dc19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:35.169 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.502+0000 7f01e2c16700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc19bcc0 con 0x7f01dc07bde0 2023-11-16T16:39:35.169 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.502+0000 7f01e09b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 0x7f01dc19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:35.169 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.502+0000 7f01e09b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 0x7f01dc19b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36432/0 (socket says 172.21.15.93:36432) 2023-11-16T16:39:35.170 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.502+0000 7f01e09b2700 1 -- 172.21.15.93:0/527308078 learned_addr learned my addr 172.21.15.93:0/527308078 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:35.170 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.503+0000 7f01e09b2700 1 -- 172.21.15.93:0/527308078 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d0009550 con 0x7f01dc07bde0 2023-11-16T16:39:35.170 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.503+0000 7f01e09b2700 1 --2- 172.21.15.93:0/527308078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 0x7f01dc19b780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01d0006af0 tx=0x7f01d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:35.170 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.503+0000 7f01d9ffb700 1 -- 172.21.15.93:0/527308078 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01d0003a40 con 0x7f01dc07bde0 2023-11-16T16:39:35.170 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.503+0000 7f01d9ffb700 1 -- 172.21.15.93:0/527308078 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f01d0003ba0 con 0x7f01dc07bde0 2023-11-16T16:39:35.170 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.503+0000 7f01d9ffb700 1 -- 172.21.15.93:0/527308078 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01d001a700 con 0x7f01dc07bde0 2023-11-16T16:39:35.171 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.503+0000 7f01e2c16700 1 -- 172.21.15.93:0/527308078 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01dc10be00 con 0x7f01dc07bde0 2023-11-16T16:39:35.171 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.503+0000 7f01e2c16700 1 -- 172.21.15.93:0/527308078 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01dc10bc70 con 0x7f01dc07bde0 2023-11-16T16:39:35.171 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.504+0000 7f01d9ffb700 1 -- 172.21.15.93:0/527308078 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f01d0024ad0 con 0x7f01dc07bde0 2023-11-16T16:39:35.171 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.504+0000 7f01e2c16700 1 -- 172.21.15.93:0/527308078 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01dc062120 con 0x7f01dc07bde0 2023-11-16T16:39:35.171 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.505+0000 7f01d9ffb700 1 --2- 172.21.15.93:0/527308078 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f01cc038820 0x7f01cc03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:35.171 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.505+0000 7f01d9ffb700 1 -- 172.21.15.93:0/527308078 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f01d004c790 con 0x7f01dc07bde0 2023-11-16T16:39:35.171 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.505+0000 7f01dbfff700 1 --2- 172.21.15.93:0/527308078 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f01cc038820 0x7f01cc03acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:35.172 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.506+0000 7f01dbfff700 1 --2- 172.21.15.93:0/527308078 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f01cc038820 0x7f01cc03acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f01c8006fd0 tx=0x7f01c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:35.172 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.510+0000 7f01d9ffb700 1 -- 172.21.15.93:0/527308078 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f01d0024430 con 0x7f01dc07bde0 2023-11-16T16:39:35.172 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:31.711+0000 7f01e2c16700 1 -- 172.21.15.93:0/527308078 --> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi093", "addr": "172.21.15.93", "target": ["mon-mgr", ""]}) v1 -- 0x7f01dc102390 con 0x7f01cc038820 2023-11-16T16:39:35.172 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:34.713+0000 7f01d9ffb700 1 -- 172.21.15.93:0/527308078 <== mgr.14120 v2:172.21.15.93:6800/2198359920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f01dc102390 con 0x7f01cc038820 2023-11-16T16:39:35.172 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:34.716+0000 7f01e2c16700 1 -- 172.21.15.93:0/527308078 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f01cc038820 msgr2=0x7f01cc03acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:35.172 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:34.716+0000 7f01e2c16700 1 --2- 172.21.15.93:0/527308078 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f01cc038820 0x7f01cc03acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f01c8006fd0 tx=0x7f01c8006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:35.172 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:34.716+0000 7f01e2c16700 1 -- 172.21.15.93:0/527308078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 msgr2=0x7f01dc19b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:35.173 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:34.716+0000 7f01e2c16700 1 --2- 172.21.15.93:0/527308078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 0x7f01dc19b780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01d0006af0 tx=0x7f01d0011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:35.173 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:34.716+0000 7f01e2c16700 1 -- 172.21.15.93:0/527308078 shutdown_connections 2023-11-16T16:39:35.173 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:34.716+0000 7f01e2c16700 1 --2- 172.21.15.93:0/527308078 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f01cc038820 0x7f01cc03acd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:35.173 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:34.716+0000 7f01e2c16700 1 --2- 172.21.15.93:0/527308078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc07bde0 0x7f01dc19b780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:35.173 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:34.716+0000 7f01e2c16700 1 -- 172.21.15.93:0/527308078 >> 172.21.15.93:0/527308078 conn(0x7f01dc100fa0 msgr2=0x7f01dc101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:35.173 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:34.716+0000 7f01e2c16700 1 -- 172.21.15.93:0/527308078 shutdown_connections 2023-11-16T16:39:35.173 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:34.716+0000 7f01e2c16700 1 -- 172.21.15.93:0/527308078 wait complete. 2023-11-16T16:39:35.173 INFO:teuthology.orchestra.run.smithi093.stdout:Deploying unmanaged mon service... 2023-11-16T16:39:35.983 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:35 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:35.984 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:35 smithi093 ceph-mon[97863]: Added host smithi093 2023-11-16T16:39:35.984 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:35 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:36.435 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-11-16T16:39:36.435 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.725+0000 7f438b59e700 1 Processor -- start 2023-11-16T16:39:36.435 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.725+0000 7f438b59e700 1 -- start start 2023-11-16T16:39:36.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.726+0000 7f438b59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 0x7f438c105b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:36.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.726+0000 7f438b59e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c1060a0 con 0x7f438c105750 2023-11-16T16:39:36.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.726+0000 7f438a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 0x7f438c105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:36.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.726+0000 7f438a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 0x7f438c105b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36436/0 (socket says 172.21.15.93:36436) 2023-11-16T16:39:36.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.726+0000 7f438a59c700 1 -- 172.21.15.93:0/908050493 learned_addr learned my addr 172.21.15.93:0/908050493 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:36.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.726+0000 7f438a59c700 1 -- 172.21.15.93:0/908050493 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c1061e0 con 0x7f438c105750 2023-11-16T16:39:36.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.726+0000 7f438a59c700 1 --2- 172.21.15.93:0/908050493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 0x7f438c105b60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f437c0098a0 tx=0x7f437c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db024b0efb15d079 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:36.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.727+0000 7f438959a700 1 -- 172.21.15.93:0/908050493 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f437c004160 con 0x7f438c105750 2023-11-16T16:39:36.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.727+0000 7f438959a700 1 -- 172.21.15.93:0/908050493 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f437c0042c0 con 0x7f438c105750 2023-11-16T16:39:36.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.727+0000 7f438959a700 1 -- 172.21.15.93:0/908050493 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f437c011a00 con 0x7f438c105750 2023-11-16T16:39:36.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.727+0000 7f438b59e700 1 -- 172.21.15.93:0/908050493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 msgr2=0x7f438c105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:36.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.727+0000 7f438b59e700 1 --2- 172.21.15.93:0/908050493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 0x7f438c105b60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f437c0098a0 tx=0x7f437c009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:36.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.727+0000 7f438b59e700 1 -- 172.21.15.93:0/908050493 shutdown_connections 2023-11-16T16:39:36.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.727+0000 7f438b59e700 1 --2- 172.21.15.93:0/908050493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 0x7f438c105b60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:36.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.727+0000 7f438b59e700 1 -- 172.21.15.93:0/908050493 >> 172.21.15.93:0/908050493 conn(0x7f438c100dc0 msgr2=0x7f438c1031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:36.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.727+0000 7f438b59e700 1 -- 172.21.15.93:0/908050493 shutdown_connections 2023-11-16T16:39:36.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.728+0000 7f438b59e700 1 -- 172.21.15.93:0/908050493 wait complete. 2023-11-16T16:39:36.438 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.728+0000 7f438b59e700 1 Processor -- start 2023-11-16T16:39:36.438 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.729+0000 7f438b59e700 1 -- start start 2023-11-16T16:39:36.438 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.729+0000 7f438b59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 0x7f438c19b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:36.438 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.729+0000 7f438b59e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c19bb20 con 0x7f438c105750 2023-11-16T16:39:36.438 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.729+0000 7f438a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 0x7f438c19b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:36.438 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.729+0000 7f438a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 0x7f438c19b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36444/0 (socket says 172.21.15.93:36444) 2023-11-16T16:39:36.438 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.729+0000 7f438a59c700 1 -- 172.21.15.93:0/2984497360 learned_addr learned my addr 172.21.15.93:0/2984497360 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:36.438 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.729+0000 7f438a59c700 1 -- 172.21.15.93:0/2984497360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437c009550 con 0x7f438c105750 2023-11-16T16:39:36.438 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.730+0000 7f438a59c700 1 --2- 172.21.15.93:0/2984497360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 0x7f438c19b5e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f437c009130 tx=0x7f437c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:36.439 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.730+0000 7f437b7fe700 1 -- 172.21.15.93:0/2984497360 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f437c003b70 con 0x7f438c105750 2023-11-16T16:39:36.439 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.730+0000 7f437b7fe700 1 -- 172.21.15.93:0/2984497360 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f437c003cd0 con 0x7f438c105750 2023-11-16T16:39:36.439 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.730+0000 7f438b59e700 1 -- 172.21.15.93:0/2984497360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438c1077a0 con 0x7f438c105750 2023-11-16T16:39:36.439 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.730+0000 7f437b7fe700 1 -- 172.21.15.93:0/2984497360 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f437c0197d0 con 0x7f438c105750 2023-11-16T16:39:36.439 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.730+0000 7f438b59e700 1 -- 172.21.15.93:0/2984497360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438c107610 con 0x7f438c105750 2023-11-16T16:39:36.439 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.731+0000 7f437b7fe700 1 -- 172.21.15.93:0/2984497360 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f437c0199c0 con 0x7f438c105750 2023-11-16T16:39:36.439 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.731+0000 7f438b59e700 1 -- 172.21.15.93:0/2984497360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438c062120 con 0x7f438c105750 2023-11-16T16:39:36.439 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.731+0000 7f437b7fe700 1 --2- 172.21.15.93:0/2984497360 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f4374038820 0x7f437403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:36.440 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.732+0000 7f437b7fe700 1 -- 172.21.15.93:0/2984497360 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f437c04cb30 con 0x7f438c105750 2023-11-16T16:39:36.440 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.732+0000 7f4389d9b700 1 --2- 172.21.15.93:0/2984497360 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f4374038820 0x7f437403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:36.440 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.732+0000 7f4389d9b700 1 --2- 172.21.15.93:0/2984497360 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f4374038820 0x7f437403acd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4380006fd0 tx=0x7f4380006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:36.440 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.737+0000 7f437b7fe700 1 -- 172.21.15.93:0/2984497360 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f437c018530 con 0x7f438c105750 2023-11-16T16:39:36.440 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.953+0000 7f438b59e700 1 -- 172.21.15.93:0/2984497360 --> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f438c102150 con 0x7f4374038820 2023-11-16T16:39:36.440 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.958+0000 7f437b7fe700 1 -- 172.21.15.93:0/2984497360 <== mgr.14120 v2:172.21.15.93:6800/2198359920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f438c102150 con 0x7f4374038820 2023-11-16T16:39:36.440 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.960+0000 7f438b59e700 1 -- 172.21.15.93:0/2984497360 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f4374038820 msgr2=0x7f437403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:36.440 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.960+0000 7f438b59e700 1 --2- 172.21.15.93:0/2984497360 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f4374038820 0x7f437403acd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4380006fd0 tx=0x7f4380006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:36.440 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.960+0000 7f438b59e700 1 -- 172.21.15.93:0/2984497360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 msgr2=0x7f438c19b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:36.441 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.960+0000 7f438b59e700 1 --2- 172.21.15.93:0/2984497360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 0x7f438c19b5e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f437c009130 tx=0x7f437c011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:36.441 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.960+0000 7f438b59e700 1 -- 172.21.15.93:0/2984497360 shutdown_connections 2023-11-16T16:39:36.441 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.960+0000 7f438b59e700 1 --2- 172.21.15.93:0/2984497360 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f4374038820 0x7f437403acd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:36.441 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.960+0000 7f438b59e700 1 --2- 172.21.15.93:0/2984497360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f438c105750 0x7f438c19b5e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:36.441 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.960+0000 7f438b59e700 1 -- 172.21.15.93:0/2984497360 >> 172.21.15.93:0/2984497360 conn(0x7f438c100dc0 msgr2=0x7f438c101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:36.441 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.961+0000 7f438b59e700 1 -- 172.21.15.93:0/2984497360 shutdown_connections 2023-11-16T16:39:36.441 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:35.961+0000 7f438b59e700 1 -- 172.21.15.93:0/2984497360 wait complete. 2023-11-16T16:39:36.441 INFO:teuthology.orchestra.run.smithi093.stdout:Deploying unmanaged mgr service... 2023-11-16T16:39:37.113 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:36 smithi093 ceph-mon[97863]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:37.113 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:36 smithi093 ceph-mon[97863]: Saving service mon spec with placement count:5 2023-11-16T16:39:37.113 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:36 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:37.819 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-11-16T16:39:37.819 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.156+0000 7f317f86e700 1 Processor -- start 2023-11-16T16:39:37.819 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.156+0000 7f317f86e700 1 -- start start 2023-11-16T16:39:37.819 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.156+0000 7f317f86e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 0x7f317807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:37.820 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.156+0000 7f317f86e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317807a820 con 0x7f317807bde0 2023-11-16T16:39:37.820 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.157+0000 7f317d60a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 0x7f317807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:37.820 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.157+0000 7f317d60a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 0x7f317807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38252/0 (socket says 172.21.15.93:38252) 2023-11-16T16:39:37.820 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.157+0000 7f317d60a700 1 -- 172.21.15.93:0/3938755292 learned_addr learned my addr 172.21.15.93:0/3938755292 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:37.820 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.157+0000 7f317d60a700 1 -- 172.21.15.93:0/3938755292 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317807a960 con 0x7f317807bde0 2023-11-16T16:39:37.820 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.157+0000 7f317d60a700 1 --2- 172.21.15.93:0/3938755292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 0x7f317807a2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f31740098a0 tx=0x7f3174009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801b2a826e9d1e86 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:37.820 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.157+0000 7f316ffff700 1 -- 172.21.15.93:0/3938755292 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3174004160 con 0x7f317807bde0 2023-11-16T16:39:37.820 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.157+0000 7f316ffff700 1 -- 172.21.15.93:0/3938755292 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f31740042c0 con 0x7f317807bde0 2023-11-16T16:39:37.820 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.157+0000 7f316ffff700 1 -- 172.21.15.93:0/3938755292 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3174011a00 con 0x7f317807bde0 2023-11-16T16:39:37.821 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.158+0000 7f317f86e700 1 -- 172.21.15.93:0/3938755292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 msgr2=0x7f317807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:37.821 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.158+0000 7f317f86e700 1 --2- 172.21.15.93:0/3938755292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 0x7f317807a2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f31740098a0 tx=0x7f3174009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:37.821 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.158+0000 7f317f86e700 1 -- 172.21.15.93:0/3938755292 shutdown_connections 2023-11-16T16:39:37.821 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.158+0000 7f317f86e700 1 --2- 172.21.15.93:0/3938755292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 0x7f317807a2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:37.821 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.158+0000 7f317f86e700 1 -- 172.21.15.93:0/3938755292 >> 172.21.15.93:0/3938755292 conn(0x7f3178100fa0 msgr2=0x7f31781033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:37.821 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.158+0000 7f317f86e700 1 -- 172.21.15.93:0/3938755292 shutdown_connections 2023-11-16T16:39:37.821 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.158+0000 7f317f86e700 1 -- 172.21.15.93:0/3938755292 wait complete. 2023-11-16T16:39:37.821 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.159+0000 7f317f86e700 1 Processor -- start 2023-11-16T16:39:37.821 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.159+0000 7f317f86e700 1 -- start start 2023-11-16T16:39:37.822 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.160+0000 7f317f86e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 0x7f317819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:37.822 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.160+0000 7f317f86e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317819bcc0 con 0x7f317807bde0 2023-11-16T16:39:37.822 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.160+0000 7f317d60a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 0x7f317819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:37.822 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.160+0000 7f317d60a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 0x7f317819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38262/0 (socket says 172.21.15.93:38262) 2023-11-16T16:39:37.822 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.160+0000 7f317d60a700 1 -- 172.21.15.93:0/937657709 learned_addr learned my addr 172.21.15.93:0/937657709 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:37.822 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.160+0000 7f317d60a700 1 -- 172.21.15.93:0/937657709 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3174009550 con 0x7f317807bde0 2023-11-16T16:39:37.822 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.160+0000 7f317d60a700 1 --2- 172.21.15.93:0/937657709 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 0x7f317819b780 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3174006af0 tx=0x7f3174011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:37.822 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.161+0000 7f316e7fc700 1 -- 172.21.15.93:0/937657709 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3174003a40 con 0x7f317807bde0 2023-11-16T16:39:37.822 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.161+0000 7f316e7fc700 1 -- 172.21.15.93:0/937657709 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3174003ba0 con 0x7f317807bde0 2023-11-16T16:39:37.822 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.161+0000 7f317f86e700 1 -- 172.21.15.93:0/937657709 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317810be00 con 0x7f317807bde0 2023-11-16T16:39:37.823 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.161+0000 7f316e7fc700 1 -- 172.21.15.93:0/937657709 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f317401a700 con 0x7f317807bde0 2023-11-16T16:39:37.823 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.161+0000 7f317f86e700 1 -- 172.21.15.93:0/937657709 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317810bc70 con 0x7f317807bde0 2023-11-16T16:39:37.823 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.162+0000 7f316e7fc700 1 -- 172.21.15.93:0/937657709 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f3174024430 con 0x7f317807bde0 2023-11-16T16:39:37.823 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.162+0000 7f317f86e700 1 -- 172.21.15.93:0/937657709 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f317810bf90 con 0x7f317807bde0 2023-11-16T16:39:37.823 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.162+0000 7f316e7fc700 1 --2- 172.21.15.93:0/937657709 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f3164038820 0x7f316403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:37.823 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.162+0000 7f316e7fc700 1 -- 172.21.15.93:0/937657709 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f317404c710 con 0x7f317807bde0 2023-11-16T16:39:37.823 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.163+0000 7f317ce09700 1 --2- 172.21.15.93:0/937657709 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f3164038820 0x7f316403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:37.823 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.163+0000 7f317ce09700 1 --2- 172.21.15.93:0/937657709 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f3164038820 0x7f316403acd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3168006fd0 tx=0x7f3168006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:37.823 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.167+0000 7f316e7fc700 1 -- 172.21.15.93:0/937657709 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f31740189a0 con 0x7f317807bde0 2023-11-16T16:39:37.824 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.380+0000 7f317f86e700 1 -- 172.21.15.93:0/937657709 --> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3178102390 con 0x7f3164038820 2023-11-16T16:39:37.824 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.388+0000 7f316e7fc700 1 -- 172.21.15.93:0/937657709 <== mgr.14120 v2:172.21.15.93:6800/2198359920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3178102390 con 0x7f3164038820 2023-11-16T16:39:37.824 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.390+0000 7f317f86e700 1 -- 172.21.15.93:0/937657709 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f3164038820 msgr2=0x7f316403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:37.824 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.390+0000 7f317f86e700 1 --2- 172.21.15.93:0/937657709 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f3164038820 0x7f316403acd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3168006fd0 tx=0x7f3168006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:37.824 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.390+0000 7f317f86e700 1 -- 172.21.15.93:0/937657709 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 msgr2=0x7f317819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:37.824 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.390+0000 7f317f86e700 1 --2- 172.21.15.93:0/937657709 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 0x7f317819b780 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3174006af0 tx=0x7f3174011ad0 comp rx=0 tx=0).stop 2023-11-16T16:39:37.824 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.390+0000 7f317f86e700 1 -- 172.21.15.93:0/937657709 shutdown_connections 2023-11-16T16:39:37.824 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.390+0000 7f317f86e700 1 --2- 172.21.15.93:0/937657709 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f3164038820 0x7f316403acd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:37.824 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.390+0000 7f317f86e700 1 --2- 172.21.15.93:0/937657709 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f317807bde0 0x7f317819b780 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:37.825 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.390+0000 7f317f86e700 1 -- 172.21.15.93:0/937657709 >> 172.21.15.93:0/937657709 conn(0x7f3178100fa0 msgr2=0x7f3178101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:37.825 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.390+0000 7f317f86e700 1 -- 172.21.15.93:0/937657709 shutdown_connections 2023-11-16T16:39:37.825 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:37.391+0000 7f317f86e700 1 -- 172.21.15.93:0/937657709 wait complete. 2023-11-16T16:39:38.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:38 smithi093 ceph-mon[97863]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:38.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:38 smithi093 ceph-mon[97863]: Saving service mgr spec with placement count:2 2023-11-16T16:39:38.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:38 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:38.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:38 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:38.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:38 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:39.154 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.477+0000 7fd3e87d7700 1 Processor -- start 2023-11-16T16:39:39.154 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.477+0000 7fd3e87d7700 1 -- start start 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.478+0000 7fd3e87d7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 0x7fd3e0105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.478+0000 7fd3e87d7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e0106260 con 0x7fd3e0105910 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.478+0000 7fd3e6573700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 0x7fd3e0105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.478+0000 7fd3e6573700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 0x7fd3e0105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38272/0 (socket says 172.21.15.93:38272) 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.478+0000 7fd3e6573700 1 -- 172.21.15.93:0/3742306643 learned_addr learned my addr 172.21.15.93:0/3742306643 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.479+0000 7fd3e6573700 1 -- 172.21.15.93:0/3742306643 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e01063a0 con 0x7fd3e0105910 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.479+0000 7fd3e6573700 1 --2- 172.21.15.93:0/3742306643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 0x7fd3e0105d20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0098a0 tx=0x7fd3dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad28cf9cf00c1931 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.479+0000 7fd3e5571700 1 -- 172.21.15.93:0/3742306643 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3dc004160 con 0x7fd3e0105910 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.479+0000 7fd3e5571700 1 -- 172.21.15.93:0/3742306643 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3dc0042c0 con 0x7fd3e0105910 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.479+0000 7fd3e5571700 1 -- 172.21.15.93:0/3742306643 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3dc0119b0 con 0x7fd3e0105910 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.480+0000 7fd3e87d7700 1 -- 172.21.15.93:0/3742306643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 msgr2=0x7fd3e0105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.480+0000 7fd3e87d7700 1 --2- 172.21.15.93:0/3742306643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 0x7fd3e0105d20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0098a0 tx=0x7fd3dc009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:39.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.480+0000 7fd3e87d7700 1 -- 172.21.15.93:0/3742306643 shutdown_connections 2023-11-16T16:39:39.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.480+0000 7fd3e87d7700 1 --2- 172.21.15.93:0/3742306643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 0x7fd3e0105d20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:39.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.480+0000 7fd3e87d7700 1 -- 172.21.15.93:0/3742306643 >> 172.21.15.93:0/3742306643 conn(0x7fd3e0100fa0 msgr2=0x7fd3e01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:39.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.480+0000 7fd3e87d7700 1 -- 172.21.15.93:0/3742306643 shutdown_connections 2023-11-16T16:39:39.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.480+0000 7fd3e87d7700 1 -- 172.21.15.93:0/3742306643 wait complete. 2023-11-16T16:39:39.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.481+0000 7fd3e87d7700 1 Processor -- start 2023-11-16T16:39:39.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.481+0000 7fd3e87d7700 1 -- start start 2023-11-16T16:39:39.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.482+0000 7fd3e87d7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 0x7fd3e019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:39.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.482+0000 7fd3e87d7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e019bca0 con 0x7fd3e0105910 2023-11-16T16:39:39.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.482+0000 7fd3e6573700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 0x7fd3e019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:39.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.482+0000 7fd3e6573700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 0x7fd3e019b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38286/0 (socket says 172.21.15.93:38286) 2023-11-16T16:39:39.156 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.482+0000 7fd3e6573700 1 -- 172.21.15.93:0/3108248364 learned_addr learned my addr 172.21.15.93:0/3108248364 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:39.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.482+0000 7fd3e6573700 1 -- 172.21.15.93:0/3108248364 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc009550 con 0x7fd3e0105910 2023-11-16T16:39:39.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.483+0000 7fd3e6573700 1 --2- 172.21.15.93:0/3108248364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 0x7fd3e019b760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc011980 tx=0x7fd3dc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:39.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.483+0000 7fd3d77fe700 1 -- 172.21.15.93:0/3108248364 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3dc003a60 con 0x7fd3e0105910 2023-11-16T16:39:39.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.483+0000 7fd3e87d7700 1 -- 172.21.15.93:0/3108248364 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e01079a0 con 0x7fd3e0105910 2023-11-16T16:39:39.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.483+0000 7fd3d77fe700 1 -- 172.21.15.93:0/3108248364 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3dc003bc0 con 0x7fd3e0105910 2023-11-16T16:39:39.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.483+0000 7fd3d77fe700 1 -- 172.21.15.93:0/3108248364 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3dc01a670 con 0x7fd3e0105910 2023-11-16T16:39:39.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.483+0000 7fd3e87d7700 1 -- 172.21.15.93:0/3108248364 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e0107810 con 0x7fd3e0105910 2023-11-16T16:39:39.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.484+0000 7fd3d77fe700 1 -- 172.21.15.93:0/3108248364 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fd3dc024440 con 0x7fd3e0105910 2023-11-16T16:39:39.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.484+0000 7fd3d57fa700 1 -- 172.21.15.93:0/3108248364 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e0109da0 con 0x7fd3e0105910 2023-11-16T16:39:39.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.485+0000 7fd3d77fe700 1 --2- 172.21.15.93:0/3108248364 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fd3cc038820 0x7fd3cc03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:39.157 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.485+0000 7fd3d77fe700 1 -- 172.21.15.93:0/3108248364 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd3dc04c8b0 con 0x7fd3e0105910 2023-11-16T16:39:39.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.485+0000 7fd3e5d72700 1 --2- 172.21.15.93:0/3108248364 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fd3cc038820 0x7fd3cc03acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:39.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.486+0000 7fd3e5d72700 1 --2- 172.21.15.93:0/3108248364 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fd3cc038820 0x7fd3cc03acd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0006fd0 tx=0x7fd3d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:39.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.489+0000 7fd3d77fe700 1 -- 172.21.15.93:0/3108248364 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd3dc0246f0 con 0x7fd3e0105910 2023-11-16T16:39:39.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.694+0000 7fd3d57fa700 1 -- 172.21.15.93:0/3108248364 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fd3e0061b20 con 0x7fd3e0105910 2023-11-16T16:39:39.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.700+0000 7fd3d77fe700 1 -- 172.21.15.93:0/3108248364 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd3dc018350 con 0x7fd3e0105910 2023-11-16T16:39:39.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.703+0000 7fd3d57fa700 1 -- 172.21.15.93:0/3108248364 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fd3cc038820 msgr2=0x7fd3cc03acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:39.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.703+0000 7fd3d57fa700 1 --2- 172.21.15.93:0/3108248364 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fd3cc038820 0x7fd3cc03acd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0006fd0 tx=0x7fd3d0006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:39.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.703+0000 7fd3d57fa700 1 -- 172.21.15.93:0/3108248364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 msgr2=0x7fd3e019b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:39.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.703+0000 7fd3d57fa700 1 --2- 172.21.15.93:0/3108248364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 0x7fd3e019b760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc011980 tx=0x7fd3dc011a60 comp rx=0 tx=0).stop 2023-11-16T16:39:39.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.703+0000 7fd3d57fa700 1 -- 172.21.15.93:0/3108248364 shutdown_connections 2023-11-16T16:39:39.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.703+0000 7fd3d57fa700 1 --2- 172.21.15.93:0/3108248364 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fd3cc038820 0x7fd3cc03acd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:39.158 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.703+0000 7fd3d57fa700 1 --2- 172.21.15.93:0/3108248364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3e0105910 0x7fd3e019b760 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:39.159 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.703+0000 7fd3d57fa700 1 -- 172.21.15.93:0/3108248364 >> 172.21.15.93:0/3108248364 conn(0x7fd3e0100fa0 msgr2=0x7fd3e0101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:39.159 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.703+0000 7fd3d57fa700 1 -- 172.21.15.93:0/3108248364 shutdown_connections 2023-11-16T16:39:39.159 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:38.703+0000 7fd3d57fa700 1 -- 172.21.15.93:0/3108248364 wait complete. 2023-11-16T16:39:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:39 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3108248364' entity='client.admin' 2023-11-16T16:39:40.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.738+0000 7f32c9479700 1 Processor -- start 2023-11-16T16:39:40.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.738+0000 7f32c9479700 1 -- start start 2023-11-16T16:39:40.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.739+0000 7f32c9479700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 0x7f32c4105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:40.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.739+0000 7f32c9479700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c4106260 con 0x7f32c4105910 2023-11-16T16:39:40.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.739+0000 7f32c2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 0x7f32c4105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:40.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.739+0000 7f32c2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 0x7f32c4105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38292/0 (socket says 172.21.15.93:38292) 2023-11-16T16:39:40.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.739+0000 7f32c2ffd700 1 -- 172.21.15.93:0/2646190616 learned_addr learned my addr 172.21.15.93:0/2646190616 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:40.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.739+0000 7f32c2ffd700 1 -- 172.21.15.93:0/2646190616 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c41063a0 con 0x7f32c4105910 2023-11-16T16:39:40.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.740+0000 7f32c2ffd700 1 --2- 172.21.15.93:0/2646190616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 0x7f32c4105d20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f32b40098a0 tx=0x7f32b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a64e0b340252b1d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:40.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.740+0000 7f32c1ffb700 1 -- 172.21.15.93:0/2646190616 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32b4004160 con 0x7f32c4105910 2023-11-16T16:39:40.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.740+0000 7f32c1ffb700 1 -- 172.21.15.93:0/2646190616 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32b40042c0 con 0x7f32c4105910 2023-11-16T16:39:40.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.740+0000 7f32c1ffb700 1 -- 172.21.15.93:0/2646190616 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32b40119b0 con 0x7f32c4105910 2023-11-16T16:39:40.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.740+0000 7f32c9479700 1 -- 172.21.15.93:0/2646190616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 msgr2=0x7f32c4105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:40.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.740+0000 7f32c9479700 1 --2- 172.21.15.93:0/2646190616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 0x7f32c4105d20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f32b40098a0 tx=0x7f32b4009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:40.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.741+0000 7f32c9479700 1 -- 172.21.15.93:0/2646190616 shutdown_connections 2023-11-16T16:39:40.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.741+0000 7f32c9479700 1 --2- 172.21.15.93:0/2646190616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 0x7f32c4105d20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:40.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.741+0000 7f32c9479700 1 -- 172.21.15.93:0/2646190616 >> 172.21.15.93:0/2646190616 conn(0x7f32c4100fa0 msgr2=0x7f32c41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:40.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.741+0000 7f32c9479700 1 -- 172.21.15.93:0/2646190616 shutdown_connections 2023-11-16T16:39:40.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.741+0000 7f32c9479700 1 -- 172.21.15.93:0/2646190616 wait complete. 2023-11-16T16:39:40.540 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.742+0000 7f32c9479700 1 Processor -- start 2023-11-16T16:39:40.540 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.743+0000 7f32c9479700 1 -- start start 2023-11-16T16:39:40.540 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.743+0000 7f32c9479700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 0x7f32c407bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:40.540 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.743+0000 7f32c9479700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c407c1b0 con 0x7f32c4105910 2023-11-16T16:39:40.540 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.744+0000 7f32c2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 0x7f32c407bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:40.540 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.744+0000 7f32c2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 0x7f32c407bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38306/0 (socket says 172.21.15.93:38306) 2023-11-16T16:39:40.540 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.744+0000 7f32c2ffd700 1 -- 172.21.15.93:0/1461351679 learned_addr learned my addr 172.21.15.93:0/1461351679 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:40.540 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.744+0000 7f32c2ffd700 1 -- 172.21.15.93:0/1461351679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b4009550 con 0x7f32c4105910 2023-11-16T16:39:40.540 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.744+0000 7f32c2ffd700 1 --2- 172.21.15.93:0/1461351679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 0x7f32c407bc70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32b4011980 tx=0x7f32b4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:40.540 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.745+0000 7f32abfff700 1 -- 172.21.15.93:0/1461351679 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32b4003a60 con 0x7f32c4105910 2023-11-16T16:39:40.540 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.745+0000 7f32abfff700 1 -- 172.21.15.93:0/1461351679 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32b4003bc0 con 0x7f32c4105910 2023-11-16T16:39:40.540 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.745+0000 7f32c9479700 1 -- 172.21.15.93:0/1461351679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c41079a0 con 0x7f32c4105910 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.745+0000 7f32abfff700 1 -- 172.21.15.93:0/1461351679 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32b401a670 con 0x7f32c4105910 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.745+0000 7f32c9479700 1 -- 172.21.15.93:0/1461351679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c4107810 con 0x7f32c4105910 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.746+0000 7f32abfff700 1 -- 172.21.15.93:0/1461351679 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f32b401a8e0 con 0x7f32c4105910 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.746+0000 7f32c9479700 1 -- 172.21.15.93:0/1461351679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c4062120 con 0x7f32c4105910 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.746+0000 7f32abfff700 1 --2- 172.21.15.93:0/1461351679 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f32ac038820 0x7f32ac03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.746+0000 7f32abfff700 1 -- 172.21.15.93:0/1461351679 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f32b404c840 con 0x7f32c4105910 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.747+0000 7f32c27fc700 1 --2- 172.21.15.93:0/1461351679 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f32ac038820 0x7f32ac03acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.747+0000 7f32c27fc700 1 --2- 172.21.15.93:0/1461351679 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f32ac038820 0x7f32ac03acd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f32b8006fd0 tx=0x7f32b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.751+0000 7f32abfff700 1 -- 172.21.15.93:0/1461351679 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f32b401ab90 con 0x7f32c4105910 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.955+0000 7f32c9479700 1 -- 172.21.15.93:0/1461351679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f32c407b180 con 0x7f32c4105910 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.958+0000 7f32abfff700 1 -- 172.21.15.93:0/1461351679 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f32b4018350 con 0x7f32c4105910 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.961+0000 7f32c9479700 1 -- 172.21.15.93:0/1461351679 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f32ac038820 msgr2=0x7f32ac03acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:40.541 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.961+0000 7f32c9479700 1 --2- 172.21.15.93:0/1461351679 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f32ac038820 0x7f32ac03acd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f32b8006fd0 tx=0x7f32b8006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:40.542 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.961+0000 7f32c9479700 1 -- 172.21.15.93:0/1461351679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 msgr2=0x7f32c407bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:40.542 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.961+0000 7f32c9479700 1 --2- 172.21.15.93:0/1461351679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 0x7f32c407bc70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32b4011980 tx=0x7f32b4011a60 comp rx=0 tx=0).stop 2023-11-16T16:39:40.542 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.961+0000 7f32c9479700 1 -- 172.21.15.93:0/1461351679 shutdown_connections 2023-11-16T16:39:40.542 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.961+0000 7f32c9479700 1 --2- 172.21.15.93:0/1461351679 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f32ac038820 0x7f32ac03acd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:40.542 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.961+0000 7f32c9479700 1 --2- 172.21.15.93:0/1461351679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c4105910 0x7f32c407bc70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:40.542 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.961+0000 7f32c9479700 1 -- 172.21.15.93:0/1461351679 >> 172.21.15.93:0/1461351679 conn(0x7f32c4100fa0 msgr2=0x7f32c4101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:40.542 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.961+0000 7f32c9479700 1 -- 172.21.15.93:0/1461351679 shutdown_connections 2023-11-16T16:39:40.542 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:39.961+0000 7f32c9479700 1 -- 172.21.15.93:0/1461351679 wait complete. 2023-11-16T16:39:40.542 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling the dashboard module... 2023-11-16T16:39:41.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:40 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1461351679' entity='client.admin' 2023-11-16T16:39:41.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:40 smithi093 ceph-mon[97863]: from='mgr.14120 172.21.15.93:0/939328234' entity='mgr.x' 2023-11-16T16:39:42.261 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:41 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/399422970' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-11-16T16:39:42.262 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:42 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: ignoring --setuser ceph since I am not root 2023-11-16T16:39:42.262 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:42 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: ignoring --setgroup ceph since I am not root 2023-11-16T16:39:42.420 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.092+0000 7ff1a8f7b700 1 Processor -- start 2023-11-16T16:39:42.420 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.092+0000 7ff1a8f7b700 1 -- start start 2023-11-16T16:39:42.420 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.092+0000 7ff1a8f7b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 0x7ff1a4105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:42.420 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.092+0000 7ff1a8f7b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4106260 con 0x7ff1a4105910 2023-11-16T16:39:42.420 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.093+0000 7ff1a259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 0x7ff1a4105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:42.420 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.093+0000 7ff1a259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 0x7ff1a4105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38322/0 (socket says 172.21.15.93:38322) 2023-11-16T16:39:42.420 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.093+0000 7ff1a259c700 1 -- 172.21.15.93:0/1589902365 learned_addr learned my addr 172.21.15.93:0/1589902365 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:42.421 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.093+0000 7ff1a259c700 1 -- 172.21.15.93:0/1589902365 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a41063a0 con 0x7ff1a4105910 2023-11-16T16:39:42.421 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.093+0000 7ff1a259c700 1 --2- 172.21.15.93:0/1589902365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 0x7ff1a4105d20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff1940098a0 tx=0x7ff194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea29278abf250fdc server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:42.421 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.093+0000 7ff1a159a700 1 -- 172.21.15.93:0/1589902365 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff194004160 con 0x7ff1a4105910 2023-11-16T16:39:42.421 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.094+0000 7ff1a159a700 1 -- 172.21.15.93:0/1589902365 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1940042c0 con 0x7ff1a4105910 2023-11-16T16:39:42.421 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.094+0000 7ff1a159a700 1 -- 172.21.15.93:0/1589902365 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1940119b0 con 0x7ff1a4105910 2023-11-16T16:39:42.421 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.094+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/1589902365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 msgr2=0x7ff1a4105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:42.421 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.094+0000 7ff1a8f7b700 1 --2- 172.21.15.93:0/1589902365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 0x7ff1a4105d20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff1940098a0 tx=0x7ff194009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:42.421 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.094+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/1589902365 shutdown_connections 2023-11-16T16:39:42.421 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.094+0000 7ff1a8f7b700 1 --2- 172.21.15.93:0/1589902365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 0x7ff1a4105d20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:42.421 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.094+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/1589902365 >> 172.21.15.93:0/1589902365 conn(0x7ff1a4100fa0 msgr2=0x7ff1a41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:42.421 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.094+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/1589902365 shutdown_connections 2023-11-16T16:39:42.422 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.095+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/1589902365 wait complete. 2023-11-16T16:39:42.422 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.096+0000 7ff1a8f7b700 1 Processor -- start 2023-11-16T16:39:42.422 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.096+0000 7ff1a8f7b700 1 -- start start 2023-11-16T16:39:42.422 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.096+0000 7ff1a8f7b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 0x7ff1a419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:42.422 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.096+0000 7ff1a8f7b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a419bcd0 con 0x7ff1a4105910 2023-11-16T16:39:42.422 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.096+0000 7ff1a259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 0x7ff1a419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:42.422 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.096+0000 7ff1a259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 0x7ff1a419b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38324/0 (socket says 172.21.15.93:38324) 2023-11-16T16:39:42.422 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.096+0000 7ff1a259c700 1 -- 172.21.15.93:0/399422970 learned_addr learned my addr 172.21.15.93:0/399422970 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:42.422 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.097+0000 7ff1a259c700 1 -- 172.21.15.93:0/399422970 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff194009550 con 0x7ff1a4105910 2023-11-16T16:39:42.422 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.097+0000 7ff1a259c700 1 --2- 172.21.15.93:0/399422970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 0x7ff1a419b790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff194003e60 tx=0x7ff194011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:42.422 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.097+0000 7ff1937fe700 1 -- 172.21.15.93:0/399422970 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff194003970 con 0x7ff1a4105910 2023-11-16T16:39:42.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.097+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/399422970 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a41079a0 con 0x7ff1a4105910 2023-11-16T16:39:42.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.097+0000 7ff1937fe700 1 -- 172.21.15.93:0/399422970 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff194003ad0 con 0x7ff1a4105910 2023-11-16T16:39:42.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.097+0000 7ff1937fe700 1 -- 172.21.15.93:0/399422970 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1940195e0 con 0x7ff1a4105910 2023-11-16T16:39:42.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.097+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/399422970 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a4107810 con 0x7ff1a4105910 2023-11-16T16:39:42.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.098+0000 7ff1937fe700 1 -- 172.21.15.93:0/399422970 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ff1940198a0 con 0x7ff1a4105910 2023-11-16T16:39:42.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.098+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/399422970 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a4109de0 con 0x7ff1a4105910 2023-11-16T16:39:42.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.099+0000 7ff1937fe700 1 --2- 172.21.15.93:0/399422970 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7ff18c038780 0x7ff18c03ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:42.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.099+0000 7ff1937fe700 1 -- 172.21.15.93:0/399422970 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff19404cb60 con 0x7ff1a4105910 2023-11-16T16:39:42.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.099+0000 7ff1a1d9b700 1 --2- 172.21.15.93:0/399422970 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7ff18c038780 0x7ff18c03ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:42.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.100+0000 7ff1a1d9b700 1 --2- 172.21.15.93:0/399422970 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7ff18c038780 0x7ff18c03ac30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff198006fd0 tx=0x7ff198006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:42.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.104+0000 7ff1937fe700 1 -- 172.21.15.93:0/399422970 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff19402b340 con 0x7ff1a4105910 2023-11-16T16:39:42.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.395+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/399422970 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7ff1a419f3a0 con 0x7ff1a4105910 2023-11-16T16:39:42.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.966+0000 7ff1937fe700 1 -- 172.21.15.93:0/399422970 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7ff19402da50 con 0x7ff1a4105910 2023-11-16T16:39:42.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.969+0000 7ff1937fe700 1 -- 172.21.15.93:0/399422970 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7ff19404c4b0 con 0x7ff1a4105910 2023-11-16T16:39:42.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.971+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/399422970 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7ff18c038780 msgr2=0x7ff18c03ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:42.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.971+0000 7ff1a8f7b700 1 --2- 172.21.15.93:0/399422970 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7ff18c038780 0x7ff18c03ac30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff198006fd0 tx=0x7ff198006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:42.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.971+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/399422970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 msgr2=0x7ff1a419b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:42.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.971+0000 7ff1a8f7b700 1 --2- 172.21.15.93:0/399422970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 0x7ff1a419b790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff194003e60 tx=0x7ff194011aa0 comp rx=0 tx=0).stop 2023-11-16T16:39:42.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.971+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/399422970 shutdown_connections 2023-11-16T16:39:42.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.971+0000 7ff1a8f7b700 1 --2- 172.21.15.93:0/399422970 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7ff18c038780 0x7ff18c03ac30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:42.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.971+0000 7ff1a8f7b700 1 --2- 172.21.15.93:0/399422970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1a4105910 0x7ff1a419b790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:42.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.971+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/399422970 >> 172.21.15.93:0/399422970 conn(0x7ff1a4100fa0 msgr2=0x7ff1a4101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:42.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.971+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/399422970 shutdown_connections 2023-11-16T16:39:42.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:41.971+0000 7ff1a8f7b700 1 -- 172.21.15.93:0/399422970 wait complete. 2023-11-16T16:39:42.608 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:42 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:42.259+0000 7f191115b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:39:42.609 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:42 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:42.364+0000 7f191115b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:39:42.609 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:42 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:42.484+0000 7f191115b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:39:42.609 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:42 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:42.606+0000 7f191115b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:39:42.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:42 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:42.850+0000 7f191115b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:39:43.250 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:42 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/399422970' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-11-16T16:39:43.251 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:42 smithi093 ceph-mon[97863]: mgrmap e9: x(active, since 17s) 2023-11-16T16:39:43.543 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:43.249+0000 7f191115b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:39:43.543 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:43.374+0000 7f191115b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:39:43.543 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:43.541+0000 7f191115b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:39:43.707 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:39:43.707 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-11-16T16:39:43.707 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-16T16:39:43.707 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-16T16:39:43.707 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-16T16:39:43.707 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:39:43.707 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.000+0000 7fc355d7d700 1 Processor -- start 2023-11-16T16:39:43.707 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.001+0000 7fc355d7d700 1 -- start start 2023-11-16T16:39:43.708 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.001+0000 7fc355d7d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 0x7fc35007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:43.708 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.001+0000 7fc355d7d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35007a840 con 0x7fc35007be00 2023-11-16T16:39:43.708 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.001+0000 7fc34f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 0x7fc35007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:43.708 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.002+0000 7fc34f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 0x7fc35007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38356/0 (socket says 172.21.15.93:38356) 2023-11-16T16:39:43.708 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.002+0000 7fc34f7fe700 1 -- 172.21.15.93:0/1830768142 learned_addr learned my addr 172.21.15.93:0/1830768142 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:43.708 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.002+0000 7fc34f7fe700 1 -- 172.21.15.93:0/1830768142 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35007a980 con 0x7fc35007be00 2023-11-16T16:39:43.708 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.002+0000 7fc34f7fe700 1 --2- 172.21.15.93:0/1830768142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 0x7fc35007a300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc3400098a0 tx=0x7fc340009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97fa7b1f1ba619 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:43.708 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.002+0000 7fc34e7fc700 1 -- 172.21.15.93:0/1830768142 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc340004160 con 0x7fc35007be00 2023-11-16T16:39:43.708 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.002+0000 7fc34e7fc700 1 -- 172.21.15.93:0/1830768142 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3400042c0 con 0x7fc35007be00 2023-11-16T16:39:43.709 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.002+0000 7fc34e7fc700 1 -- 172.21.15.93:0/1830768142 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3400119b0 con 0x7fc35007be00 2023-11-16T16:39:43.709 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.003+0000 7fc355d7d700 1 -- 172.21.15.93:0/1830768142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 msgr2=0x7fc35007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:43.709 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.003+0000 7fc355d7d700 1 --2- 172.21.15.93:0/1830768142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 0x7fc35007a300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc3400098a0 tx=0x7fc340009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:43.709 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.003+0000 7fc355d7d700 1 -- 172.21.15.93:0/1830768142 shutdown_connections 2023-11-16T16:39:43.709 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.003+0000 7fc355d7d700 1 --2- 172.21.15.93:0/1830768142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 0x7fc35007a300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:43.709 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.003+0000 7fc355d7d700 1 -- 172.21.15.93:0/1830768142 >> 172.21.15.93:0/1830768142 conn(0x7fc350100fa0 msgr2=0x7fc3501033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:43.709 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.003+0000 7fc355d7d700 1 -- 172.21.15.93:0/1830768142 shutdown_connections 2023-11-16T16:39:43.709 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.003+0000 7fc355d7d700 1 -- 172.21.15.93:0/1830768142 wait complete. 2023-11-16T16:39:43.709 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.005+0000 7fc355d7d700 1 Processor -- start 2023-11-16T16:39:43.710 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.005+0000 7fc355d7d700 1 -- start start 2023-11-16T16:39:43.710 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.005+0000 7fc355d7d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 0x7fc35019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:43.710 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.005+0000 7fc355d7d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35019bd10 con 0x7fc35007be00 2023-11-16T16:39:43.710 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.006+0000 7fc34f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 0x7fc35019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:43.710 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.006+0000 7fc34f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 0x7fc35019b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38372/0 (socket says 172.21.15.93:38372) 2023-11-16T16:39:43.710 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.006+0000 7fc34f7fe700 1 -- 172.21.15.93:0/3196859705 learned_addr learned my addr 172.21.15.93:0/3196859705 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:43.710 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.006+0000 7fc34f7fe700 1 -- 172.21.15.93:0/3196859705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc340009550 con 0x7fc35007be00 2023-11-16T16:39:43.710 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.006+0000 7fc34f7fe700 1 --2- 172.21.15.93:0/3196859705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 0x7fc35019b7d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc340003e60 tx=0x7fc340011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:43.711 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.006+0000 7fc34cff9700 1 -- 172.21.15.93:0/3196859705 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc340003af0 con 0x7fc35007be00 2023-11-16T16:39:43.711 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.006+0000 7fc34cff9700 1 -- 172.21.15.93:0/3196859705 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc340003c50 con 0x7fc35007be00 2023-11-16T16:39:43.711 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.007+0000 7fc34cff9700 1 -- 172.21.15.93:0/3196859705 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc340024800 con 0x7fc35007be00 2023-11-16T16:39:43.711 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.007+0000 7fc355d7d700 1 -- 172.21.15.93:0/3196859705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35010be30 con 0x7fc35007be00 2023-11-16T16:39:43.711 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.007+0000 7fc355d7d700 1 -- 172.21.15.93:0/3196859705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35010bca0 con 0x7fc35007be00 2023-11-16T16:39:43.711 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.008+0000 7fc34cff9700 1 -- 172.21.15.93:0/3196859705 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fc34001a430 con 0x7fc35007be00 2023-11-16T16:39:43.711 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.008+0000 7fc355d7d700 1 -- 172.21.15.93:0/3196859705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35010bfc0 con 0x7fc35007be00 2023-11-16T16:39:43.711 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.008+0000 7fc34cff9700 1 --2- 172.21.15.93:0/3196859705 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fc338038820 0x7fc33803acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:43.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.008+0000 7fc34cff9700 1 -- 172.21.15.93:0/3196859705 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc34004ca70 con 0x7fc35007be00 2023-11-16T16:39:43.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.008+0000 7fc34effd700 1 -- 172.21.15.93:0/3196859705 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fc338038820 msgr2=0x7fc33803acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2198359920 2023-11-16T16:39:43.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.008+0000 7fc34effd700 1 --2- 172.21.15.93:0/3196859705 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fc338038820 0x7fc33803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:39:43.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.013+0000 7fc34cff9700 1 -- 172.21.15.93:0/3196859705 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc340018530 con 0x7fc35007be00 2023-11-16T16:39:43.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.209+0000 7fc34effd700 1 -- 172.21.15.93:0/3196859705 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fc338038820 msgr2=0x7fc33803acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2198359920 2023-11-16T16:39:43.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.209+0000 7fc34effd700 1 --2- 172.21.15.93:0/3196859705 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fc338038820 0x7fc33803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T16:39:43.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.269+0000 7fc355d7d700 1 -- 172.21.15.93:0/3196859705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc35019f3e0 con 0x7fc35007be00 2023-11-16T16:39:43.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.270+0000 7fc34cff9700 1 -- 172.21.15.93:0/3196859705 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fc340018350 con 0x7fc35007be00 2023-11-16T16:39:43.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.272+0000 7fc355d7d700 1 -- 172.21.15.93:0/3196859705 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fc338038820 msgr2=0x7fc33803acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:39:43.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.272+0000 7fc355d7d700 1 --2- 172.21.15.93:0/3196859705 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fc338038820 0x7fc33803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:43.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.272+0000 7fc355d7d700 1 -- 172.21.15.93:0/3196859705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 msgr2=0x7fc35019b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:43.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.272+0000 7fc355d7d700 1 --2- 172.21.15.93:0/3196859705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 0x7fc35019b7d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc340003e60 tx=0x7fc340011980 comp rx=0 tx=0).stop 2023-11-16T16:39:43.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.272+0000 7fc355d7d700 1 -- 172.21.15.93:0/3196859705 shutdown_connections 2023-11-16T16:39:43.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.272+0000 7fc355d7d700 1 --2- 172.21.15.93:0/3196859705 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7fc338038820 0x7fc33803acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:43.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.272+0000 7fc355d7d700 1 --2- 172.21.15.93:0/3196859705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc35007be00 0x7fc35019b7d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:43.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.272+0000 7fc355d7d700 1 -- 172.21.15.93:0/3196859705 >> 172.21.15.93:0/3196859705 conn(0x7fc350100fa0 msgr2=0x7fc350101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:43.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.272+0000 7fc355d7d700 1 -- 172.21.15.93:0/3196859705 shutdown_connections 2023-11-16T16:39:43.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:43.272+0000 7fc355d7d700 1 -- 172.21.15.93:0/3196859705 wait complete. 2023-11-16T16:39:43.713 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for the mgr to restart... 2023-11-16T16:39:43.714 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mgr epoch 9... 2023-11-16T16:39:43.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:43.665+0000 7f191115b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:39:44.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:43 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3196859705' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-16T16:39:44.650 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:44 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:44.242+0000 7f191115b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:39:45.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:44 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:44.974+0000 7f191115b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:39:45.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:45 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:45.181+0000 7f191115b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:39:46.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:46 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:46.114+0000 7f191115b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:39:46.401 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:46 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:46.232+0000 7f191115b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:39:46.811 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:46 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:46.452+0000 7f191115b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:39:47.091 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:46 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:46.809+0000 7f191115b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:39:47.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:47 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:47.087+0000 7f191115b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:39:47.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:47 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:47.277+0000 7f191115b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:39:47.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:47 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:47.620+0000 7f191115b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:39:48.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:48.540+0000 7f191115b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:39:48.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:48.710+0000 7f191115b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:39:49.379 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:48.956+0000 7f191115b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:39:49.379 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:49 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:49.103+0000 7f191115b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:39:49.650 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:49 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:49.377+0000 7f191115b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:39:50.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:39:50 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:39:50.429+0000 7f191115b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:39:51.471 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: Active manager daemon x restarted 2023-11-16T16:39:51.472 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: Activating manager daemon x 2023-11-16T16:39:51.472 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: osdmap e3: 0 total, 0 up, 0 in 2023-11-16T16:39:51.472 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: mgrmap e10: x(active, starting, since 0.00380085s) 2023-11-16T16:39:51.472 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:39:51.473 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-16T16:39:51.473 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-16T16:39:51.473 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-16T16:39:51.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-16T16:39:51.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: Manager daemon x is now available 2023-11-16T16:39:51.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:51.475 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:51.475 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-16T16:39:51.475 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-16T16:39:51.915 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:39:51.915 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-11-16T16:39:51.915 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-16T16:39:51.915 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:39:51.915 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.220+0000 7f892b07c700 1 Processor -- start 2023-11-16T16:39:51.916 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.220+0000 7f892b07c700 1 -- start start 2023-11-16T16:39:51.916 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.220+0000 7f892b07c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 0x7f8924105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:51.916 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.220+0000 7f892b07c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8924106260 con 0x7f8924105910 2023-11-16T16:39:51.916 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.220+0000 7f8928e18700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 0x7f8924105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:51.916 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.220+0000 7f8928e18700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 0x7f8924105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38374/0 (socket says 172.21.15.93:38374) 2023-11-16T16:39:51.916 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.221+0000 7f8928e18700 1 -- 172.21.15.93:0/4191672346 learned_addr learned my addr 172.21.15.93:0/4191672346 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:51.917 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.221+0000 7f8928e18700 1 -- 172.21.15.93:0/4191672346 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89241063a0 con 0x7f8924105910 2023-11-16T16:39:51.917 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.221+0000 7f8928e18700 1 --2- 172.21.15.93:0/4191672346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 0x7f8924105d20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89180098a0 tx=0x7f8918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25ec4cfee2056c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:51.917 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.221+0000 7f89237fe700 1 -- 172.21.15.93:0/4191672346 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8918004160 con 0x7f8924105910 2023-11-16T16:39:51.917 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.221+0000 7f89237fe700 1 -- 172.21.15.93:0/4191672346 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89180042c0 con 0x7f8924105910 2023-11-16T16:39:51.917 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.221+0000 7f89237fe700 1 -- 172.21.15.93:0/4191672346 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89180119b0 con 0x7f8924105910 2023-11-16T16:39:51.918 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.222+0000 7f892b07c700 1 -- 172.21.15.93:0/4191672346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 msgr2=0x7f8924105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:51.918 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.222+0000 7f892b07c700 1 --2- 172.21.15.93:0/4191672346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 0x7f8924105d20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89180098a0 tx=0x7f8918009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:51.918 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.222+0000 7f892b07c700 1 -- 172.21.15.93:0/4191672346 shutdown_connections 2023-11-16T16:39:51.918 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.222+0000 7f892b07c700 1 --2- 172.21.15.93:0/4191672346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 0x7f8924105d20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:51.918 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.222+0000 7f892b07c700 1 -- 172.21.15.93:0/4191672346 >> 172.21.15.93:0/4191672346 conn(0x7f8924100fa0 msgr2=0x7f89241033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:51.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.223+0000 7f892b07c700 1 -- 172.21.15.93:0/4191672346 shutdown_connections 2023-11-16T16:39:51.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.223+0000 7f892b07c700 1 -- 172.21.15.93:0/4191672346 wait complete. 2023-11-16T16:39:51.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.224+0000 7f892b07c700 1 Processor -- start 2023-11-16T16:39:51.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.224+0000 7f892b07c700 1 -- start start 2023-11-16T16:39:51.919 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.225+0000 7f892b07c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 0x7f892419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:51.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.225+0000 7f892b07c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892419bcc0 con 0x7f8924105910 2023-11-16T16:39:51.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.225+0000 7f8928e18700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 0x7f892419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:51.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.225+0000 7f8928e18700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 0x7f892419b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38382/0 (socket says 172.21.15.93:38382) 2023-11-16T16:39:51.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.225+0000 7f8928e18700 1 -- 172.21.15.93:0/3248969907 learned_addr learned my addr 172.21.15.93:0/3248969907 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:51.920 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.225+0000 7f8928e18700 1 -- 172.21.15.93:0/3248969907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8918009550 con 0x7f8924105910 2023-11-16T16:39:51.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.226+0000 7f8928e18700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 0x7f892419b780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8918011980 tx=0x7f8918011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:51.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.226+0000 7f8921ffb700 1 -- 172.21.15.93:0/3248969907 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8918003a60 con 0x7f8924105910 2023-11-16T16:39:51.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.226+0000 7f8921ffb700 1 -- 172.21.15.93:0/3248969907 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8918003bc0 con 0x7f8924105910 2023-11-16T16:39:51.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.226+0000 7f892b07c700 1 -- 172.21.15.93:0/3248969907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89241079a0 con 0x7f8924105910 2023-11-16T16:39:51.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.226+0000 7f8921ffb700 1 -- 172.21.15.93:0/3248969907 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f891801a670 con 0x7f8924105910 2023-11-16T16:39:51.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.226+0000 7f892b07c700 1 -- 172.21.15.93:0/3248969907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8924107810 con 0x7f8924105910 2023-11-16T16:39:51.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.227+0000 7f8921ffb700 1 -- 172.21.15.93:0/3248969907 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f8918024a00 con 0x7f8924105910 2023-11-16T16:39:51.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.227+0000 7f8921ffb700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 0x7f891403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:51.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.227+0000 7f8921ffb700 1 -- 172.21.15.93:0/3248969907 --> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f891403b3e0 con 0x7f8914038820 2023-11-16T16:39:51.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.228+0000 7f8921ffb700 1 -- 172.21.15.93:0/3248969907 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f891804c8b0 con 0x7f8924105910 2023-11-16T16:39:51.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.228+0000 7f8923fff700 1 -- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 msgr2=0x7f891403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2198359920 2023-11-16T16:39:51.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.228+0000 7f8923fff700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 0x7f891403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:39:51.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.429+0000 7f8923fff700 1 -- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 msgr2=0x7f891403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2198359920 2023-11-16T16:39:51.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.429+0000 7f8923fff700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 0x7f891403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T16:39:51.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.830+0000 7f8923fff700 1 -- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 msgr2=0x7f891403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2198359920 2023-11-16T16:39:51.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:44.830+0000 7f8923fff700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 0x7f891403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-16T16:39:51.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:45.631+0000 7f8923fff700 1 -- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 msgr2=0x7f891403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2198359920 2023-11-16T16:39:51.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:45.631+0000 7f8923fff700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 0x7f891403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-16T16:39:51.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:47.232+0000 7f8923fff700 1 -- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 msgr2=0x7f891403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2198359920 2023-11-16T16:39:51.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:47.232+0000 7f8923fff700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 0x7f891403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-16T16:39:51.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:50.434+0000 7f8921ffb700 1 -- 172.21.15.93:0/3248969907 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7f891802d910 con 0x7f8924105910 2023-11-16T16:39:51.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:50.434+0000 7f8921ffb700 1 -- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 msgr2=0x7f891403acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:39:51.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:50.434+0000 7f8921ffb700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 0x7f891403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:51.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.436+0000 7f8921ffb700 1 -- 172.21.15.93:0/3248969907 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f891804df90 con 0x7f8924105910 2023-11-16T16:39:51.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.436+0000 7f8921ffb700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f891403c150 0x7f891403e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:51.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.436+0000 7f8921ffb700 1 -- 172.21.15.93:0/3248969907 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f891403b3e0 con 0x7f891403c150 2023-11-16T16:39:51.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.437+0000 7f8923fff700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f891403c150 0x7f891403e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:51.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.438+0000 7f8923fff700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f891403c150 0x7f891403e530 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8910003a10 tx=0x7f89100092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:51.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.439+0000 7f8921ffb700 1 -- 172.21.15.93:0/3248969907 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f891403b3e0 con 0x7f891403c150 2023-11-16T16:39:51.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.446+0000 7f892b07c700 1 -- 172.21.15.93:0/3248969907 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f892419cac0 con 0x7f891403c150 2023-11-16T16:39:51.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.447+0000 7f8921ffb700 1 -- 172.21.15.93:0/3248969907 <== mgr.14152 v2:172.21.15.93:6800/3283196047 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f892419cac0 con 0x7f891403c150 2023-11-16T16:39:51.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.447+0000 7f892b07c700 1 -- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f891403c150 msgr2=0x7f891403e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:51.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.447+0000 7f892b07c700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f891403c150 0x7f891403e530 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8910003a10 tx=0x7f89100092b0 comp rx=0 tx=0).stop 2023-11-16T16:39:51.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.447+0000 7f892b07c700 1 -- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 msgr2=0x7f892419b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:51.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.447+0000 7f892b07c700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 0x7f892419b780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8918011980 tx=0x7f8918011a60 comp rx=0 tx=0).stop 2023-11-16T16:39:51.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.447+0000 7f892b07c700 1 -- 172.21.15.93:0/3248969907 shutdown_connections 2023-11-16T16:39:51.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.447+0000 7f892b07c700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f891403c150 0x7f891403e530 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:51.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.447+0000 7f892b07c700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924105910 0x7f892419b780 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:51.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.447+0000 7f892b07c700 1 --2- 172.21.15.93:0/3248969907 >> [v2:172.21.15.93:6800/2198359920,v1:172.21.15.93:6801/2198359920] conn(0x7f8914038820 0x7f891403acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:51.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.447+0000 7f892b07c700 1 -- 172.21.15.93:0/3248969907 >> 172.21.15.93:0/3248969907 conn(0x7f8924100fa0 msgr2=0x7f8924101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:51.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.447+0000 7f892b07c700 1 -- 172.21.15.93:0/3248969907 shutdown_connections 2023-11-16T16:39:51.929 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:51.447+0000 7f892b07c700 1 -- 172.21.15.93:0/3248969907 wait complete. 2023-11-16T16:39:51.929 INFO:teuthology.orchestra.run.smithi093.stdout:mgr epoch 9 is available 2023-11-16T16:39:51.929 INFO:teuthology.orchestra.run.smithi093.stdout:Generating a dashboard self-signed certificate... 2023-11-16T16:39:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:52 smithi093 ceph-mon[97863]: mgrmap e11: x(active, since 1.00695s) 2023-11-16T16:39:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:52 smithi093 ceph-mon[97863]: [16/Nov/2023:16:39:51] ENGINE Bus STARTING 2023-11-16T16:39:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:52 smithi093 ceph-mon[97863]: [16/Nov/2023:16:39:51] ENGINE Serving on http://172.21.15.93:8765 2023-11-16T16:39:52.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:52 smithi093 ceph-mon[97863]: [16/Nov/2023:16:39:51] ENGINE Serving on https://172.21.15.93:7150 2023-11-16T16:39:52.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:52 smithi093 ceph-mon[97863]: [16/Nov/2023:16:39:51] ENGINE Bus STARTED 2023-11-16T16:39:53.196 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-11-16T16:39:53.196 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.459+0000 7f272342b700 1 Processor -- start 2023-11-16T16:39:53.196 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.459+0000 7f272342b700 1 -- start start 2023-11-16T16:39:53.196 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.459+0000 7f272342b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 0x7f271c105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:53.196 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.459+0000 7f272342b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c106260 con 0x7f271c105910 2023-11-16T16:39:53.196 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.459+0000 7f27211c7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 0x7f271c105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:53.196 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.460+0000 7f27211c7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 0x7f271c105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58290/0 (socket says 172.21.15.93:58290) 2023-11-16T16:39:53.196 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.460+0000 7f27211c7700 1 -- 172.21.15.93:0/914342127 learned_addr learned my addr 172.21.15.93:0/914342127 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:53.197 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.460+0000 7f27211c7700 1 -- 172.21.15.93:0/914342127 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c1063a0 con 0x7f271c105910 2023-11-16T16:39:53.197 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.460+0000 7f27211c7700 1 --2- 172.21.15.93:0/914342127 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 0x7f271c105d20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27180098a0 tx=0x7f2718009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace3e9af9a7919c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:53.197 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.460+0000 7f2713fff700 1 -- 172.21.15.93:0/914342127 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2718004160 con 0x7f271c105910 2023-11-16T16:39:53.197 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.461+0000 7f2713fff700 1 -- 172.21.15.93:0/914342127 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27180042c0 con 0x7f271c105910 2023-11-16T16:39:53.197 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.461+0000 7f2713fff700 1 -- 172.21.15.93:0/914342127 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27180119b0 con 0x7f271c105910 2023-11-16T16:39:53.197 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.461+0000 7f272342b700 1 -- 172.21.15.93:0/914342127 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 msgr2=0x7f271c105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:53.197 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.461+0000 7f272342b700 1 --2- 172.21.15.93:0/914342127 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 0x7f271c105d20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27180098a0 tx=0x7f2718009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:53.197 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.461+0000 7f272342b700 1 -- 172.21.15.93:0/914342127 shutdown_connections 2023-11-16T16:39:53.197 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.461+0000 7f272342b700 1 --2- 172.21.15.93:0/914342127 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 0x7f271c105d20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:53.197 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.461+0000 7f272342b700 1 -- 172.21.15.93:0/914342127 >> 172.21.15.93:0/914342127 conn(0x7f271c100fa0 msgr2=0x7f271c1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:53.197 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.461+0000 7f272342b700 1 -- 172.21.15.93:0/914342127 shutdown_connections 2023-11-16T16:39:53.197 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.461+0000 7f272342b700 1 -- 172.21.15.93:0/914342127 wait complete. 2023-11-16T16:39:53.198 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.462+0000 7f272342b700 1 Processor -- start 2023-11-16T16:39:53.198 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.463+0000 7f272342b700 1 -- start start 2023-11-16T16:39:53.198 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.463+0000 7f272342b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 0x7f271c07bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:53.198 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.463+0000 7f272342b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c07c190 con 0x7f271c105910 2023-11-16T16:39:53.198 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.463+0000 7f27211c7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 0x7f271c07bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:53.198 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.463+0000 7f27211c7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 0x7f271c07bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58296/0 (socket says 172.21.15.93:58296) 2023-11-16T16:39:53.198 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.463+0000 7f27211c7700 1 -- 172.21.15.93:0/4154726012 learned_addr learned my addr 172.21.15.93:0/4154726012 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:53.198 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.463+0000 7f27211c7700 1 -- 172.21.15.93:0/4154726012 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2718009550 con 0x7f271c105910 2023-11-16T16:39:53.198 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.464+0000 7f27211c7700 1 --2- 172.21.15.93:0/4154726012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 0x7f271c07bc50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2718011980 tx=0x7f2718011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:53.198 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.464+0000 7f27127fc700 1 -- 172.21.15.93:0/4154726012 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2718003a60 con 0x7f271c105910 2023-11-16T16:39:53.198 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.464+0000 7f27127fc700 1 -- 172.21.15.93:0/4154726012 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2718003bc0 con 0x7f271c105910 2023-11-16T16:39:53.198 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.464+0000 7f272342b700 1 -- 172.21.15.93:0/4154726012 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c1079a0 con 0x7f271c105910 2023-11-16T16:39:53.199 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.464+0000 7f27127fc700 1 -- 172.21.15.93:0/4154726012 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f271801a670 con 0x7f271c105910 2023-11-16T16:39:53.199 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.464+0000 7f272342b700 1 -- 172.21.15.93:0/4154726012 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c107810 con 0x7f271c105910 2023-11-16T16:39:53.199 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.465+0000 7f27127fc700 1 -- 172.21.15.93:0/4154726012 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f2718024a00 con 0x7f271c105910 2023-11-16T16:39:53.199 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.465+0000 7f272342b700 1 -- 172.21.15.93:0/4154726012 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271c062120 con 0x7f271c105910 2023-11-16T16:39:53.199 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.466+0000 7f27127fc700 1 --2- 172.21.15.93:0/4154726012 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f2708038870 0x7f270803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:53.199 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.466+0000 7f27127fc700 1 -- 172.21.15.93:0/4154726012 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f271804ca10 con 0x7f271c105910 2023-11-16T16:39:53.199 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.466+0000 7f27209c6700 1 --2- 172.21.15.93:0/4154726012 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f2708038870 0x7f270803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:53.199 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.467+0000 7f27209c6700 1 --2- 172.21.15.93:0/4154726012 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f2708038870 0x7f270803ad20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f270c006fd0 tx=0x7f270c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:53.199 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.471+0000 7f27127fc700 1 -- 172.21.15.93:0/4154726012 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2718024440 con 0x7f271c105910 2023-11-16T16:39:53.199 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.686+0000 7f272342b700 1 -- 172.21.15.93:0/4154726012 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f271c07b140 con 0x7f2708038870 2023-11-16T16:39:53.200 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.740+0000 7f27127fc700 1 -- 172.21.15.93:0/4154726012 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f271c07b140 con 0x7f2708038870 2023-11-16T16:39:53.200 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.742+0000 7f272342b700 1 -- 172.21.15.93:0/4154726012 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f2708038870 msgr2=0x7f270803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:53.200 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.742+0000 7f272342b700 1 --2- 172.21.15.93:0/4154726012 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f2708038870 0x7f270803ad20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f270c006fd0 tx=0x7f270c006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:53.200 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.742+0000 7f272342b700 1 -- 172.21.15.93:0/4154726012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 msgr2=0x7f271c07bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:53.200 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.742+0000 7f272342b700 1 --2- 172.21.15.93:0/4154726012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 0x7f271c07bc50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2718011980 tx=0x7f2718011a60 comp rx=0 tx=0).stop 2023-11-16T16:39:53.200 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.742+0000 7f272342b700 1 -- 172.21.15.93:0/4154726012 shutdown_connections 2023-11-16T16:39:53.200 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.742+0000 7f272342b700 1 --2- 172.21.15.93:0/4154726012 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f2708038870 0x7f270803ad20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:53.200 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.742+0000 7f272342b700 1 --2- 172.21.15.93:0/4154726012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f271c105910 0x7f271c07bc50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:53.200 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.742+0000 7f272342b700 1 -- 172.21.15.93:0/4154726012 >> 172.21.15.93:0/4154726012 conn(0x7f271c100fa0 msgr2=0x7f271c101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:53.200 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.742+0000 7f272342b700 1 -- 172.21.15.93:0/4154726012 shutdown_connections 2023-11-16T16:39:53.200 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:52.743+0000 7f272342b700 1 -- 172.21.15.93:0/4154726012 wait complete. 2023-11-16T16:39:53.201 INFO:teuthology.orchestra.run.smithi093.stdout:Creating initial admin user... 2023-11-16T16:39:53.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:53 smithi093 ceph-mon[97863]: mgrmap e12: x(active, since 2s) 2023-11-16T16:39:53.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:53 smithi093 ceph-mon[97863]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:53.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:53 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:39:53.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:53 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:39:53.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:53 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:39:54.628 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:54 smithi093 ceph-mon[97863]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:54.629 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:54 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:39:54.686 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$pzsEnsdE/HTB5fXGJYmaP.tpSZwerW9iI8j046ipwBW2YnAvS6zWG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1700152794, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-11-16T16:39:54.686 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.749+0000 7ff68b15f700 1 Processor -- start 2023-11-16T16:39:54.686 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.749+0000 7ff68b15f700 1 -- start start 2023-11-16T16:39:54.686 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.750+0000 7ff68b15f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 0x7ff68407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:54.686 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.750+0000 7ff68b15f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68407a800 con 0x7ff68407bdc0 2023-11-16T16:39:54.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.750+0000 7ff688efb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 0x7ff68407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:54.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.750+0000 7ff688efb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 0x7ff68407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58300/0 (socket says 172.21.15.93:58300) 2023-11-16T16:39:54.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.750+0000 7ff688efb700 1 -- 172.21.15.93:0/3081504020 learned_addr learned my addr 172.21.15.93:0/3081504020 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:54.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.751+0000 7ff688efb700 1 -- 172.21.15.93:0/3081504020 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68407a940 con 0x7ff68407bdc0 2023-11-16T16:39:54.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.751+0000 7ff688efb700 1 --2- 172.21.15.93:0/3081504020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 0x7ff68407a2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6780098a0 tx=0x7ff678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3774de6655977950 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:54.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.751+0000 7ff6837fe700 1 -- 172.21.15.93:0/3081504020 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff678004160 con 0x7ff68407bdc0 2023-11-16T16:39:54.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.751+0000 7ff6837fe700 1 -- 172.21.15.93:0/3081504020 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6780042c0 con 0x7ff68407bdc0 2023-11-16T16:39:54.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.751+0000 7ff6837fe700 1 -- 172.21.15.93:0/3081504020 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6780119b0 con 0x7ff68407bdc0 2023-11-16T16:39:54.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.752+0000 7ff68b15f700 1 -- 172.21.15.93:0/3081504020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 msgr2=0x7ff68407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:54.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.752+0000 7ff68b15f700 1 --2- 172.21.15.93:0/3081504020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 0x7ff68407a2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6780098a0 tx=0x7ff678009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:54.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.752+0000 7ff68b15f700 1 -- 172.21.15.93:0/3081504020 shutdown_connections 2023-11-16T16:39:54.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.752+0000 7ff68b15f700 1 --2- 172.21.15.93:0/3081504020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 0x7ff68407a2c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:54.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.752+0000 7ff68b15f700 1 -- 172.21.15.93:0/3081504020 >> 172.21.15.93:0/3081504020 conn(0x7ff684100fa0 msgr2=0x7ff6841033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:54.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.752+0000 7ff68b15f700 1 -- 172.21.15.93:0/3081504020 shutdown_connections 2023-11-16T16:39:54.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.752+0000 7ff68b15f700 1 -- 172.21.15.93:0/3081504020 wait complete. 2023-11-16T16:39:54.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.753+0000 7ff68b15f700 1 Processor -- start 2023-11-16T16:39:54.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.753+0000 7ff68b15f700 1 -- start start 2023-11-16T16:39:54.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.753+0000 7ff68b15f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 0x7ff68419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:54.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.753+0000 7ff68b15f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68419bd00 con 0x7ff68407bdc0 2023-11-16T16:39:54.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.754+0000 7ff688efb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 0x7ff68419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:54.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.754+0000 7ff688efb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 0x7ff68419b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58304/0 (socket says 172.21.15.93:58304) 2023-11-16T16:39:54.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.754+0000 7ff688efb700 1 -- 172.21.15.93:0/1311198009 learned_addr learned my addr 172.21.15.93:0/1311198009 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:54.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.754+0000 7ff688efb700 1 -- 172.21.15.93:0/1311198009 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff678009550 con 0x7ff68407bdc0 2023-11-16T16:39:54.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.754+0000 7ff688efb700 1 --2- 172.21.15.93:0/1311198009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 0x7ff68419b7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff678000c00 tx=0x7ff678011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:54.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.754+0000 7ff681ffb700 1 -- 172.21.15.93:0/1311198009 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6780039f0 con 0x7ff68407bdc0 2023-11-16T16:39:54.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.754+0000 7ff681ffb700 1 -- 172.21.15.93:0/1311198009 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff678003b50 con 0x7ff68407bdc0 2023-11-16T16:39:54.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.754+0000 7ff681ffb700 1 -- 172.21.15.93:0/1311198009 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff67801a780 con 0x7ff68407bdc0 2023-11-16T16:39:54.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.755+0000 7ff68b15f700 1 -- 172.21.15.93:0/1311198009 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68410be10 con 0x7ff68407bdc0 2023-11-16T16:39:54.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.755+0000 7ff68b15f700 1 -- 172.21.15.93:0/1311198009 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68410bc80 con 0x7ff68407bdc0 2023-11-16T16:39:54.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.756+0000 7ff681ffb700 1 -- 172.21.15.93:0/1311198009 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7ff678024440 con 0x7ff68407bdc0 2023-11-16T16:39:54.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.756+0000 7ff68b15f700 1 -- 172.21.15.93:0/1311198009 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68410bfa0 con 0x7ff68407bdc0 2023-11-16T16:39:54.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.756+0000 7ff681ffb700 1 --2- 172.21.15.93:0/1311198009 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff674038870 0x7ff67403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:54.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.756+0000 7ff681ffb700 1 -- 172.21.15.93:0/1311198009 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff67804c970 con 0x7ff68407bdc0 2023-11-16T16:39:54.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.756+0000 7ff683fff700 1 --2- 172.21.15.93:0/1311198009 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff674038870 0x7ff67403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:54.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.757+0000 7ff683fff700 1 --2- 172.21.15.93:0/1311198009 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff674038870 0x7ff67403ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff670006fd0 tx=0x7ff670006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:54.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:53.761+0000 7ff681ffb700 1 -- 172.21.15.93:0/1311198009 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff678018b80 con 0x7ff68407bdc0 2023-11-16T16:39:54.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:54.013+0000 7ff68b15f700 1 -- 172.21.15.93:0/1311198009 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff684061f40 con 0x7ff674038870 2023-11-16T16:39:54.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:54.243+0000 7ff681ffb700 1 -- 172.21.15.93:0/1311198009 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7ff684061f40 con 0x7ff674038870 2023-11-16T16:39:54.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:54.245+0000 7ff68b15f700 1 -- 172.21.15.93:0/1311198009 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff674038870 msgr2=0x7ff67403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:54.691 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:54.245+0000 7ff68b15f700 1 --2- 172.21.15.93:0/1311198009 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff674038870 0x7ff67403ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff670006fd0 tx=0x7ff670006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:54.691 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:54.245+0000 7ff68b15f700 1 -- 172.21.15.93:0/1311198009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 msgr2=0x7ff68419b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:54.691 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:54.245+0000 7ff68b15f700 1 --2- 172.21.15.93:0/1311198009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 0x7ff68419b7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff678000c00 tx=0x7ff678011a30 comp rx=0 tx=0).stop 2023-11-16T16:39:54.691 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:54.245+0000 7ff68b15f700 1 -- 172.21.15.93:0/1311198009 shutdown_connections 2023-11-16T16:39:54.691 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:54.245+0000 7ff68b15f700 1 --2- 172.21.15.93:0/1311198009 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff674038870 0x7ff67403ad20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:54.691 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:54.245+0000 7ff68b15f700 1 --2- 172.21.15.93:0/1311198009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68407bdc0 0x7ff68419b7c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:54.691 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:54.245+0000 7ff68b15f700 1 -- 172.21.15.93:0/1311198009 >> 172.21.15.93:0/1311198009 conn(0x7ff684100fa0 msgr2=0x7ff684101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:54.691 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:54.245+0000 7ff68b15f700 1 -- 172.21.15.93:0/1311198009 shutdown_connections 2023-11-16T16:39:54.691 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:54.245+0000 7ff68b15f700 1 -- 172.21.15.93:0/1311198009 wait complete. 2023-11-16T16:39:54.691 INFO:teuthology.orchestra.run.smithi093.stdout:Fetching dashboard port number... 2023-11-16T16:39:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:39:55.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:55 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1466450332' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-11-16T16:39:55.972 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 8443 2023-11-16T16:39:55.972 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.267+0000 7fc829fc7700 1 Processor -- start 2023-11-16T16:39:55.972 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.268+0000 7fc829fc7700 1 -- start start 2023-11-16T16:39:55.972 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.268+0000 7fc829fc7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 0x7fc824105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:55.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.268+0000 7fc829fc7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824106260 con 0x7fc824105910 2023-11-16T16:39:55.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.268+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 0x7fc824105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:55.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.268+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 0x7fc824105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58316/0 (socket says 172.21.15.93:58316) 2023-11-16T16:39:55.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.268+0000 7fc8237fe700 1 -- 172.21.15.93:0/2392928193 learned_addr learned my addr 172.21.15.93:0/2392928193 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:55.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.269+0000 7fc8237fe700 1 -- 172.21.15.93:0/2392928193 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8241063a0 con 0x7fc824105910 2023-11-16T16:39:55.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.269+0000 7fc8237fe700 1 --2- 172.21.15.93:0/2392928193 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 0x7fc824105d20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8140098a0 tx=0x7fc814009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96514af50d19b2f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:55.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.269+0000 7fc8227fc700 1 -- 172.21.15.93:0/2392928193 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc814004160 con 0x7fc824105910 2023-11-16T16:39:55.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.269+0000 7fc8227fc700 1 -- 172.21.15.93:0/2392928193 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8140042c0 con 0x7fc824105910 2023-11-16T16:39:55.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.269+0000 7fc8227fc700 1 -- 172.21.15.93:0/2392928193 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8140119b0 con 0x7fc824105910 2023-11-16T16:39:55.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.270+0000 7fc829fc7700 1 -- 172.21.15.93:0/2392928193 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 msgr2=0x7fc824105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:55.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.270+0000 7fc829fc7700 1 --2- 172.21.15.93:0/2392928193 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 0x7fc824105d20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8140098a0 tx=0x7fc814009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:55.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.270+0000 7fc829fc7700 1 -- 172.21.15.93:0/2392928193 shutdown_connections 2023-11-16T16:39:55.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.270+0000 7fc829fc7700 1 --2- 172.21.15.93:0/2392928193 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 0x7fc824105d20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:55.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.270+0000 7fc829fc7700 1 -- 172.21.15.93:0/2392928193 >> 172.21.15.93:0/2392928193 conn(0x7fc824100fa0 msgr2=0x7fc8241033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:55.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.270+0000 7fc829fc7700 1 -- 172.21.15.93:0/2392928193 shutdown_connections 2023-11-16T16:39:55.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.270+0000 7fc829fc7700 1 -- 172.21.15.93:0/2392928193 wait complete. 2023-11-16T16:39:55.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.271+0000 7fc829fc7700 1 Processor -- start 2023-11-16T16:39:55.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.271+0000 7fc829fc7700 1 -- start start 2023-11-16T16:39:55.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.271+0000 7fc829fc7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 0x7fc82419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:55.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.271+0000 7fc829fc7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82419bc90 con 0x7fc824105910 2023-11-16T16:39:55.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.272+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 0x7fc82419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:55.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.272+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 0x7fc82419b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58324/0 (socket says 172.21.15.93:58324) 2023-11-16T16:39:55.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.272+0000 7fc8237fe700 1 -- 172.21.15.93:0/1466450332 learned_addr learned my addr 172.21.15.93:0/1466450332 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:55.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.272+0000 7fc8237fe700 1 -- 172.21.15.93:0/1466450332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc814009550 con 0x7fc824105910 2023-11-16T16:39:55.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.272+0000 7fc8237fe700 1 --2- 172.21.15.93:0/1466450332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 0x7fc82419b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc814011980 tx=0x7fc814011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:55.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.272+0000 7fc820ff9700 1 -- 172.21.15.93:0/1466450332 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc814003a60 con 0x7fc824105910 2023-11-16T16:39:55.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.272+0000 7fc820ff9700 1 -- 172.21.15.93:0/1466450332 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc814003bc0 con 0x7fc824105910 2023-11-16T16:39:55.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.272+0000 7fc829fc7700 1 -- 172.21.15.93:0/1466450332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8241079a0 con 0x7fc824105910 2023-11-16T16:39:55.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.272+0000 7fc820ff9700 1 -- 172.21.15.93:0/1466450332 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc81401a670 con 0x7fc824105910 2023-11-16T16:39:55.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.273+0000 7fc829fc7700 1 -- 172.21.15.93:0/1466450332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc824107810 con 0x7fc824105910 2023-11-16T16:39:55.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.273+0000 7fc820ff9700 1 -- 172.21.15.93:0/1466450332 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fc814024a00 con 0x7fc824105910 2023-11-16T16:39:55.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.273+0000 7fc829fc7700 1 -- 172.21.15.93:0/1466450332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc824109de0 con 0x7fc824105910 2023-11-16T16:39:55.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.274+0000 7fc820ff9700 1 --2- 172.21.15.93:0/1466450332 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc80c038820 0x7fc80c03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:55.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.274+0000 7fc820ff9700 1 -- 172.21.15.93:0/1466450332 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc81404c9c0 con 0x7fc824105910 2023-11-16T16:39:55.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.274+0000 7fc822ffd700 1 --2- 172.21.15.93:0/1466450332 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc80c038820 0x7fc80c03acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:55.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.275+0000 7fc822ffd700 1 --2- 172.21.15.93:0/1466450332 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc80c038820 0x7fc80c03acd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc818006fd0 tx=0x7fc818006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:55.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.279+0000 7fc820ff9700 1 -- 172.21.15.93:0/1466450332 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc814018b80 con 0x7fc824105910 2023-11-16T16:39:55.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.512+0000 7fc829fc7700 1 -- 172.21.15.93:0/1466450332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fc82419f360 con 0x7fc824105910 2023-11-16T16:39:55.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.512+0000 7fc820ff9700 1 -- 172.21.15.93:0/1466450332 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fc814018350 con 0x7fc824105910 2023-11-16T16:39:55.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.515+0000 7fc829fc7700 1 -- 172.21.15.93:0/1466450332 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc80c038820 msgr2=0x7fc80c03acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:55.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.515+0000 7fc829fc7700 1 --2- 172.21.15.93:0/1466450332 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc80c038820 0x7fc80c03acd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc818006fd0 tx=0x7fc818006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:55.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.515+0000 7fc829fc7700 1 -- 172.21.15.93:0/1466450332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 msgr2=0x7fc82419b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:55.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.515+0000 7fc829fc7700 1 --2- 172.21.15.93:0/1466450332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 0x7fc82419b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc814011980 tx=0x7fc814011a60 comp rx=0 tx=0).stop 2023-11-16T16:39:55.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.515+0000 7fc829fc7700 1 -- 172.21.15.93:0/1466450332 shutdown_connections 2023-11-16T16:39:55.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.515+0000 7fc829fc7700 1 --2- 172.21.15.93:0/1466450332 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc80c038820 0x7fc80c03acd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:55.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.515+0000 7fc829fc7700 1 --2- 172.21.15.93:0/1466450332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc824105910 0x7fc82419b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:55.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.515+0000 7fc829fc7700 1 -- 172.21.15.93:0/1466450332 >> 172.21.15.93:0/1466450332 conn(0x7fc824100fa0 msgr2=0x7fc824101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:55.979 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.515+0000 7fc829fc7700 1 -- 172.21.15.93:0/1466450332 shutdown_connections 2023-11-16T16:39:55.979 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:55.515+0000 7fc829fc7700 1 -- 172.21.15.93:0/1466450332 wait complete. 2023-11-16T16:39:55.987 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-16T16:39:55.987 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout disabled 2023-11-16T16:39:56.004 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-16T16:39:56.004 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-11-16T16:39:56.004 INFO:teuthology.orchestra.run.smithi093.stdout:firewalld.service is not enabled 2023-11-16T16:39:56.004 INFO:teuthology.orchestra.run.smithi093.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-11-16T16:39:56.006 INFO:teuthology.orchestra.run.smithi093.stdout:Ceph Dashboard is now available at: 2023-11-16T16:39:56.006 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:39:56.006 INFO:teuthology.orchestra.run.smithi093.stdout: URL: https://smithi093.front.sepia.ceph.com:8443/ 2023-11-16T16:39:56.007 INFO:teuthology.orchestra.run.smithi093.stdout: User: admin 2023-11-16T16:39:56.007 INFO:teuthology.orchestra.run.smithi093.stdout: Password: xhhoewapwq 2023-11-16T16:39:56.007 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:39:56.007 INFO:teuthology.orchestra.run.smithi093.stdout:Saving cluster configuration to /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config directory 2023-11-16T16:39:56.008 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling autotune for osd_memory_target 2023-11-16T16:39:57.545 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.822+0000 7fa57dc0c700 1 Processor -- start 2023-11-16T16:39:57.545 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.823+0000 7fa57dc0c700 1 -- start start 2023-11-16T16:39:57.545 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.823+0000 7fa57dc0c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 0x7fa578105cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:57.546 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.823+0000 7fa57dc0c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa578106200 con 0x7fa5781058b0 2023-11-16T16:39:57.546 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.823+0000 7fa5777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 0x7fa578105cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:57.546 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.823+0000 7fa5777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 0x7fa578105cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58334/0 (socket says 172.21.15.93:58334) 2023-11-16T16:39:57.546 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.823+0000 7fa5777fe700 1 -- 172.21.15.93:0/1258863513 learned_addr learned my addr 172.21.15.93:0/1258863513 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:57.546 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.824+0000 7fa5777fe700 1 -- 172.21.15.93:0/1258863513 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa578106340 con 0x7fa5781058b0 2023-11-16T16:39:57.546 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.824+0000 7fa5777fe700 1 --2- 172.21.15.93:0/1258863513 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 0x7fa578105cc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa5680098a0 tx=0x7fa568009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c2a8a1e890ae42 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:57.546 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.824+0000 7fa5767fc700 1 -- 172.21.15.93:0/1258863513 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa568004160 con 0x7fa5781058b0 2023-11-16T16:39:57.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.824+0000 7fa5767fc700 1 -- 172.21.15.93:0/1258863513 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5680042c0 con 0x7fa5781058b0 2023-11-16T16:39:57.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.824+0000 7fa5767fc700 1 -- 172.21.15.93:0/1258863513 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5680119b0 con 0x7fa5781058b0 2023-11-16T16:39:57.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.825+0000 7fa57dc0c700 1 -- 172.21.15.93:0/1258863513 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 msgr2=0x7fa578105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:57.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.825+0000 7fa57dc0c700 1 --2- 172.21.15.93:0/1258863513 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 0x7fa578105cc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa5680098a0 tx=0x7fa568009bb0 comp rx=0 tx=0).stop 2023-11-16T16:39:57.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.825+0000 7fa57dc0c700 1 -- 172.21.15.93:0/1258863513 shutdown_connections 2023-11-16T16:39:57.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.825+0000 7fa57dc0c700 1 --2- 172.21.15.93:0/1258863513 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 0x7fa578105cc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:57.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.825+0000 7fa57dc0c700 1 -- 172.21.15.93:0/1258863513 >> 172.21.15.93:0/1258863513 conn(0x7fa578100f00 msgr2=0x7fa578103330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:57.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.825+0000 7fa57dc0c700 1 -- 172.21.15.93:0/1258863513 shutdown_connections 2023-11-16T16:39:57.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.825+0000 7fa57dc0c700 1 -- 172.21.15.93:0/1258863513 wait complete. 2023-11-16T16:39:57.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.826+0000 7fa57dc0c700 1 Processor -- start 2023-11-16T16:39:57.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.827+0000 7fa57dc0c700 1 -- start start 2023-11-16T16:39:57.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.827+0000 7fa57dc0c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 0x7fa57807bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:57.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.827+0000 7fa57dc0c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57807c190 con 0x7fa5781058b0 2023-11-16T16:39:57.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.827+0000 7fa5777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 0x7fa57807bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:57.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.827+0000 7fa5777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 0x7fa57807bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58346/0 (socket says 172.21.15.93:58346) 2023-11-16T16:39:57.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.827+0000 7fa5777fe700 1 -- 172.21.15.93:0/783957397 learned_addr learned my addr 172.21.15.93:0/783957397 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:39:57.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.828+0000 7fa5777fe700 1 -- 172.21.15.93:0/783957397 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa568009550 con 0x7fa5781058b0 2023-11-16T16:39:57.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.828+0000 7fa5777fe700 1 --2- 172.21.15.93:0/783957397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 0x7fa57807bc50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa568006af0 tx=0x7fa568011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:57.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.828+0000 7fa574ff9700 1 -- 172.21.15.93:0/783957397 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa568003a40 con 0x7fa5781058b0 2023-11-16T16:39:57.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.828+0000 7fa574ff9700 1 -- 172.21.15.93:0/783957397 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa568003ba0 con 0x7fa5781058b0 2023-11-16T16:39:57.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.828+0000 7fa57dc0c700 1 -- 172.21.15.93:0/783957397 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa578107940 con 0x7fa5781058b0 2023-11-16T16:39:57.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.828+0000 7fa574ff9700 1 -- 172.21.15.93:0/783957397 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5680196c0 con 0x7fa5781058b0 2023-11-16T16:39:57.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.828+0000 7fa57dc0c700 1 -- 172.21.15.93:0/783957397 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5781077b0 con 0x7fa5781058b0 2023-11-16T16:39:57.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.829+0000 7fa57dc0c700 1 -- 172.21.15.93:0/783957397 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa578062120 con 0x7fa5781058b0 2023-11-16T16:39:57.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.829+0000 7fa574ff9700 1 -- 172.21.15.93:0/783957397 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fa56801aa80 con 0x7fa5781058b0 2023-11-16T16:39:57.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.830+0000 7fa574ff9700 1 --2- 172.21.15.93:0/783957397 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa5600388c0 0x7fa56003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:57.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.830+0000 7fa574ff9700 1 -- 172.21.15.93:0/783957397 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa56804cc70 con 0x7fa5781058b0 2023-11-16T16:39:57.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.830+0000 7fa576ffd700 1 --2- 172.21.15.93:0/783957397 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa5600388c0 0x7fa56003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:57.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.831+0000 7fa576ffd700 1 --2- 172.21.15.93:0/783957397 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa5600388c0 0x7fa56003ad70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa56c006fd0 tx=0x7fa56c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:57.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:56.834+0000 7fa574ff9700 1 -- 172.21.15.93:0/783957397 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa56802d430 con 0x7fa5781058b0 2023-11-16T16:39:57.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:57.034+0000 7fa57dc0c700 1 -- 172.21.15.93:0/783957397 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fa57807aa00 con 0x7fa5781058b0 2023-11-16T16:39:57.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:57.034+0000 7fa574ff9700 1 -- 172.21.15.93:0/783957397 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fa56800b3e0 con 0x7fa5781058b0 2023-11-16T16:39:57.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:57.037+0000 7fa57dc0c700 1 -- 172.21.15.93:0/783957397 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa5600388c0 msgr2=0x7fa56003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:57.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:57.037+0000 7fa57dc0c700 1 --2- 172.21.15.93:0/783957397 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa5600388c0 0x7fa56003ad70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa56c006fd0 tx=0x7fa56c006e40 comp rx=0 tx=0).stop 2023-11-16T16:39:57.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:57.037+0000 7fa57dc0c700 1 -- 172.21.15.93:0/783957397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 msgr2=0x7fa57807bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:57.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:57.037+0000 7fa57dc0c700 1 --2- 172.21.15.93:0/783957397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 0x7fa57807bc50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa568006af0 tx=0x7fa568011a10 comp rx=0 tx=0).stop 2023-11-16T16:39:57.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:57.037+0000 7fa57dc0c700 1 -- 172.21.15.93:0/783957397 shutdown_connections 2023-11-16T16:39:57.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:57.037+0000 7fa57dc0c700 1 --2- 172.21.15.93:0/783957397 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa5600388c0 0x7fa56003ad70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:57.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:57.037+0000 7fa57dc0c700 1 --2- 172.21.15.93:0/783957397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5781058b0 0x7fa57807bc50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:57.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:57.037+0000 7fa57dc0c700 1 -- 172.21.15.93:0/783957397 >> 172.21.15.93:0/783957397 conn(0x7fa578100f00 msgr2=0x7fa578101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:57.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:57.037+0000 7fa57dc0c700 1 -- 172.21.15.93:0/783957397 shutdown_connections 2023-11-16T16:39:57.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:57.037+0000 7fa57dc0c700 1 -- 172.21.15.93:0/783957397 wait complete. 2023-11-16T16:39:57.844 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:57 smithi093 ceph-mon[97863]: mgrmap e13: x(active, since 6s) 2023-11-16T16:39:58.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.074+0000 7f462d2e2700 1 Processor -- start 2023-11-16T16:40:02.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.075+0000 7f462d2e2700 1 -- start start 2023-11-16T16:40:02.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.075+0000 7f462d2e2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 0x7f4628105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:02.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.075+0000 7f462d2e2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628106260 con 0x7f4628105910 2023-11-16T16:40:02.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.075+0000 7f4626ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 0x7f4628105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:02.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.075+0000 7f4626ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 0x7f4628105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33866/0 (socket says 172.21.15.93:33866) 2023-11-16T16:40:02.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.075+0000 7f4626ffd700 1 -- 172.21.15.93:0/1944528786 learned_addr learned my addr 172.21.15.93:0/1944528786 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:40:02.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.076+0000 7f4626ffd700 1 -- 172.21.15.93:0/1944528786 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46281063a0 con 0x7f4628105910 2023-11-16T16:40:02.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.076+0000 7f4626ffd700 1 --2- 172.21.15.93:0/1944528786 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 0x7f4628105d20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f46180098a0 tx=0x7f4618009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec08aa8ca9ce0a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:02.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.076+0000 7f4625ffb700 1 -- 172.21.15.93:0/1944528786 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4618004160 con 0x7f4628105910 2023-11-16T16:40:02.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.076+0000 7f4625ffb700 1 -- 172.21.15.93:0/1944528786 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46180042c0 con 0x7f4628105910 2023-11-16T16:40:02.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.076+0000 7f4625ffb700 1 -- 172.21.15.93:0/1944528786 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46180119b0 con 0x7f4628105910 2023-11-16T16:40:02.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.077+0000 7f462d2e2700 1 -- 172.21.15.93:0/1944528786 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 msgr2=0x7f4628105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:02.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.077+0000 7f462d2e2700 1 --2- 172.21.15.93:0/1944528786 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 0x7f4628105d20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f46180098a0 tx=0x7f4618009bb0 comp rx=0 tx=0).stop 2023-11-16T16:40:02.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.077+0000 7f462d2e2700 1 -- 172.21.15.93:0/1944528786 shutdown_connections 2023-11-16T16:40:02.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.077+0000 7f462d2e2700 1 --2- 172.21.15.93:0/1944528786 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 0x7f4628105d20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.077+0000 7f462d2e2700 1 -- 172.21.15.93:0/1944528786 >> 172.21.15.93:0/1944528786 conn(0x7f4628100fa0 msgr2=0x7f46281033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.077+0000 7f462d2e2700 1 -- 172.21.15.93:0/1944528786 shutdown_connections 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.077+0000 7f462d2e2700 1 -- 172.21.15.93:0/1944528786 wait complete. 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.078+0000 7f462d2e2700 1 Processor -- start 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.078+0000 7f462d2e2700 1 -- start start 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.078+0000 7f462d2e2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 0x7f4628199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.078+0000 7f462d2e2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628199a90 con 0x7f4628105910 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.079+0000 7f4626ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 0x7f4628199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.079+0000 7f4626ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 0x7f4628199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33880/0 (socket says 172.21.15.93:33880) 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.079+0000 7f4626ffd700 1 -- 172.21.15.93:0/1232246652 learned_addr learned my addr 172.21.15.93:0/1232246652 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.079+0000 7f4626ffd700 1 -- 172.21.15.93:0/1232246652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4618009550 con 0x7f4628105910 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.079+0000 7f4626ffd700 1 --2- 172.21.15.93:0/1232246652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 0x7f4628199550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4618011980 tx=0x7f4618011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.079+0000 7f460ffff700 1 -- 172.21.15.93:0/1232246652 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4618003a60 con 0x7f4628105910 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.079+0000 7f460ffff700 1 -- 172.21.15.93:0/1232246652 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4618003bc0 con 0x7f4628105910 2023-11-16T16:40:02.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.079+0000 7f462d2e2700 1 -- 172.21.15.93:0/1232246652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46281079a0 con 0x7f4628105910 2023-11-16T16:40:02.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.079+0000 7f460ffff700 1 -- 172.21.15.93:0/1232246652 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f461801a670 con 0x7f4628105910 2023-11-16T16:40:02.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.080+0000 7f462d2e2700 1 -- 172.21.15.93:0/1232246652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4628107810 con 0x7f4628105910 2023-11-16T16:40:02.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.080+0000 7f460ffff700 1 -- 172.21.15.93:0/1232246652 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f4618024a00 con 0x7f4628105910 2023-11-16T16:40:02.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.081+0000 7f462d2e2700 1 -- 172.21.15.93:0/1232246652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4628109de0 con 0x7f4628105910 2023-11-16T16:40:02.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.081+0000 7f460ffff700 1 --2- 172.21.15.93:0/1232246652 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4610038910 0x7f461003adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:02.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.081+0000 7f460ffff700 1 -- 172.21.15.93:0/1232246652 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f461804ca70 con 0x7f4628105910 2023-11-16T16:40:02.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.081+0000 7f46267fc700 1 --2- 172.21.15.93:0/1232246652 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4610038910 0x7f461003adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:02.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.082+0000 7f46267fc700 1 --2- 172.21.15.93:0/1232246652 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4610038910 0x7f461003adc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f461c006fd0 tx=0x7f461c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:02.978 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.086+0000 7f460ffff700 1 -- 172.21.15.93:0/1232246652 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4618024440 con 0x7f4628105910 2023-11-16T16:40:03.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.383+0000 7f462d2e2700 1 -- 172.21.15.93:0/1232246652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f462819d160 con 0x7f4628105910 2023-11-16T16:40:03.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.386+0000 7f460ffff700 1 -- 172.21.15.93:0/1232246652 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f4618018350 con 0x7f4628105910 2023-11-16T16:40:03.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.388+0000 7f462d2e2700 1 -- 172.21.15.93:0/1232246652 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4610038910 msgr2=0x7f461003adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.388+0000 7f462d2e2700 1 --2- 172.21.15.93:0/1232246652 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4610038910 0x7f461003adc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f461c006fd0 tx=0x7f461c006e40 comp rx=0 tx=0).stop 2023-11-16T16:40:03.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.389+0000 7f462d2e2700 1 -- 172.21.15.93:0/1232246652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 msgr2=0x7f4628199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.389+0000 7f462d2e2700 1 --2- 172.21.15.93:0/1232246652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 0x7f4628199550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4618011980 tx=0x7f4618011a60 comp rx=0 tx=0).stop 2023-11-16T16:40:03.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.389+0000 7f462d2e2700 1 -- 172.21.15.93:0/1232246652 shutdown_connections 2023-11-16T16:40:03.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.389+0000 7f462d2e2700 1 --2- 172.21.15.93:0/1232246652 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4610038910 0x7f461003adc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.389+0000 7f462d2e2700 1 --2- 172.21.15.93:0/1232246652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4628105910 0x7f4628199550 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.389+0000 7f462d2e2700 1 -- 172.21.15.93:0/1232246652 >> 172.21.15.93:0/1232246652 conn(0x7f4628100fa0 msgr2=0x7f4628101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:03.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.389+0000 7f462d2e2700 1 -- 172.21.15.93:0/1232246652 shutdown_connections 2023-11-16T16:40:03.001 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-11-16T16:39:58.389+0000 7f462d2e2700 1 -- 172.21.15.93:0/1232246652 wait complete. 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout:Or, if you are only running a single cluster on this host: 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout:Please consider enabling telemetry to help improve Ceph: 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout: ceph telemetry on 2023-11-16T16:40:03.002 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:40:03.003 INFO:teuthology.orchestra.run.smithi093.stdout:For more information see: 2023-11-16T16:40:03.003 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:40:03.003 INFO:teuthology.orchestra.run.smithi093.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-11-16T16:40:03.003 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:40:03.003 INFO:teuthology.orchestra.run.smithi093.stdout:Bootstrap complete. 2023-11-16T16:40:03.003 INFO:tasks.cephadm:Fetching config... 2023-11-16T16:40:03.003 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:40:03.004 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-11-16T16:40:03.004 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:39:59 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1232246652' entity='client.admin' 2023-11-16T16:40:03.004 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:03.004 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:03.004 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi093", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:40:03.004 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:03.004 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:03.004 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:03.051 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-11-16T16:40:03.052 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:40:03.052 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-11-16T16:40:03.135 INFO:tasks.cephadm:Fetching mon keyring... 2023-11-16T16:40:03.136 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:40:03.136 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-11-16T16:40:03.247 INFO:tasks.cephadm:Fetching pub ssh key... 2023-11-16T16:40:03.247 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:40:03.248 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-11-16T16:40:03.330 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-11-16T16:40:03.330 DEBUG:teuthology.orchestra.run.smithi093:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZspfJ+QMr9tkiR3JeSLbtDVZX5wMU7OwNvd099622r0TeEJpEBUjFdIojO6XfKPpswRL+9JMUiua0DHDiUMWCt6y+70bn6FOygTNs/Xfjbn/LI6DOPbx0u3JkUO0MbX6xJa+xorvK5yEnmC154x0Rsw3B2WBMP1+bf4DkOXc0Jz1fX/RvfgbBff4Ejt/WKDVBgAhcRV1it6EdD8/vQdsd+hjBY+PJV5+epVuZx/OgIM4tv6vy8scg5I94KIRjCS/sBIUNpaD+ionvNdWlHaC6JmPh9dmDfBuoZC/uiD1kIU+b0jb/H409zAL1FgDeT5c+SeLT96whTnSB49VAgf9OhDqIoIGk2hyxlBDFWJZF+OA/fM0mbhc6ywUEyLTe3O6EsZdhVIDYbMgHdL3Ipdr/VA9UaNiBRfGlxEZwpK9YwY3OzbmOIh0mfLAodC1vnjaahr9BBXKFXcsKu4/7t+g6Nte3ycuTUxTPMSsdMw1vKwfomfIcjljeWAqMvmCKT/E= ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-16T16:40:03.464 INFO:teuthology.orchestra.run.smithi093.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZspfJ+QMr9tkiR3JeSLbtDVZX5wMU7OwNvd099622r0TeEJpEBUjFdIojO6XfKPpswRL+9JMUiua0DHDiUMWCt6y+70bn6FOygTNs/Xfjbn/LI6DOPbx0u3JkUO0MbX6xJa+xorvK5yEnmC154x0Rsw3B2WBMP1+bf4DkOXc0Jz1fX/RvfgbBff4Ejt/WKDVBgAhcRV1it6EdD8/vQdsd+hjBY+PJV5+epVuZx/OgIM4tv6vy8scg5I94KIRjCS/sBIUNpaD+ionvNdWlHaC6JmPh9dmDfBuoZC/uiD1kIU+b0jb/H409zAL1FgDeT5c+SeLT96whTnSB49VAgf9OhDqIoIGk2hyxlBDFWJZF+OA/fM0mbhc6ywUEyLTe3O6EsZdhVIDYbMgHdL3Ipdr/VA9UaNiBRfGlxEZwpK9YwY3OzbmOIh0mfLAodC1vnjaahr9BBXKFXcsKu4/7t+g6Nte3ycuTUxTPMSsdMw1vKwfomfIcjljeWAqMvmCKT/E= ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:40:03.493 DEBUG:teuthology.orchestra.run.smithi125:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZspfJ+QMr9tkiR3JeSLbtDVZX5wMU7OwNvd099622r0TeEJpEBUjFdIojO6XfKPpswRL+9JMUiua0DHDiUMWCt6y+70bn6FOygTNs/Xfjbn/LI6DOPbx0u3JkUO0MbX6xJa+xorvK5yEnmC154x0Rsw3B2WBMP1+bf4DkOXc0Jz1fX/RvfgbBff4Ejt/WKDVBgAhcRV1it6EdD8/vQdsd+hjBY+PJV5+epVuZx/OgIM4tv6vy8scg5I94KIRjCS/sBIUNpaD+ionvNdWlHaC6JmPh9dmDfBuoZC/uiD1kIU+b0jb/H409zAL1FgDeT5c+SeLT96whTnSB49VAgf9OhDqIoIGk2hyxlBDFWJZF+OA/fM0mbhc6ywUEyLTe3O6EsZdhVIDYbMgHdL3Ipdr/VA9UaNiBRfGlxEZwpK9YwY3OzbmOIh0mfLAodC1vnjaahr9BBXKFXcsKu4/7t+g6Nte3ycuTUxTPMSsdMw1vKwfomfIcjljeWAqMvmCKT/E= ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-16T16:40:03.540 INFO:teuthology.orchestra.run.smithi125.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZspfJ+QMr9tkiR3JeSLbtDVZX5wMU7OwNvd099622r0TeEJpEBUjFdIojO6XfKPpswRL+9JMUiua0DHDiUMWCt6y+70bn6FOygTNs/Xfjbn/LI6DOPbx0u3JkUO0MbX6xJa+xorvK5yEnmC154x0Rsw3B2WBMP1+bf4DkOXc0Jz1fX/RvfgbBff4Ejt/WKDVBgAhcRV1it6EdD8/vQdsd+hjBY+PJV5+epVuZx/OgIM4tv6vy8scg5I94KIRjCS/sBIUNpaD+ionvNdWlHaC6JmPh9dmDfBuoZC/uiD1kIU+b0jb/H409zAL1FgDeT5c+SeLT96whTnSB49VAgf9OhDqIoIGk2hyxlBDFWJZF+OA/fM0mbhc6ywUEyLTe3O6EsZdhVIDYbMgHdL3Ipdr/VA9UaNiBRfGlxEZwpK9YwY3OzbmOIh0mfLAodC1vnjaahr9BBXKFXcsKu4/7t+g6Nte3ycuTUxTPMSsdMw1vKwfomfIcjljeWAqMvmCKT/E= ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:40:03.554 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZspfJ+QMr9tkiR3JeSLbtDVZX5wMU7OwNvd099622r0TeEJpEBUjFdIojO6XfKPpswRL+9JMUiua0DHDiUMWCt6y+70bn6FOygTNs/Xfjbn/LI6DOPbx0u3JkUO0MbX6xJa+xorvK5yEnmC154x0Rsw3B2WBMP1+bf4DkOXc0Jz1fX/RvfgbBff4Ejt/WKDVBgAhcRV1it6EdD8/vQdsd+hjBY+PJV5+epVuZx/OgIM4tv6vy8scg5I94KIRjCS/sBIUNpaD+ionvNdWlHaC6JmPh9dmDfBuoZC/uiD1kIU+b0jb/H409zAL1FgDeT5c+SeLT96whTnSB49VAgf9OhDqIoIGk2hyxlBDFWJZF+OA/fM0mbhc6ywUEyLTe3O6EsZdhVIDYbMgHdL3Ipdr/VA9UaNiBRfGlxEZwpK9YwY3OzbmOIh0mfLAodC1vnjaahr9BBXKFXcsKu4/7t+g6Nte3ycuTUxTPMSsdMw1vKwfomfIcjljeWAqMvmCKT/E= ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-16T16:40:03.600 INFO:teuthology.orchestra.run.smithi159.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZspfJ+QMr9tkiR3JeSLbtDVZX5wMU7OwNvd099622r0TeEJpEBUjFdIojO6XfKPpswRL+9JMUiua0DHDiUMWCt6y+70bn6FOygTNs/Xfjbn/LI6DOPbx0u3JkUO0MbX6xJa+xorvK5yEnmC154x0Rsw3B2WBMP1+bf4DkOXc0Jz1fX/RvfgbBff4Ejt/WKDVBgAhcRV1it6EdD8/vQdsd+hjBY+PJV5+epVuZx/OgIM4tv6vy8scg5I94KIRjCS/sBIUNpaD+ionvNdWlHaC6JmPh9dmDfBuoZC/uiD1kIU+b0jb/H409zAL1FgDeT5c+SeLT96whTnSB49VAgf9OhDqIoIGk2hyxlBDFWJZF+OA/fM0mbhc6ywUEyLTe3O6EsZdhVIDYbMgHdL3Ipdr/VA9UaNiBRfGlxEZwpK9YwY3OzbmOIh0mfLAodC1vnjaahr9BBXKFXcsKu4/7t+g6Nte3ycuTUxTPMSsdMw1vKwfomfIcjljeWAqMvmCKT/E= ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:40:03.613 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-11-16T16:40:04.448 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:40:05.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.820+0000 7fb886e03700 1 -- 172.21.15.93:0/2448867133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8800ffd00 msgr2=0x7fb880100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:05.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.820+0000 7fb886e03700 1 --2- 172.21.15.93:0/2448867133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8800ffd00 0x7fb880100110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb874009910 tx=0x7fb874009c20 comp rx=0 tx=0).stop 2023-11-16T16:40:05.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.820+0000 7fb886e03700 1 -- 172.21.15.93:0/2448867133 shutdown_connections 2023-11-16T16:40:05.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.820+0000 7fb886e03700 1 --2- 172.21.15.93:0/2448867133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8800ffd00 0x7fb880100110 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:05.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.820+0000 7fb886e03700 1 -- 172.21.15.93:0/2448867133 >> 172.21.15.93:0/2448867133 conn(0x7fb8800fb2b0 msgr2=0x7fb8800fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:05.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.820+0000 7fb886e03700 1 -- 172.21.15.93:0/2448867133 shutdown_connections 2023-11-16T16:40:05.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.820+0000 7fb886e03700 1 -- 172.21.15.93:0/2448867133 wait complete. 2023-11-16T16:40:05.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.821+0000 7fb886e03700 1 Processor -- start 2023-11-16T16:40:05.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.821+0000 7fb886e03700 1 -- start start 2023-11-16T16:40:05.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.821+0000 7fb886e03700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8800ffd00 0x7fb880196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:05.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.821+0000 7fb886e03700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb880197530 con 0x7fb8800ffd00 2023-11-16T16:40:05.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.822+0000 7fb884b9f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8800ffd00 0x7fb880196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:05.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.822+0000 7fb884b9f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8800ffd00 0x7fb880196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33902/0 (socket says 172.21.15.93:33902) 2023-11-16T16:40:05.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.822+0000 7fb884b9f700 1 -- 172.21.15.93:0/1142636635 learned_addr learned my addr 172.21.15.93:0/1142636635 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:40:05.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.822+0000 7fb884b9f700 1 -- 172.21.15.93:0/1142636635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8740095f0 con 0x7fb8800ffd00 2023-11-16T16:40:05.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.822+0000 7fb884b9f700 1 --2- 172.21.15.93:0/1142636635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8800ffd00 0x7fb880196ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb874004f40 tx=0x7fb8740057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:05.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.822+0000 7fb87dffb700 1 -- 172.21.15.93:0/1142636635 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb87401d070 con 0x7fb8800ffd00 2023-11-16T16:40:05.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.823+0000 7fb87dffb700 1 -- 172.21.15.93:0/1142636635 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb87400bc10 con 0x7fb8800ffd00 2023-11-16T16:40:05.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.823+0000 7fb886e03700 1 -- 172.21.15.93:0/1142636635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb880105bd0 con 0x7fb8800ffd00 2023-11-16T16:40:05.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.823+0000 7fb87dffb700 1 -- 172.21.15.93:0/1142636635 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb874003a70 con 0x7fb8800ffd00 2023-11-16T16:40:05.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.823+0000 7fb886e03700 1 -- 172.21.15.93:0/1142636635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb880105a40 con 0x7fb8800ffd00 2023-11-16T16:40:05.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.824+0000 7fb87dffb700 1 -- 172.21.15.93:0/1142636635 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fb874003ce0 con 0x7fb8800ffd00 2023-11-16T16:40:05.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.824+0000 7fb886e03700 1 -- 172.21.15.93:0/1142636635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb880105d60 con 0x7fb8800ffd00 2023-11-16T16:40:05.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.824+0000 7fb87dffb700 1 --2- 172.21.15.93:0/1142636635 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb8700388c0 0x7fb87003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:05.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.824+0000 7fb87dffb700 1 -- 172.21.15.93:0/1142636635 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb87404c470 con 0x7fb8800ffd00 2023-11-16T16:40:05.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.824+0000 7fb87ffff700 1 --2- 172.21.15.93:0/1142636635 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb8700388c0 0x7fb87003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:05.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.825+0000 7fb87ffff700 1 --2- 172.21.15.93:0/1142636635 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb8700388c0 0x7fb87003ad70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb86c006fd0 tx=0x7fb86c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:05.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:05.829+0000 7fb87dffb700 1 -- 172.21.15.93:0/1142636635 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb87402aba0 con 0x7fb8800ffd00 2023-11-16T16:40:06.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:06.027+0000 7fb886e03700 1 -- 172.21.15.93:0/1142636635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fb880197b20 con 0x7fb8800ffd00 2023-11-16T16:40:06.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:06.031+0000 7fb87dffb700 1 -- 172.21.15.93:0/1142636635 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb874026070 con 0x7fb8800ffd00 2023-11-16T16:40:06.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:06.033+0000 7fb886e03700 1 -- 172.21.15.93:0/1142636635 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb8700388c0 msgr2=0x7fb87003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:06.033+0000 7fb886e03700 1 --2- 172.21.15.93:0/1142636635 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb8700388c0 0x7fb87003ad70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb86c006fd0 tx=0x7fb86c006e40 comp rx=0 tx=0).stop 2023-11-16T16:40:06.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:06.033+0000 7fb886e03700 1 -- 172.21.15.93:0/1142636635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8800ffd00 msgr2=0x7fb880196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:06.033+0000 7fb886e03700 1 --2- 172.21.15.93:0/1142636635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8800ffd00 0x7fb880196ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb874004f40 tx=0x7fb8740057c0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:06.033+0000 7fb886e03700 1 -- 172.21.15.93:0/1142636635 shutdown_connections 2023-11-16T16:40:06.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:06.033+0000 7fb886e03700 1 --2- 172.21.15.93:0/1142636635 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb8700388c0 0x7fb87003ad70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:06.033+0000 7fb886e03700 1 --2- 172.21.15.93:0/1142636635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8800ffd00 0x7fb880196ff0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:06.033+0000 7fb886e03700 1 -- 172.21.15.93:0/1142636635 >> 172.21.15.93:0/1142636635 conn(0x7fb8800fb2b0 msgr2=0x7fb8800fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:06.033+0000 7fb886e03700 1 -- 172.21.15.93:0/1142636635 shutdown_connections 2023-11-16T16:40:06.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:06.033+0000 7fb886e03700 1 -- 172.21.15.93:0/1142636635 wait complete. 2023-11-16T16:40:06.422 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-11-16T16:40:06.422 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-11-16T16:40:07.261 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:40:07.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:07 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1142636635' entity='client.admin' 2023-11-16T16:40:07.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:07 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:07.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:07 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:08.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.662+0000 7f91dd545700 1 -- 172.21.15.93:0/3417284439 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91d80ffd00 msgr2=0x7f91d8100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:08.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.662+0000 7f91dd545700 1 --2- 172.21.15.93:0/3417284439 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91d80ffd00 0x7f91d8100110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009910 tx=0x7f91c8009c20 comp rx=0 tx=0).stop 2023-11-16T16:40:08.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.662+0000 7f91dd545700 1 -- 172.21.15.93:0/3417284439 shutdown_connections 2023-11-16T16:40:08.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.662+0000 7f91dd545700 1 --2- 172.21.15.93:0/3417284439 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91d80ffd00 0x7f91d8100110 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:08.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.662+0000 7f91dd545700 1 -- 172.21.15.93:0/3417284439 >> 172.21.15.93:0/3417284439 conn(0x7f91d80fb2b0 msgr2=0x7f91d80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:08.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.662+0000 7f91dd545700 1 -- 172.21.15.93:0/3417284439 shutdown_connections 2023-11-16T16:40:08.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.662+0000 7f91dd545700 1 -- 172.21.15.93:0/3417284439 wait complete. 2023-11-16T16:40:08.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.663+0000 7f91dd545700 1 Processor -- start 2023-11-16T16:40:08.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.663+0000 7f91dd545700 1 -- start start 2023-11-16T16:40:08.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.663+0000 7f91dd545700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91d80ffd00 0x7f91d8196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:08.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.663+0000 7f91dd545700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8197530 con 0x7f91d80ffd00 2023-11-16T16:40:08.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.663+0000 7f91d6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91d80ffd00 0x7f91d8196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:08.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.663+0000 7f91d6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91d80ffd00 0x7f91d8196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40478/0 (socket says 172.21.15.93:40478) 2023-11-16T16:40:08.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.663+0000 7f91d6ffd700 1 -- 172.21.15.93:0/3163207370 learned_addr learned my addr 172.21.15.93:0/3163207370 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:40:08.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.664+0000 7f91d6ffd700 1 -- 172.21.15.93:0/3163207370 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c80095f0 con 0x7f91d80ffd00 2023-11-16T16:40:08.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.664+0000 7f91d6ffd700 1 --2- 172.21.15.93:0/3163207370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91d80ffd00 0x7f91d8196ff0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f91c8004f40 tx=0x7f91c80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:08.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.664+0000 7f91bffff700 1 -- 172.21.15.93:0/3163207370 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91c801d070 con 0x7f91d80ffd00 2023-11-16T16:40:08.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.664+0000 7f91bffff700 1 -- 172.21.15.93:0/3163207370 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91c800bc10 con 0x7f91d80ffd00 2023-11-16T16:40:08.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.664+0000 7f91dd545700 1 -- 172.21.15.93:0/3163207370 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d8105bd0 con 0x7f91d80ffd00 2023-11-16T16:40:08.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.664+0000 7f91bffff700 1 -- 172.21.15.93:0/3163207370 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91c8003a70 con 0x7f91d80ffd00 2023-11-16T16:40:08.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.664+0000 7f91dd545700 1 -- 172.21.15.93:0/3163207370 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d8105a40 con 0x7f91d80ffd00 2023-11-16T16:40:08.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.665+0000 7f91bffff700 1 -- 172.21.15.93:0/3163207370 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f91c8003ce0 con 0x7f91d80ffd00 2023-11-16T16:40:08.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.666+0000 7f91dd545700 1 -- 172.21.15.93:0/3163207370 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d8105d60 con 0x7f91d80ffd00 2023-11-16T16:40:08.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.666+0000 7f91bffff700 1 --2- 172.21.15.93:0/3163207370 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f91c00388c0 0x7f91c003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:08.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.666+0000 7f91bffff700 1 -- 172.21.15.93:0/3163207370 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f91c804c470 con 0x7f91d80ffd00 2023-11-16T16:40:08.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.666+0000 7f91d67fc700 1 --2- 172.21.15.93:0/3163207370 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f91c00388c0 0x7f91c003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:08.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.667+0000 7f91d67fc700 1 --2- 172.21.15.93:0/3163207370 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f91c00388c0 0x7f91c003ad70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f91cc006fd0 tx=0x7f91cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:08.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.671+0000 7f91bffff700 1 -- 172.21.15.93:0/3163207370 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f91c802aba0 con 0x7f91d80ffd00 2023-11-16T16:40:08.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.871+0000 7f91dd545700 1 -- 172.21.15.93:0/3163207370 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f91d8060f30 con 0x7f91c00388c0 2023-11-16T16:40:08.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.875+0000 7f91bffff700 1 -- 172.21.15.93:0/3163207370 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f91d8060f30 con 0x7f91c00388c0 2023-11-16T16:40:08.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.877+0000 7f91dd545700 1 -- 172.21.15.93:0/3163207370 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f91c00388c0 msgr2=0x7f91c003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:08.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.878+0000 7f91dd545700 1 --2- 172.21.15.93:0/3163207370 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f91c00388c0 0x7f91c003ad70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f91cc006fd0 tx=0x7f91cc006e40 comp rx=0 tx=0).stop 2023-11-16T16:40:08.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.878+0000 7f91dd545700 1 -- 172.21.15.93:0/3163207370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91d80ffd00 msgr2=0x7f91d8196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:08.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.878+0000 7f91dd545700 1 --2- 172.21.15.93:0/3163207370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91d80ffd00 0x7f91d8196ff0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f91c8004f40 tx=0x7f91c80057c0 comp rx=0 tx=0).stop 2023-11-16T16:40:08.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.878+0000 7f91dd545700 1 -- 172.21.15.93:0/3163207370 shutdown_connections 2023-11-16T16:40:08.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.878+0000 7f91dd545700 1 --2- 172.21.15.93:0/3163207370 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f91c00388c0 0x7f91c003ad70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:08.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.878+0000 7f91dd545700 1 --2- 172.21.15.93:0/3163207370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91d80ffd00 0x7f91d8196ff0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:08.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.878+0000 7f91dd545700 1 -- 172.21.15.93:0/3163207370 >> 172.21.15.93:0/3163207370 conn(0x7f91d80fb2b0 msgr2=0x7f91d80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:08.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.878+0000 7f91dd545700 1 -- 172.21.15.93:0/3163207370 shutdown_connections 2023-11-16T16:40:08.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:08.878+0000 7f91dd545700 1 -- 172.21.15.93:0/3163207370 wait complete. 2023-11-16T16:40:09.336 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi125 2023-11-16T16:40:09.336 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-11-16T16:40:09.336 DEBUG:teuthology.orchestra.run.smithi125:> dd of=/etc/ceph/ceph.conf 2023-11-16T16:40:09.357 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-11-16T16:40:09.357 DEBUG:teuthology.orchestra.run.smithi125:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:09.416 INFO:tasks.cephadm:Adding host smithi125 to orchestrator... 2023-11-16T16:40:09.416 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch host add smithi125 2023-11-16T16:40:10.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:09 smithi093 ceph-mon[97863]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:40:10.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:10.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:10.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:10.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:10.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:09 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:40:10.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:09 smithi093 ceph-mon[97863]: Updating smithi093:/etc/ceph/ceph.conf 2023-11-16T16:40:10.308 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:40:10.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:10 smithi093 ceph-mon[97863]: Updating smithi093:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:10.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:10 smithi093 ceph-mon[97863]: Updating smithi093:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:11.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.926+0000 7f8a04ae6700 1 -- 172.21.15.93:0/3199493453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a000fff90 msgr2=0x7f8a00100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:11.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.926+0000 7f8a04ae6700 1 --2- 172.21.15.93:0/3199493453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a000fff90 0x7f8a00100360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009910 tx=0x7f89f0009c20 comp rx=0 tx=0).stop 2023-11-16T16:40:11.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.926+0000 7f8a04ae6700 1 -- 172.21.15.93:0/3199493453 shutdown_connections 2023-11-16T16:40:11.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.926+0000 7f8a04ae6700 1 --2- 172.21.15.93:0/3199493453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a000fff90 0x7f8a00100360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:11.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.926+0000 7f8a04ae6700 1 -- 172.21.15.93:0/3199493453 >> 172.21.15.93:0/3199493453 conn(0x7f8a000fb850 msgr2=0x7f8a000fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:11.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.926+0000 7f8a04ae6700 1 -- 172.21.15.93:0/3199493453 shutdown_connections 2023-11-16T16:40:11.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.926+0000 7f8a04ae6700 1 -- 172.21.15.93:0/3199493453 wait complete. 2023-11-16T16:40:11.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.927+0000 7f8a04ae6700 1 Processor -- start 2023-11-16T16:40:11.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.927+0000 7f8a04ae6700 1 -- start start 2023-11-16T16:40:11.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.927+0000 7f8a04ae6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a000fff90 0x7f8a00072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:11.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.927+0000 7f8a04ae6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a00074d70 con 0x7f8a000fff90 2023-11-16T16:40:11.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.928+0000 7f89fe59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a000fff90 0x7f8a00072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:11.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.928+0000 7f89fe59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a000fff90 0x7f8a00072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40492/0 (socket says 172.21.15.93:40492) 2023-11-16T16:40:11.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.928+0000 7f89fe59c700 1 -- 172.21.15.93:0/1892647061 learned_addr learned my addr 172.21.15.93:0/1892647061 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:40:11.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.928+0000 7f89fe59c700 1 -- 172.21.15.93:0/1892647061 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f00095f0 con 0x7f8a000fff90 2023-11-16T16:40:11.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.928+0000 7f89fe59c700 1 --2- 172.21.15.93:0/1892647061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a000fff90 0x7f8a00072df0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f89f0006010 tx=0x7f89f0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:11.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.928+0000 7f89ef7fe700 1 -- 172.21.15.93:0/1892647061 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89f001d070 con 0x7f8a000fff90 2023-11-16T16:40:11.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.928+0000 7f89ef7fe700 1 -- 172.21.15.93:0/1892647061 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89f000bc10 con 0x7f8a000fff90 2023-11-16T16:40:11.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.928+0000 7f8a04ae6700 1 -- 172.21.15.93:0/1892647061 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0010a170 con 0x7f8a000fff90 2023-11-16T16:40:11.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.928+0000 7f89ef7fe700 1 -- 172.21.15.93:0/1892647061 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89f0003910 con 0x7f8a000fff90 2023-11-16T16:40:11.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.929+0000 7f8a04ae6700 1 -- 172.21.15.93:0/1892647061 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a00109fe0 con 0x7f8a000fff90 2023-11-16T16:40:11.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.930+0000 7f89ef7fe700 1 -- 172.21.15.93:0/1892647061 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f89f000bd80 con 0x7f8a000fff90 2023-11-16T16:40:11.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.930+0000 7f8a04ae6700 1 -- 172.21.15.93:0/1892647061 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0010a300 con 0x7f8a000fff90 2023-11-16T16:40:11.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.930+0000 7f89ef7fe700 1 --2- 172.21.15.93:0/1892647061 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f89e80388c0 0x7f89e803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:11.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.930+0000 7f89ef7fe700 1 -- 172.21.15.93:0/1892647061 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f89f004c4a0 con 0x7f8a000fff90 2023-11-16T16:40:11.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.930+0000 7f89fdd9b700 1 --2- 172.21.15.93:0/1892647061 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f89e80388c0 0x7f89e803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:11.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.931+0000 7f89fdd9b700 1 --2- 172.21.15.93:0/1892647061 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f89e80388c0 0x7f89e803ad70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f89f4006fd0 tx=0x7f89f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:11.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:11.935+0000 7f89ef7fe700 1 -- 172.21.15.93:0/1892647061 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f89f0029360 con 0x7f8a000fff90 2023-11-16T16:40:12.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:12.136+0000 7f8a04ae6700 1 -- 172.21.15.93:0/1892647061 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi125", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a00074320 con 0x7f89e80388c0 2023-11-16T16:40:12.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:12 smithi093 ceph-mon[97863]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:12.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:12 smithi093 ceph-mon[97863]: Updating smithi093:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:40:12.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:12.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:12.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:14.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:14.249+0000 7f89ef7fe700 1 -- 172.21.15.93:0/1892647061 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8a00074320 con 0x7f89e80388c0 2023-11-16T16:40:14.251 INFO:teuthology.orchestra.run.smithi093.stdout:Added host 'smithi125' with addr '172.21.15.125' 2023-11-16T16:40:14.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:14.251+0000 7f8a04ae6700 1 -- 172.21.15.93:0/1892647061 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f89e80388c0 msgr2=0x7f89e803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:14.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:14.251+0000 7f8a04ae6700 1 --2- 172.21.15.93:0/1892647061 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f89e80388c0 0x7f89e803ad70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f89f4006fd0 tx=0x7f89f4006e40 comp rx=0 tx=0).stop 2023-11-16T16:40:14.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:14.251+0000 7f8a04ae6700 1 -- 172.21.15.93:0/1892647061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a000fff90 msgr2=0x7f8a00072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:14.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:14.251+0000 7f8a04ae6700 1 --2- 172.21.15.93:0/1892647061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a000fff90 0x7f8a00072df0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f89f0006010 tx=0x7f89f0004ff0 comp rx=0 tx=0).stop 2023-11-16T16:40:14.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:14.251+0000 7f8a04ae6700 1 -- 172.21.15.93:0/1892647061 shutdown_connections 2023-11-16T16:40:14.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:14.251+0000 7f8a04ae6700 1 --2- 172.21.15.93:0/1892647061 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f89e80388c0 0x7f89e803ad70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:14.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:14.251+0000 7f8a04ae6700 1 --2- 172.21.15.93:0/1892647061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a000fff90 0x7f8a00072df0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:14.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:14.251+0000 7f8a04ae6700 1 -- 172.21.15.93:0/1892647061 >> 172.21.15.93:0/1892647061 conn(0x7f8a000fb850 msgr2=0x7f8a00104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:14.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:14.252+0000 7f8a04ae6700 1 -- 172.21.15.93:0/1892647061 shutdown_connections 2023-11-16T16:40:14.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:14.252+0000 7f8a04ae6700 1 -- 172.21.15.93:0/1892647061 wait complete. 2023-11-16T16:40:14.275 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:14 smithi093 ceph-mon[97863]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi125", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:40:14.275 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:14 smithi093 ceph-mon[97863]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:14.275 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:14 smithi093 ceph-mon[97863]: Deploying cephadm binary to smithi125 2023-11-16T16:40:14.592 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch host ls --format=json 2023-11-16T16:40:15.437 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:40:15.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:15 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:15.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:15 smithi093 ceph-mon[97863]: Added host smithi125 2023-11-16T16:40:15.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:15 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:15.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:15 smithi093 ceph-mon[97863]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:15.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:15 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:16.744 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:16 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:16.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.919+0000 7fa2000c6700 1 -- 172.21.15.93:0/1442127840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1f80fff90 msgr2=0x7fa1f8100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:16.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.919+0000 7fa2000c6700 1 --2- 172.21.15.93:0/1442127840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1f80fff90 0x7fa1f8100360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009910 tx=0x7fa1f4009c20 comp rx=0 tx=0).stop 2023-11-16T16:40:16.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.919+0000 7fa2000c6700 1 -- 172.21.15.93:0/1442127840 shutdown_connections 2023-11-16T16:40:16.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.919+0000 7fa2000c6700 1 --2- 172.21.15.93:0/1442127840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1f80fff90 0x7fa1f8100360 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:16.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.919+0000 7fa2000c6700 1 -- 172.21.15.93:0/1442127840 >> 172.21.15.93:0/1442127840 conn(0x7fa1f80fb850 msgr2=0x7fa1f80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:16.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.919+0000 7fa2000c6700 1 -- 172.21.15.93:0/1442127840 shutdown_connections 2023-11-16T16:40:16.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.920+0000 7fa2000c6700 1 -- 172.21.15.93:0/1442127840 wait complete. 2023-11-16T16:40:16.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.921+0000 7fa2000c6700 1 Processor -- start 2023-11-16T16:40:16.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.921+0000 7fa2000c6700 1 -- start start 2023-11-16T16:40:16.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.921+0000 7fa2000c6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1f80fff90 0x7fa1f81951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:16.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.921+0000 7fa2000c6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f8195700 con 0x7fa1f80fff90 2023-11-16T16:40:16.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.921+0000 7fa1fde62700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1f80fff90 0x7fa1f81951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:16.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.921+0000 7fa1fde62700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1f80fff90 0x7fa1f81951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:60234/0 (socket says 172.21.15.93:60234) 2023-11-16T16:40:16.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.921+0000 7fa1fde62700 1 -- 172.21.15.93:0/1251141434 learned_addr learned my addr 172.21.15.93:0/1251141434 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:40:16.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.922+0000 7fa1fde62700 1 -- 172.21.15.93:0/1251141434 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f40095f0 con 0x7fa1f80fff90 2023-11-16T16:40:16.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.922+0000 7fa1fde62700 1 --2- 172.21.15.93:0/1251141434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1f80fff90 0x7fa1f81951c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4006010 tx=0x7fa1f4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:16.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.922+0000 7fa1eeffd700 1 -- 172.21.15.93:0/1251141434 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1f401d070 con 0x7fa1f80fff90 2023-11-16T16:40:16.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.922+0000 7fa1eeffd700 1 -- 172.21.15.93:0/1251141434 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1f400bc10 con 0x7fa1f80fff90 2023-11-16T16:40:16.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.922+0000 7fa1eeffd700 1 -- 172.21.15.93:0/1251141434 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1f4003910 con 0x7fa1f80fff90 2023-11-16T16:40:16.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.922+0000 7fa2000c6700 1 -- 172.21.15.93:0/1251141434 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f810a170 con 0x7fa1f80fff90 2023-11-16T16:40:16.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.922+0000 7fa2000c6700 1 -- 172.21.15.93:0/1251141434 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f8109fe0 con 0x7fa1f80fff90 2023-11-16T16:40:16.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.923+0000 7fa1eeffd700 1 -- 172.21.15.93:0/1251141434 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fa1f400bd80 con 0x7fa1f80fff90 2023-11-16T16:40:16.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.923+0000 7fa2000c6700 1 -- 172.21.15.93:0/1251141434 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f810a300 con 0x7fa1f80fff90 2023-11-16T16:40:16.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.924+0000 7fa1eeffd700 1 --2- 172.21.15.93:0/1251141434 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa1e40388c0 0x7fa1e403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:16.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.924+0000 7fa1eeffd700 1 -- 172.21.15.93:0/1251141434 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa1f404c560 con 0x7fa1f80fff90 2023-11-16T16:40:16.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.924+0000 7fa1fd661700 1 --2- 172.21.15.93:0/1251141434 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa1e40388c0 0x7fa1e403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:16.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.925+0000 7fa1fd661700 1 --2- 172.21.15.93:0/1251141434 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa1e40388c0 0x7fa1e403ad70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8006fd0 tx=0x7fa1e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:16.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:16.929+0000 7fa1eeffd700 1 -- 172.21.15.93:0/1251141434 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa1f4029360 con 0x7fa1f80fff90 2023-11-16T16:40:17.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:17.129+0000 7fa2000c6700 1 -- 172.21.15.93:0/1251141434 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1f81080d0 con 0x7fa1e40388c0 2023-11-16T16:40:17.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:17.130+0000 7fa1eeffd700 1 -- 172.21.15.93:0/1251141434 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fa1f81080d0 con 0x7fa1e40388c0 2023-11-16T16:40:17.132 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:40:17.132 INFO:teuthology.orchestra.run.smithi093.stdout:[{"addr": "172.21.15.93", "hostname": "smithi093", "labels": [], "status": ""}, {"addr": "172.21.15.125", "hostname": "smithi125", "labels": [], "status": ""}] 2023-11-16T16:40:17.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:17.132+0000 7fa2000c6700 1 -- 172.21.15.93:0/1251141434 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa1e40388c0 msgr2=0x7fa1e403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:17.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:17.132+0000 7fa2000c6700 1 --2- 172.21.15.93:0/1251141434 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa1e40388c0 0x7fa1e403ad70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8006fd0 tx=0x7fa1e8006e40 comp rx=0 tx=0).stop 2023-11-16T16:40:17.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:17.132+0000 7fa2000c6700 1 -- 172.21.15.93:0/1251141434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1f80fff90 msgr2=0x7fa1f81951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:17.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:17.132+0000 7fa2000c6700 1 --2- 172.21.15.93:0/1251141434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1f80fff90 0x7fa1f81951c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4006010 tx=0x7fa1f4004ff0 comp rx=0 tx=0).stop 2023-11-16T16:40:17.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:17.132+0000 7fa2000c6700 1 -- 172.21.15.93:0/1251141434 shutdown_connections 2023-11-16T16:40:17.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:17.132+0000 7fa2000c6700 1 --2- 172.21.15.93:0/1251141434 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa1e40388c0 0x7fa1e403ad70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:17.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:17.132+0000 7fa2000c6700 1 --2- 172.21.15.93:0/1251141434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1f80fff90 0x7fa1f81951c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:17.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:17.132+0000 7fa2000c6700 1 -- 172.21.15.93:0/1251141434 >> 172.21.15.93:0/1251141434 conn(0x7fa1f80fb850 msgr2=0x7fa1f8104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:17.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:17.132+0000 7fa2000c6700 1 -- 172.21.15.93:0/1251141434 shutdown_connections 2023-11-16T16:40:17.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:17.133+0000 7fa2000c6700 1 -- 172.21.15.93:0/1251141434 wait complete. 2023-11-16T16:40:17.513 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi159 2023-11-16T16:40:17.513 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-11-16T16:40:17.513 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.conf 2023-11-16T16:40:17.534 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-11-16T16:40:17.535 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:17.594 INFO:tasks.cephadm:Adding host smithi159 to orchestrator... 2023-11-16T16:40:17.594 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch host add smithi159 2023-11-16T16:40:18.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:17 smithi093 ceph-mon[97863]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:18.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:18.433 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:40:18.951 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:18 smithi093 ceph-mon[97863]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:40:19.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.829+0000 7fbb1f962700 1 -- 172.21.15.93:0/1899555935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb180fff90 msgr2=0x7fbb18100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:19.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.829+0000 7fbb1f962700 1 --2- 172.21.15.93:0/1899555935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb180fff90 0x7fbb18100360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009910 tx=0x7fbb14009c20 comp rx=0 tx=0).stop 2023-11-16T16:40:19.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.829+0000 7fbb1f962700 1 -- 172.21.15.93:0/1899555935 shutdown_connections 2023-11-16T16:40:19.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.829+0000 7fbb1f962700 1 --2- 172.21.15.93:0/1899555935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb180fff90 0x7fbb18100360 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:19.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.829+0000 7fbb1f962700 1 -- 172.21.15.93:0/1899555935 >> 172.21.15.93:0/1899555935 conn(0x7fbb180fb850 msgr2=0x7fbb180fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:19.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.829+0000 7fbb1f962700 1 -- 172.21.15.93:0/1899555935 shutdown_connections 2023-11-16T16:40:19.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.830+0000 7fbb1f962700 1 -- 172.21.15.93:0/1899555935 wait complete. 2023-11-16T16:40:19.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.831+0000 7fbb1f962700 1 Processor -- start 2023-11-16T16:40:19.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.831+0000 7fbb1f962700 1 -- start start 2023-11-16T16:40:19.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.831+0000 7fbb1f962700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb180fff90 0x7fbb181951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:19.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.831+0000 7fbb1f962700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18195700 con 0x7fbb180fff90 2023-11-16T16:40:19.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.831+0000 7fbb1d6fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb180fff90 0x7fbb181951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:19.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.831+0000 7fbb1d6fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb180fff90 0x7fbb181951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:60246/0 (socket says 172.21.15.93:60246) 2023-11-16T16:40:19.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.831+0000 7fbb1d6fe700 1 -- 172.21.15.93:0/1576909295 learned_addr learned my addr 172.21.15.93:0/1576909295 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:40:19.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.832+0000 7fbb1d6fe700 1 -- 172.21.15.93:0/1576909295 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb140095f0 con 0x7fbb180fff90 2023-11-16T16:40:19.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.832+0000 7fbb1d6fe700 1 --2- 172.21.15.93:0/1576909295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb180fff90 0x7fbb181951c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb14006010 tx=0x7fbb14004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:19.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.832+0000 7fbb0e7fc700 1 -- 172.21.15.93:0/1576909295 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb1401d070 con 0x7fbb180fff90 2023-11-16T16:40:19.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.832+0000 7fbb0e7fc700 1 -- 172.21.15.93:0/1576909295 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb1400bc10 con 0x7fbb180fff90 2023-11-16T16:40:19.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.832+0000 7fbb1f962700 1 -- 172.21.15.93:0/1576909295 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1810a170 con 0x7fbb180fff90 2023-11-16T16:40:19.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.832+0000 7fbb0e7fc700 1 -- 172.21.15.93:0/1576909295 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb14003910 con 0x7fbb180fff90 2023-11-16T16:40:19.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.833+0000 7fbb1f962700 1 -- 172.21.15.93:0/1576909295 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb18109fe0 con 0x7fbb180fff90 2023-11-16T16:40:19.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.834+0000 7fbb0e7fc700 1 -- 172.21.15.93:0/1576909295 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fbb14003bd0 con 0x7fbb180fff90 2023-11-16T16:40:19.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.834+0000 7fbb1f962700 1 -- 172.21.15.93:0/1576909295 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1810a300 con 0x7fbb180fff90 2023-11-16T16:40:19.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.834+0000 7fbb0e7fc700 1 --2- 172.21.15.93:0/1576909295 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbb04038870 0x7fbb0403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:19.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.834+0000 7fbb0e7fc700 1 -- 172.21.15.93:0/1576909295 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbb1404c7a0 con 0x7fbb180fff90 2023-11-16T16:40:19.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.834+0000 7fbb1cefd700 1 --2- 172.21.15.93:0/1576909295 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbb04038870 0x7fbb0403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:19.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.835+0000 7fbb1cefd700 1 --2- 172.21.15.93:0/1576909295 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbb04038870 0x7fbb0403ad20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006fd0 tx=0x7fbb08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:19.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:19.839+0000 7fbb0e7fc700 1 -- 172.21.15.93:0/1576909295 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbb1402aba0 con 0x7fbb180fff90 2023-11-16T16:40:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:19 smithi093 ceph-mon[97863]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi125", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:40:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:20.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:20.043+0000 7fbb1f962700 1 -- 172.21.15.93:0/1576909295 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb181080d0 con 0x7fbb04038870 2023-11-16T16:40:21.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:20 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:40:21.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:20 smithi093 ceph-mon[97863]: Updating smithi125:/etc/ceph/ceph.conf 2023-11-16T16:40:21.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:20 smithi093 ceph-mon[97863]: Updating smithi125:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:21.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:20 smithi093 ceph-mon[97863]: Updating smithi125:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:21.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:20 smithi093 ceph-mon[97863]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:40:21.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:20 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:21.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:20 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:21.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:20 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:22.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:22.143+0000 7fbb0e7fc700 1 -- 172.21.15.93:0/1576909295 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbb181080d0 con 0x7fbb04038870 2023-11-16T16:40:22.145 INFO:teuthology.orchestra.run.smithi093.stdout:Added host 'smithi159' with addr '172.21.15.159' 2023-11-16T16:40:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:22.145+0000 7fbb1f962700 1 -- 172.21.15.93:0/1576909295 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbb04038870 msgr2=0x7fbb0403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:22.145+0000 7fbb1f962700 1 --2- 172.21.15.93:0/1576909295 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbb04038870 0x7fbb0403ad20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006fd0 tx=0x7fbb08006e40 comp rx=0 tx=0).stop 2023-11-16T16:40:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:22.145+0000 7fbb1f962700 1 -- 172.21.15.93:0/1576909295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb180fff90 msgr2=0x7fbb181951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:22.145+0000 7fbb1f962700 1 --2- 172.21.15.93:0/1576909295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb180fff90 0x7fbb181951c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb14006010 tx=0x7fbb14004ff0 comp rx=0 tx=0).stop 2023-11-16T16:40:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:22.145+0000 7fbb1f962700 1 -- 172.21.15.93:0/1576909295 shutdown_connections 2023-11-16T16:40:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:22.145+0000 7fbb1f962700 1 --2- 172.21.15.93:0/1576909295 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbb04038870 0x7fbb0403ad20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:22.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:22.145+0000 7fbb1f962700 1 --2- 172.21.15.93:0/1576909295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb180fff90 0x7fbb181951c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:22.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:22.145+0000 7fbb1f962700 1 -- 172.21.15.93:0/1576909295 >> 172.21.15.93:0/1576909295 conn(0x7fbb180fb850 msgr2=0x7fbb18104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:22.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:22.145+0000 7fbb1f962700 1 -- 172.21.15.93:0/1576909295 shutdown_connections 2023-11-16T16:40:22.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:22.145+0000 7fbb1f962700 1 -- 172.21.15.93:0/1576909295 wait complete. 2023-11-16T16:40:22.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:21 smithi093 ceph-mon[97863]: Updating smithi125:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:40:22.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:21 smithi093 ceph-mon[97863]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:22.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:21 smithi093 ceph-mon[97863]: Deploying cephadm binary to smithi159 2023-11-16T16:40:22.613 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch host ls --format=json 2023-11-16T16:40:23.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:23.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:23 smithi093 ceph-mon[97863]: Added host smithi159 2023-11-16T16:40:23.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:23.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:23 smithi093 ceph-mon[97863]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:23.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:23.456 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:40:24.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:24.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.948+0000 7fde531fe700 1 -- 172.21.15.93:0/733451798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde4c0fff90 msgr2=0x7fde4c100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:24.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.948+0000 7fde531fe700 1 --2- 172.21.15.93:0/733451798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde4c0fff90 0x7fde4c100360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fde40009910 tx=0x7fde40009c20 comp rx=0 tx=0).stop 2023-11-16T16:40:24.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.949+0000 7fde531fe700 1 -- 172.21.15.93:0/733451798 shutdown_connections 2023-11-16T16:40:24.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.949+0000 7fde531fe700 1 --2- 172.21.15.93:0/733451798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde4c0fff90 0x7fde4c100360 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:24.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.949+0000 7fde531fe700 1 -- 172.21.15.93:0/733451798 >> 172.21.15.93:0/733451798 conn(0x7fde4c0fb850 msgr2=0x7fde4c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:24.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.949+0000 7fde531fe700 1 -- 172.21.15.93:0/733451798 shutdown_connections 2023-11-16T16:40:24.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.949+0000 7fde531fe700 1 -- 172.21.15.93:0/733451798 wait complete. 2023-11-16T16:40:24.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.950+0000 7fde531fe700 1 Processor -- start 2023-11-16T16:40:24.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.950+0000 7fde531fe700 1 -- start start 2023-11-16T16:40:24.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.950+0000 7fde531fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde4c0fff90 0x7fde4c1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:24.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.950+0000 7fde531fe700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c1978e0 con 0x7fde4c0fff90 2023-11-16T16:40:24.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.950+0000 7fde50f9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde4c0fff90 0x7fde4c1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:24.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.951+0000 7fde50f9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde4c0fff90 0x7fde4c1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:60254/0 (socket says 172.21.15.93:60254) 2023-11-16T16:40:24.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.951+0000 7fde50f9a700 1 -- 172.21.15.93:0/4027606577 learned_addr learned my addr 172.21.15.93:0/4027606577 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:40:24.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.951+0000 7fde50f9a700 1 -- 172.21.15.93:0/4027606577 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde400095f0 con 0x7fde4c0fff90 2023-11-16T16:40:24.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.951+0000 7fde50f9a700 1 --2- 172.21.15.93:0/4027606577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde4c0fff90 0x7fde4c1973a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fde40006010 tx=0x7fde40004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:24.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.951+0000 7fde49ffb700 1 -- 172.21.15.93:0/4027606577 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde4001d070 con 0x7fde4c0fff90 2023-11-16T16:40:24.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.951+0000 7fde49ffb700 1 -- 172.21.15.93:0/4027606577 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde4000bc10 con 0x7fde4c0fff90 2023-11-16T16:40:24.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.951+0000 7fde531fe700 1 -- 172.21.15.93:0/4027606577 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c10a170 con 0x7fde4c0fff90 2023-11-16T16:40:24.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.951+0000 7fde49ffb700 1 -- 172.21.15.93:0/4027606577 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde40003910 con 0x7fde4c0fff90 2023-11-16T16:40:24.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.952+0000 7fde531fe700 1 -- 172.21.15.93:0/4027606577 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c109fe0 con 0x7fde4c0fff90 2023-11-16T16:40:24.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.952+0000 7fde49ffb700 1 -- 172.21.15.93:0/4027606577 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fde4000bd80 con 0x7fde4c0fff90 2023-11-16T16:40:24.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.953+0000 7fde531fe700 1 -- 172.21.15.93:0/4027606577 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c10a300 con 0x7fde4c0fff90 2023-11-16T16:40:24.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.953+0000 7fde49ffb700 1 --2- 172.21.15.93:0/4027606577 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fde3c0388c0 0x7fde3c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:24.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.953+0000 7fde49ffb700 1 -- 172.21.15.93:0/4027606577 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fde4004c560 con 0x7fde4c0fff90 2023-11-16T16:40:24.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.953+0000 7fde4bfff700 1 --2- 172.21.15.93:0/4027606577 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fde3c0388c0 0x7fde3c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:24.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.954+0000 7fde4bfff700 1 --2- 172.21.15.93:0/4027606577 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fde3c0388c0 0x7fde3c03ad70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fde38006fd0 tx=0x7fde38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:24.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:24.958+0000 7fde49ffb700 1 -- 172.21.15.93:0/4027606577 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fde40029360 con 0x7fde4c0fff90 2023-11-16T16:40:25.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:25.159+0000 7fde531fe700 1 -- 172.21.15.93:0/4027606577 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde4c197ee0 con 0x7fde3c0388c0 2023-11-16T16:40:25.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:25.160+0000 7fde49ffb700 1 -- 172.21.15.93:0/4027606577 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fde4c197ee0 con 0x7fde3c0388c0 2023-11-16T16:40:25.162 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:40:25.162 INFO:teuthology.orchestra.run.smithi093.stdout:[{"addr": "172.21.15.93", "hostname": "smithi093", "labels": [], "status": ""}, {"addr": "172.21.15.125", "hostname": "smithi125", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}] 2023-11-16T16:40:25.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:25.163+0000 7fde531fe700 1 -- 172.21.15.93:0/4027606577 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fde3c0388c0 msgr2=0x7fde3c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:25.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:25.163+0000 7fde531fe700 1 --2- 172.21.15.93:0/4027606577 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fde3c0388c0 0x7fde3c03ad70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fde38006fd0 tx=0x7fde38006e40 comp rx=0 tx=0).stop 2023-11-16T16:40:25.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:25.163+0000 7fde531fe700 1 -- 172.21.15.93:0/4027606577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde4c0fff90 msgr2=0x7fde4c1973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:25.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:25.163+0000 7fde531fe700 1 --2- 172.21.15.93:0/4027606577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde4c0fff90 0x7fde4c1973a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fde40006010 tx=0x7fde40004ff0 comp rx=0 tx=0).stop 2023-11-16T16:40:25.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:25.163+0000 7fde531fe700 1 -- 172.21.15.93:0/4027606577 shutdown_connections 2023-11-16T16:40:25.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:25.163+0000 7fde531fe700 1 --2- 172.21.15.93:0/4027606577 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fde3c0388c0 0x7fde3c03ad70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:25.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:25.163+0000 7fde531fe700 1 --2- 172.21.15.93:0/4027606577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde4c0fff90 0x7fde4c1973a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:25.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:25.163+0000 7fde531fe700 1 -- 172.21.15.93:0/4027606577 >> 172.21.15.93:0/4027606577 conn(0x7fde4c0fb850 msgr2=0x7fde4c1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:25.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:25.163+0000 7fde531fe700 1 -- 172.21.15.93:0/4027606577 shutdown_connections 2023-11-16T16:40:25.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:25.163+0000 7fde531fe700 1 -- 172.21.15.93:0/4027606577 wait complete. 2023-11-16T16:40:25.631 INFO:tasks.cephadm:Setting crush tunables to default 2023-11-16T16:40:25.632 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd crush tunables default 2023-11-16T16:40:25.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:25 smithi093 ceph-mon[97863]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:25.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:25 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:26.475 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:40:27.006 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:26 smithi093 ceph-mon[97863]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:40:27.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.885+0000 7f28e0a50700 1 -- 172.21.15.93:0/3652110292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28dc0fff90 msgr2=0x7f28dc100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:27.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.885+0000 7f28e0a50700 1 --2- 172.21.15.93:0/3652110292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28dc0fff90 0x7f28dc100360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009910 tx=0x7f28cc009c20 comp rx=0 tx=0).stop 2023-11-16T16:40:27.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.885+0000 7f28e0a50700 1 -- 172.21.15.93:0/3652110292 shutdown_connections 2023-11-16T16:40:27.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.885+0000 7f28e0a50700 1 --2- 172.21.15.93:0/3652110292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28dc0fff90 0x7f28dc100360 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:27.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.885+0000 7f28e0a50700 1 -- 172.21.15.93:0/3652110292 >> 172.21.15.93:0/3652110292 conn(0x7f28dc0fb850 msgr2=0x7f28dc0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:27.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.885+0000 7f28e0a50700 1 -- 172.21.15.93:0/3652110292 shutdown_connections 2023-11-16T16:40:27.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.885+0000 7f28e0a50700 1 -- 172.21.15.93:0/3652110292 wait complete. 2023-11-16T16:40:27.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.886+0000 7f28e0a50700 1 Processor -- start 2023-11-16T16:40:27.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.886+0000 7f28e0a50700 1 -- start start 2023-11-16T16:40:27.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.886+0000 7f28e0a50700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28dc0fff90 0x7f28dc072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:27.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.886+0000 7f28e0a50700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc1991f0 con 0x7f28dc0fff90 2023-11-16T16:40:27.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.887+0000 7f28da59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28dc0fff90 0x7f28dc072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:27.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.887+0000 7f28da59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28dc0fff90 0x7f28dc072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44534/0 (socket says 172.21.15.93:44534) 2023-11-16T16:40:27.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.887+0000 7f28da59c700 1 -- 172.21.15.93:0/740505949 learned_addr learned my addr 172.21.15.93:0/740505949 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:40:27.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.887+0000 7f28da59c700 1 -- 172.21.15.93:0/740505949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc0095f0 con 0x7f28dc0fff90 2023-11-16T16:40:27.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.887+0000 7f28da59c700 1 --2- 172.21.15.93:0/740505949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28dc0fff90 0x7f28dc072df0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f28cc006010 tx=0x7f28cc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:27.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.887+0000 7f28cb7fe700 1 -- 172.21.15.93:0/740505949 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28cc01d070 con 0x7f28dc0fff90 2023-11-16T16:40:27.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.887+0000 7f28cb7fe700 1 -- 172.21.15.93:0/740505949 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28cc00bc10 con 0x7f28dc0fff90 2023-11-16T16:40:27.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.887+0000 7f28e0a50700 1 -- 172.21.15.93:0/740505949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28dc10a170 con 0x7f28dc0fff90 2023-11-16T16:40:27.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.887+0000 7f28cb7fe700 1 -- 172.21.15.93:0/740505949 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28cc003910 con 0x7f28dc0fff90 2023-11-16T16:40:27.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.888+0000 7f28e0a50700 1 -- 172.21.15.93:0/740505949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc109fe0 con 0x7f28dc0fff90 2023-11-16T16:40:27.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.889+0000 7f28cb7fe700 1 -- 172.21.15.93:0/740505949 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f28cc00bd80 con 0x7f28dc0fff90 2023-11-16T16:40:27.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.889+0000 7f28e0a50700 1 -- 172.21.15.93:0/740505949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28dc10a300 con 0x7f28dc0fff90 2023-11-16T16:40:27.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.890+0000 7f28cb7fe700 1 --2- 172.21.15.93:0/740505949 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f28c4038870 0x7f28c403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:27.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.890+0000 7f28cb7fe700 1 -- 172.21.15.93:0/740505949 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f28cc04c600 con 0x7f28dc0fff90 2023-11-16T16:40:27.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.890+0000 7f28d9d9b700 1 --2- 172.21.15.93:0/740505949 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f28c4038870 0x7f28c403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:27.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.890+0000 7f28d9d9b700 1 --2- 172.21.15.93:0/740505949 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f28c4038870 0x7f28c403ad20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f28d0006fd0 tx=0x7f28d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:27.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:27.895+0000 7f28cb7fe700 1 -- 172.21.15.93:0/740505949 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f28cc029360 con 0x7f28dc0fff90 2023-11-16T16:40:27.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:27 smithi093 ceph-mon[97863]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:27.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:27 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:27.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:27 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:27.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:27 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:27.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:27 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:27.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:27 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:40:27.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:27 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:27.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:27 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:28.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:28.100+0000 7f28e0a50700 1 -- 172.21.15.93:0/740505949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f28dc074320 con 0x7f28dc0fff90 2023-11-16T16:40:28.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:28.706+0000 7f28cb7fe700 1 -- 172.21.15.93:0/740505949 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f28cc026020 con 0x7f28dc0fff90 2023-11-16T16:40:28.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:28.709+0000 7f28e0a50700 1 -- 172.21.15.93:0/740505949 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f28c4038870 msgr2=0x7f28c403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:28.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:28.709+0000 7f28e0a50700 1 --2- 172.21.15.93:0/740505949 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f28c4038870 0x7f28c403ad20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f28d0006fd0 tx=0x7f28d0006e40 comp rx=0 tx=0).stop 2023-11-16T16:40:28.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:28.709+0000 7f28e0a50700 1 -- 172.21.15.93:0/740505949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28dc0fff90 msgr2=0x7f28dc072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:28.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:28.709+0000 7f28e0a50700 1 --2- 172.21.15.93:0/740505949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28dc0fff90 0x7f28dc072df0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f28cc006010 tx=0x7f28cc004ff0 comp rx=0 tx=0).stop 2023-11-16T16:40:28.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:28.709+0000 7f28e0a50700 1 -- 172.21.15.93:0/740505949 shutdown_connections 2023-11-16T16:40:28.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:28.709+0000 7f28e0a50700 1 --2- 172.21.15.93:0/740505949 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f28c4038870 0x7f28c403ad20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:28.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:28.709+0000 7f28e0a50700 1 --2- 172.21.15.93:0/740505949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28dc0fff90 0x7f28dc072df0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:28.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:28.709+0000 7f28e0a50700 1 -- 172.21.15.93:0/740505949 >> 172.21.15.93:0/740505949 conn(0x7f28dc0fb850 msgr2=0x7f28dc104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:28.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:28.709+0000 7f28e0a50700 1 -- 172.21.15.93:0/740505949 shutdown_connections 2023-11-16T16:40:28.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:28.709+0000 7f28e0a50700 1 -- 172.21.15.93:0/740505949 wait complete. 2023-11-16T16:40:28.712 INFO:teuthology.orchestra.run.smithi093.stderr:adjusted tunables profile to default 2023-11-16T16:40:29.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:28 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:40:29.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:28 smithi093 ceph-mon[97863]: Updating smithi159:/etc/ceph/ceph.conf 2023-11-16T16:40:29.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:28 smithi093 ceph-mon[97863]: Updating smithi159:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:29.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:28 smithi093 ceph-mon[97863]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:29.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:28 smithi093 ceph-mon[97863]: Updating smithi159:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:40:29.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:28 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/740505949' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-16T16:40:29.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:29.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:29.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:29.257 INFO:tasks.cephadm:Adding mon.a on smithi093 2023-11-16T16:40:29.258 INFO:tasks.cephadm:Adding mon.b on smithi125 2023-11-16T16:40:29.258 INFO:tasks.cephadm:Adding mon.c on smithi159 2023-11-16T16:40:29.258 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch apply mon '3;smithi093:172.21.15.93=a;smithi125:172.21.15.125=b;smithi159:172.21.15.159=c' 2023-11-16T16:40:29.931 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:30.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:29 smithi093 ceph-mon[97863]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:30.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:29 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/740505949' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-16T16:40:30.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:29 smithi093 ceph-mon[97863]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:40:31.130 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.128+0000 7fc335ff5700 1 -- 172.21.15.159:0/507694634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3300fff90 msgr2=0x7fc330100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.128+0000 7fc335ff5700 1 --2- 172.21.15.159:0/507694634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3300fff90 0x7fc330100360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc320009910 tx=0x7fc320009c20 comp rx=0 tx=0).stop 2023-11-16T16:40:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.128+0000 7fc335ff5700 1 -- 172.21.15.159:0/507694634 shutdown_connections 2023-11-16T16:40:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.128+0000 7fc335ff5700 1 --2- 172.21.15.159:0/507694634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3300fff90 0x7fc330100360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.128+0000 7fc335ff5700 1 -- 172.21.15.159:0/507694634 >> 172.21.15.159:0/507694634 conn(0x7fc3300fb850 msgr2=0x7fc3300fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.128+0000 7fc335ff5700 1 -- 172.21.15.159:0/507694634 shutdown_connections 2023-11-16T16:40:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.128+0000 7fc335ff5700 1 -- 172.21.15.159:0/507694634 wait complete. 2023-11-16T16:40:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.129+0000 7fc335ff5700 1 Processor -- start 2023-11-16T16:40:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.129+0000 7fc335ff5700 1 -- start start 2023-11-16T16:40:31.132 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.129+0000 7fc335ff5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3300fff90 0x7fc330072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:31.132 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.129+0000 7fc335ff5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc330074d70 con 0x7fc3300fff90 2023-11-16T16:40:31.132 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.130+0000 7fc32f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3300fff90 0x7fc330072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:31.133 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.130+0000 7fc32f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3300fff90 0x7fc330072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.159:52638/0 (socket says 172.21.15.159:52638) 2023-11-16T16:40:31.133 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.130+0000 7fc32f7fe700 1 -- 172.21.15.159:0/1740014147 learned_addr learned my addr 172.21.15.159:0/1740014147 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-11-16T16:40:31.133 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.130+0000 7fc32f7fe700 1 -- 172.21.15.159:0/1740014147 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3200095f0 con 0x7fc3300fff90 2023-11-16T16:40:31.133 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.130+0000 7fc32f7fe700 1 --2- 172.21.15.159:0/1740014147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3300fff90 0x7fc330072df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc320006010 tx=0x7fc320004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:31.133 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.130+0000 7fc32cff9700 1 -- 172.21.15.159:0/1740014147 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc32001d070 con 0x7fc3300fff90 2023-11-16T16:40:31.133 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.130+0000 7fc335ff5700 1 -- 172.21.15.159:0/1740014147 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33010a170 con 0x7fc3300fff90 2023-11-16T16:40:31.133 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.131+0000 7fc32cff9700 1 -- 172.21.15.159:0/1740014147 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc32000bc10 con 0x7fc3300fff90 2023-11-16T16:40:31.133 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.131+0000 7fc32cff9700 1 -- 172.21.15.159:0/1740014147 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc320003910 con 0x7fc3300fff90 2023-11-16T16:40:31.133 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.131+0000 7fc335ff5700 1 -- 172.21.15.159:0/1740014147 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc330109fe0 con 0x7fc3300fff90 2023-11-16T16:40:31.134 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.132+0000 7fc32cff9700 1 -- 172.21.15.159:0/1740014147 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fc32000bd80 con 0x7fc3300fff90 2023-11-16T16:40:31.134 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.132+0000 7fc32cff9700 1 --2- 172.21.15.159:0/1740014147 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc318038870 0x7fc31803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:31.134 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.132+0000 7fc32cff9700 1 -- 172.21.15.159:0/1740014147 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc32004c540 con 0x7fc3300fff90 2023-11-16T16:40:31.134 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.132+0000 7fc335ff5700 1 -- 172.21.15.159:0/1740014147 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33010a300 con 0x7fc3300fff90 2023-11-16T16:40:31.134 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.132+0000 7fc32effd700 1 --2- 172.21.15.159:0/1740014147 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc318038870 0x7fc31803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:31.134 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.133+0000 7fc32effd700 1 --2- 172.21.15.159:0/1740014147 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc318038870 0x7fc31803ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc324006fd0 tx=0x7fc324006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:31.139 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.138+0000 7fc32cff9700 1 -- 172.21.15.159:0/1740014147 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc320026020 con 0x7fc3300fff90 2023-11-16T16:40:31.334 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.332+0000 7fc335ff5700 1 -- 172.21.15.159:0/1740014147 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi093:172.21.15.93=a;smithi125:172.21.15.125=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fc330074320 con 0x7fc318038870 2023-11-16T16:40:31.341 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.340+0000 7fc32cff9700 1 -- 172.21.15.159:0/1740014147 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc330074320 con 0x7fc318038870 2023-11-16T16:40:31.341 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mon update... 2023-11-16T16:40:31.343 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.341+0000 7fc335ff5700 1 -- 172.21.15.159:0/1740014147 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc318038870 msgr2=0x7fc31803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:31.343 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.341+0000 7fc335ff5700 1 --2- 172.21.15.159:0/1740014147 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc318038870 0x7fc31803ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc324006fd0 tx=0x7fc324006e40 comp rx=0 tx=0).stop 2023-11-16T16:40:31.343 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.341+0000 7fc335ff5700 1 -- 172.21.15.159:0/1740014147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3300fff90 msgr2=0x7fc330072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:31.343 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.341+0000 7fc335ff5700 1 --2- 172.21.15.159:0/1740014147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3300fff90 0x7fc330072df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc320006010 tx=0x7fc320004ff0 comp rx=0 tx=0).stop 2023-11-16T16:40:31.343 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.341+0000 7fc335ff5700 1 -- 172.21.15.159:0/1740014147 shutdown_connections 2023-11-16T16:40:31.343 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.341+0000 7fc335ff5700 1 --2- 172.21.15.159:0/1740014147 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc318038870 0x7fc31803ad20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:31.343 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.341+0000 7fc335ff5700 1 --2- 172.21.15.159:0/1740014147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3300fff90 0x7fc330072df0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:31.343 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.341+0000 7fc335ff5700 1 -- 172.21.15.159:0/1740014147 >> 172.21.15.159:0/1740014147 conn(0x7fc3300fb850 msgr2=0x7fc330104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:31.343 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.341+0000 7fc335ff5700 1 -- 172.21.15.159:0/1740014147 shutdown_connections 2023-11-16T16:40:31.343 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:31.341+0000 7fc335ff5700 1 -- 172.21.15.159:0/1740014147 wait complete. 2023-11-16T16:40:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:31 smithi093 ceph-mon[97863]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:31.787 DEBUG:teuthology.orchestra.run.smithi125:mon.b> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.b.service 2023-11-16T16:40:31.789 DEBUG:teuthology.orchestra.run.smithi159:mon.c> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.c.service 2023-11-16T16:40:31.791 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-16T16:40:31.791 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph mon dump -f json 2023-11-16T16:40:31.820 INFO:journalctl@ceph.mon.c.smithi159.stdout:-- Logs begin at Thu 2023-11-16 16:15:38 UTC. -- 2023-11-16T16:40:31.826 INFO:journalctl@ceph.mon.b.smithi125.stdout:-- Logs begin at Thu 2023-11-16 16:14:41 UTC. -- 2023-11-16T16:40:32.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:32 smithi093 ceph-mon[97863]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi093:172.21.15.93=a;smithi125:172.21.15.125=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:40:32.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:32 smithi093 ceph-mon[97863]: Saving service mon spec with placement smithi093:172.21.15.93=a;smithi125:172.21.15.125=b;smithi159:172.21.15.159=c;count:3 2023-11-16T16:40:32.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:32 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:32.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:32 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:32.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:32 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:32.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:32 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:32.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:32 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:40:32.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:32 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:32.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:32 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:40:32.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:32 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:32.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:32 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:32.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:32 smithi093 ceph-mon[97863]: Deploying daemon mon.c on smithi159 2023-11-16T16:40:32.761 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:33.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:33 smithi093 ceph-mon[97863]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:34.365 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.362+0000 7f090a2d7700 1 -- 172.21.15.159:0/454548685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0904068320 msgr2=0x7f09040fed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:34.366 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.362+0000 7f090a2d7700 1 --2- 172.21.15.159:0/454548685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0904068320 0x7f09040fed70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009910 tx=0x7f08f4009c20 comp rx=0 tx=0).stop 2023-11-16T16:40:34.366 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.362+0000 7f090a2d7700 1 -- 172.21.15.159:0/454548685 shutdown_connections 2023-11-16T16:40:34.366 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.362+0000 7f090a2d7700 1 --2- 172.21.15.159:0/454548685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0904068320 0x7f09040fed70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:34.366 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.362+0000 7f090a2d7700 1 -- 172.21.15.159:0/454548685 >> 172.21.15.159:0/454548685 conn(0x7f0904074350 msgr2=0x7f0904074750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:34.366 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.362+0000 7f090a2d7700 1 -- 172.21.15.159:0/454548685 shutdown_connections 2023-11-16T16:40:34.366 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.362+0000 7f090a2d7700 1 -- 172.21.15.159:0/454548685 wait complete. 2023-11-16T16:40:34.366 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.363+0000 7f090a2d7700 1 Processor -- start 2023-11-16T16:40:34.366 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.363+0000 7f090a2d7700 1 -- start start 2023-11-16T16:40:34.366 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.363+0000 7f090a2d7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0904068320 0x7f0904197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:34.366 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.363+0000 7f090a2d7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0904197850 con 0x7f0904068320 2023-11-16T16:40:34.366 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.363+0000 7f0903fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0904068320 0x7f0904197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:34.367 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.363+0000 7f0903fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0904068320 0x7f0904197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.159:52644/0 (socket says 172.21.15.159:52644) 2023-11-16T16:40:34.367 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.363+0000 7f0903fff700 1 -- 172.21.15.159:0/3875621855 learned_addr learned my addr 172.21.15.159:0/3875621855 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-11-16T16:40:34.367 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.364+0000 7f0903fff700 1 -- 172.21.15.159:0/3875621855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40095f0 con 0x7f0904068320 2023-11-16T16:40:34.367 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.364+0000 7f0903fff700 1 --2- 172.21.15.159:0/3875621855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0904068320 0x7f0904197310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08f4004f40 tx=0x7f08f4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:34.367 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.364+0000 7f09017fa700 1 -- 172.21.15.159:0/3875621855 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08f401d070 con 0x7f0904068320 2023-11-16T16:40:34.367 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.364+0000 7f090a2d7700 1 -- 172.21.15.159:0/3875621855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090410c3f0 con 0x7f0904068320 2023-11-16T16:40:34.367 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.364+0000 7f09017fa700 1 -- 172.21.15.159:0/3875621855 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08f4004370 con 0x7f0904068320 2023-11-16T16:40:34.367 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.364+0000 7f09017fa700 1 -- 172.21.15.159:0/3875621855 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08f40039b0 con 0x7f0904068320 2023-11-16T16:40:34.367 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.364+0000 7f090a2d7700 1 -- 172.21.15.159:0/3875621855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090410c260 con 0x7f0904068320 2023-11-16T16:40:34.368 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.365+0000 7f090a2d7700 1 -- 172.21.15.159:0/3875621855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090410c580 con 0x7f0904068320 2023-11-16T16:40:34.368 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.365+0000 7f09017fa700 1 -- 172.21.15.159:0/3875621855 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f08f4003c70 con 0x7f0904068320 2023-11-16T16:40:34.368 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.365+0000 7f09017fa700 1 --2- 172.21.15.159:0/3875621855 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f08ec038870 0x7f08ec03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:34.368 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.365+0000 7f09017fa700 1 -- 172.21.15.159:0/3875621855 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f08f404c580 con 0x7f0904068320 2023-11-16T16:40:34.368 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.366+0000 7f09037fe700 1 --2- 172.21.15.159:0/3875621855 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f08ec038870 0x7f08ec03ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:34.368 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.366+0000 7f09037fe700 1 --2- 172.21.15.159:0/3875621855 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f08ec038870 0x7f08ec03ad20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f08f8006fd0 tx=0x7f08f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:34.372 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.371+0000 7f09017fa700 1 -- 172.21.15.159:0/3875621855 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f08f4029770 con 0x7f0904068320 2023-11-16T16:40:34.627 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:34 smithi159 systemd[1]: Starting Ceph mon.c for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:40:34.660 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.658+0000 7f090a2d7700 1 -- 172.21.15.159:0/3875621855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0904062120 con 0x7f0904068320 2023-11-16T16:40:34.661 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.659+0000 7f09017fa700 1 -- 172.21.15.159:0/3875621855 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f08f4026070 con 0x7f0904068320 2023-11-16T16:40:34.661 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:40:34.661 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":1,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","modified":"2023-11-16T16:38:51.851994Z","created":"2023-11-16T16:38:51.851994Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:3300","nonce":0},{"type":"v1","addr":"172.21.15.93:6789","nonce":0}]},"addr":"172.21.15.93:6789/0","public_addr":"172.21.15.93:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-11-16T16:40:34.663 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.661+0000 7f090a2d7700 1 -- 172.21.15.159:0/3875621855 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f08ec038870 msgr2=0x7f08ec03ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:34.663 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.661+0000 7f090a2d7700 1 --2- 172.21.15.159:0/3875621855 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f08ec038870 0x7f08ec03ad20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f08f8006fd0 tx=0x7f08f8006e40 comp rx=0 tx=0).stop 2023-11-16T16:40:34.663 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.661+0000 7f090a2d7700 1 -- 172.21.15.159:0/3875621855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0904068320 msgr2=0x7f0904197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:34.663 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.661+0000 7f090a2d7700 1 --2- 172.21.15.159:0/3875621855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0904068320 0x7f0904197310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08f4004f40 tx=0x7f08f4005020 comp rx=0 tx=0).stop 2023-11-16T16:40:34.663 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.661+0000 7f090a2d7700 1 -- 172.21.15.159:0/3875621855 shutdown_connections 2023-11-16T16:40:34.664 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.661+0000 7f090a2d7700 1 --2- 172.21.15.159:0/3875621855 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f08ec038870 0x7f08ec03ad20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:34.664 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.661+0000 7f090a2d7700 1 --2- 172.21.15.159:0/3875621855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0904068320 0x7f0904197310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:34.664 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.661+0000 7f090a2d7700 1 -- 172.21.15.159:0/3875621855 >> 172.21.15.159:0/3875621855 conn(0x7f0904074350 msgr2=0x7f09040fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:34.664 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.662+0000 7f090a2d7700 1 -- 172.21.15.159:0/3875621855 shutdown_connections 2023-11-16T16:40:34.664 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:34.662+0000 7f090a2d7700 1 -- 172.21.15.159:0/3875621855 wait complete. 2023-11-16T16:40:34.664 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 1 2023-11-16T16:40:34.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:34 smithi159 podman[131462]: 2023-11-16T16:40:35.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 bash[131462]: 9d282f28b78b46ea2838f43ef6b9a0dcd4cfdec9d455ce89b44d373a88cbfbc6 2023-11-16T16:40:35.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: set uid:gid to 167:167 (ceph:ceph) 2023-11-16T16:40:35.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev), process ceph-mon, pid 7 2023-11-16T16:40:35.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: pidfile_write: ignore empty --pid-file 2023-11-16T16:40:35.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: load: jerasure load: lrc 2023-11-16T16:40:35.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: RocksDB version: 7.9.2 2023-11-16T16:40:35.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Git sha 0 2023-11-16T16:40:35.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Compile date 2023-11-15 23:53:54 2023-11-16T16:40:35.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: DB SUMMARY 2023-11-16T16:40:35.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: DB Session ID: E552YZ6SFKZFGMGZOPSG 2023-11-16T16:40:35.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: CURRENT file: CURRENT 2023-11-16T16:40:35.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: IDENTITY file: IDENTITY 2023-11-16T16:40:35.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-16T16:40:35.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.error_if_exists: 0 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.create_if_missing: 0 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.paranoid_checks: 1 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.env: 0x5592e3dfc720 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.fs: PosixFileSystem 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.info_log: 0x5592e6c093c0 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_file_opening_threads: 16 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.statistics: (nil) 2023-11-16T16:40:35.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.use_fsync: 0 2023-11-16T16:40:35.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_log_file_size: 0 2023-11-16T16:40:35.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-16T16:40:35.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-16T16:40:35.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.keep_log_file_num: 1000 2023-11-16T16:40:35.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.recycle_log_file_num: 0 2023-11-16T16:40:35.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.allow_fallocate: 1 2023-11-16T16:40:35.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.allow_mmap_reads: 0 2023-11-16T16:40:35.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.allow_mmap_writes: 0 2023-11-16T16:40:35.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.use_direct_reads: 0 2023-11-16T16:40:35.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-16T16:40:35.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.create_missing_column_families: 0 2023-11-16T16:40:35.489 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.db_log_dir: 2023-11-16T16:40:35.489 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.wal_dir: 2023-11-16T16:40:35.490 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-16T16:40:35.490 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-16T16:40:35.490 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-16T16:40:35.490 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-16T16:40:35.491 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-16T16:40:35.491 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-16T16:40:35.491 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.advise_random_on_open: 1 2023-11-16T16:40:35.491 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.db_write_buffer_size: 0 2023-11-16T16:40:35.491 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.write_buffer_manager: 0x5592e5e7a3c0 2023-11-16T16:40:35.491 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-16T16:40:35.491 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-16T16:40:35.491 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-16T16:40:35.491 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.rate_limiter: (nil) 2023-11-16T16:40:35.491 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-16T16:40:35.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.wal_recovery_mode: 2 2023-11-16T16:40:35.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.enable_thread_tracking: 0 2023-11-16T16:40:35.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.enable_pipelined_write: 0 2023-11-16T16:40:35.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.unordered_write: 0 2023-11-16T16:40:35.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-16T16:40:35.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-16T16:40:35.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-16T16:40:35.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-16T16:40:35.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.row_cache: None 2023-11-16T16:40:35.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.wal_filter: None 2023-11-16T16:40:35.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-16T16:40:35.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.allow_ingest_behind: 0 2023-11-16T16:40:35.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.two_write_queues: 0 2023-11-16T16:40:35.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.manual_wal_flush: 0 2023-11-16T16:40:35.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.wal_compression: 0 2023-11-16T16:40:35.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.atomic_flush: 0 2023-11-16T16:40:35.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-16T16:40:35.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-16T16:40:35.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-16T16:40:35.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.log_readahead_size: 0 2023-11-16T16:40:35.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-16T16:40:35.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.best_efforts_recovery: 0 2023-11-16T16:40:35.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.allow_data_in_errors: 0 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.db_host_id: __hostname__ 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_background_jobs: 2 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_background_compactions: -1 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_subcompactions: 1 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_total_wal_size: 0 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-16T16:40:35.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_open_files: -1 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bytes_per_sync: 0 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_readahead_size: 0 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_background_flushes: -1 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Compression algorithms supported: 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: kZSTD supported: 0 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: kXpressCompression supported: 0 2023-11-16T16:40:35.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: kLZ4HCCompression supported: 1 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: kZlibCompression supported: 1 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: kSnappyCompression supported: 1 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: kLZ4Compression supported: 1 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: kBZip2Compression supported: 0 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.merge_operator: 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_filter: None 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_filter_factory: None 2023-11-16T16:40:35.497 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.sst_partitioner_factory: None 2023-11-16T16:40:35.498 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-16T16:40:35.498 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-16T16:40:35.498 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5592e6c094e0) 2023-11-16T16:40:35.498 INFO:journalctl@ceph.mon.c.smithi159.stdout: cache_index_and_filter_blocks: 1 2023-11-16T16:40:35.498 INFO:journalctl@ceph.mon.c.smithi159.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-16T16:40:35.498 INFO:journalctl@ceph.mon.c.smithi159.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-16T16:40:35.498 INFO:journalctl@ceph.mon.c.smithi159.stdout: pin_top_level_index_and_filter: 1 2023-11-16T16:40:35.498 INFO:journalctl@ceph.mon.c.smithi159.stdout: index_type: 0 2023-11-16T16:40:35.498 INFO:journalctl@ceph.mon.c.smithi159.stdout: data_block_index_type: 0 2023-11-16T16:40:35.498 INFO:journalctl@ceph.mon.c.smithi159.stdout: index_shortening: 1 2023-11-16T16:40:35.498 INFO:journalctl@ceph.mon.c.smithi159.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-16T16:40:35.499 INFO:journalctl@ceph.mon.c.smithi159.stdout: checksum: 4 2023-11-16T16:40:35.499 INFO:journalctl@ceph.mon.c.smithi159.stdout: no_block_cache: 0 2023-11-16T16:40:35.499 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_cache: 0x5592e5f1b1f0 2023-11-16T16:40:35.499 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_cache_name: BinnedLRUCache 2023-11-16T16:40:35.499 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_cache_options: 2023-11-16T16:40:35.499 INFO:journalctl@ceph.mon.c.smithi159.stdout: capacity : 536870912 2023-11-16T16:40:35.499 INFO:journalctl@ceph.mon.c.smithi159.stdout: num_shard_bits : 4 2023-11-16T16:40:35.499 INFO:journalctl@ceph.mon.c.smithi159.stdout: strict_capacity_limit : 0 2023-11-16T16:40:35.499 INFO:journalctl@ceph.mon.c.smithi159.stdout: high_pri_pool_ratio: 0.000 2023-11-16T16:40:35.499 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_cache_compressed: (nil) 2023-11-16T16:40:35.499 INFO:journalctl@ceph.mon.c.smithi159.stdout: persistent_cache: (nil) 2023-11-16T16:40:35.499 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_size: 4096 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_size_deviation: 10 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_restart_interval: 16 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: index_block_restart_interval: 1 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: metadata_block_size: 4096 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: partition_filters: 0 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: use_delta_encoding: 1 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: filter_policy: bloomfilter 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: whole_key_filtering: 1 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: verify_compression: 0 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: read_amp_bytes_per_bit: 0 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: format_version: 5 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: enable_index_compression: 1 2023-11-16T16:40:35.500 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_align: 0 2023-11-16T16:40:35.501 INFO:journalctl@ceph.mon.c.smithi159.stdout: max_auto_readahead_size: 262144 2023-11-16T16:40:35.501 INFO:journalctl@ceph.mon.c.smithi159.stdout: prepopulate_block_cache: 0 2023-11-16T16:40:35.501 INFO:journalctl@ceph.mon.c.smithi159.stdout: initial_auto_readahead_size: 8192 2023-11-16T16:40:35.501 INFO:journalctl@ceph.mon.c.smithi159.stdout: num_file_reads_for_auto_readahead: 2 2023-11-16T16:40:35.501 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.write_buffer_size: 33554432 2023-11-16T16:40:35.501 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_write_buffer_number: 2 2023-11-16T16:40:35.501 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compression: NoCompression 2023-11-16T16:40:35.501 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bottommost_compression: Disabled 2023-11-16T16:40:35.501 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.prefix_extractor: nullptr 2023-11-16T16:40:35.501 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-16T16:40:35.501 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.num_levels: 7 2023-11-16T16:40:35.501 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-16T16:40:35.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-16T16:40:35.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-16T16:40:35.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-16T16:40:35.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-16T16:40:35.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-16T16:40:35.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-16T16:40:35.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-16T16:40:35.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-16T16:40:35.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-16T16:40:35.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-16T16:40:35.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-16T16:40:35.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-16T16:40:35.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compression_opts.level: 32767 2023-11-16T16:40:35.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compression_opts.strategy: 0 2023-11-16T16:40:35.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-16T16:40:35.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-16T16:40:35.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-16T16:40:35.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-16T16:40:35.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compression_opts.enabled: false 2023-11-16T16:40:35.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-16T16:40:35.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-16T16:40:35.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-16T16:40:35.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-16T16:40:35.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.target_file_size_base: 67108864 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-16T16:40:35.504 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-16T16:40:35.505 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.arena_block_size: 1048576 2023-11-16T16:40:35.505 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-16T16:40:35.505 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-16T16:40:35.505 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.disable_auto_compactions: 0 2023-11-16T16:40:35.505 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-16T16:40:35.505 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-16T16:40:35.505 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-16T16:40:35.505 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-16T16:40:35.505 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-16T16:40:35.505 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-16T16:40:35.505 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-16T16:40:35.505 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-16T16:40:35.506 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-16T16:40:35.506 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-16T16:40:35.506 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.table_properties_collectors: 2023-11-16T16:40:35.506 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.inplace_update_support: 0 2023-11-16T16:40:35.506 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-16T16:40:35.506 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-16T16:40:35.506 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-16T16:40:35.506 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-16T16:40:35.506 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.bloom_locality: 0 2023-11-16T16:40:35.506 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.max_successive_merges: 0 2023-11-16T16:40:35.506 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-16T16:40:35.506 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.paranoid_file_checks: 0 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.force_consistency_checks: 1 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.report_bg_io_stats: 0 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.ttl: 2592000 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.enable_blob_files: false 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.min_blob_size: 0 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.blob_file_size: 268435456 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-16T16:40:35.507 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-16T16:40:35.508 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-16T16:40:35.508 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.blob_file_starting_level: 0 2023-11-16T16:40:35.508 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-16T16:40:35.508 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-16T16:40:35.508 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-16T16:40:35.508 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e8dba84c-5771-4963-a51b-3d15ee595810 2023-11-16T16:40:35.508 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152835070087, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-16T16:40:35.508 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-16T16:40:35.508 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152835070495, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1700152835, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e8dba84c-5771-4963-a51b-3d15ee595810", "db_session_id": "E552YZ6SFKZFGMGZOPSG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-16T16:40:35.508 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152835070561, "job": 1, "event": "recovery_finished"} 2023-11-16T16:40:35.508 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-16T16:40:35.508 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-16T16:40:35.509 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5592e5fb8000 2023-11-16T16:40:35.509 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: DB pointer 0x5592e5fa4000 2023-11-16T16:40:35.509 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-16T16:40:35.509 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-16T16:40:35.509 INFO:journalctl@ceph.mon.c.smithi159.stdout: ** DB Stats ** 2023-11-16T16:40:35.509 INFO:journalctl@ceph.mon.c.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T16:40:35.509 INFO:journalctl@ceph.mon.c.smithi159.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-16T16:40:35.509 INFO:journalctl@ceph.mon.c.smithi159.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T16:40:35.509 INFO:journalctl@ceph.mon.c.smithi159.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T16:40:35.509 INFO:journalctl@ceph.mon.c.smithi159.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-16T16:40:35.509 INFO:journalctl@ceph.mon.c.smithi159.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T16:40:35.509 INFO:journalctl@ceph.mon.c.smithi159.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T16:40:35.510 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2023-11-16T16:40:35.510 INFO:journalctl@ceph.mon.c.smithi159.stdout: ** Compaction Stats [default] ** 2023-11-16T16:40:35.510 INFO:journalctl@ceph.mon.c.smithi159.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T16:40:35.510 INFO:journalctl@ceph.mon.c.smithi159.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-16T16:40:35.510 INFO:journalctl@ceph.mon.c.smithi159.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T16:40:35.510 INFO:journalctl@ceph.mon.c.smithi159.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T16:40:35.510 INFO:journalctl@ceph.mon.c.smithi159.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T16:40:35.510 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2023-11-16T16:40:35.510 INFO:journalctl@ceph.mon.c.smithi159.stdout: ** Compaction Stats [default] ** 2023-11-16T16:40:35.510 INFO:journalctl@ceph.mon.c.smithi159.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T16:40:35.510 INFO:journalctl@ceph.mon.c.smithi159.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-16T16:40:35.511 INFO:journalctl@ceph.mon.c.smithi159.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T16:40:35.511 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2023-11-16T16:40:35.511 INFO:journalctl@ceph.mon.c.smithi159.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-16T16:40:35.511 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2023-11-16T16:40:35.511 INFO:journalctl@ceph.mon.c.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T16:40:35.511 INFO:journalctl@ceph.mon.c.smithi159.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-16T16:40:35.511 INFO:journalctl@ceph.mon.c.smithi159.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-16T16:40:35.511 INFO:journalctl@ceph.mon.c.smithi159.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-16T16:40:35.511 INFO:journalctl@ceph.mon.c.smithi159.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-16T16:40:35.511 INFO:journalctl@ceph.mon.c.smithi159.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-16T16:40:35.511 INFO:journalctl@ceph.mon.c.smithi159.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T16:40:35.511 INFO:journalctl@ceph.mon.c.smithi159.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout: Block cache BinnedLRUCache@0x5592e5f1b1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: using public_addr v2:172.21.15.159:0/0 -> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: starting mon.c rank -1 at public addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] at bind addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(???) e0 preinit fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).mds e1 new map 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).mds e1 print_map 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout: e1 2023-11-16T16:40:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout: legacy client fscid: -1 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout: No filesystems configured 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:40:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:40:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:40:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:40:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Updating smithi125:/etc/ceph/ceph.conf 2023-11-16T16:40:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Updating smithi125:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Updating smithi125:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:40:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Updating smithi125:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:40:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Deploying cephadm binary to smithi159 2023-11-16T16:40:35.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Added host smithi159 2023-11-16T16:40:35.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:35.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:35.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:35.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:40:35.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:35.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:40:35.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:35.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:35.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:40:35.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Updating smithi159:/etc/ceph/ceph.conf 2023-11-16T16:40:35.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Updating smithi159:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:35.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:35.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Updating smithi159:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:40:35.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/740505949' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-16T16:40:35.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:35.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/740505949' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-16T16:40:35.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:40:35.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:35.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi093:172.21.15.93=a;smithi125:172.21.15.125=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:40:35.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Saving service mon spec with placement smithi093:172.21.15.93=a;smithi125:172.21.15.125=b;smithi159:172.21.15.159=c;count:3 2023-11-16T16:40:35.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:35.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:35.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:35.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:40:35.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:35.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:40:35.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:35.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:35.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: Deploying daemon mon.c on smithi159 2023-11-16T16:40:35.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:35.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-16T16:40:35.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-11-16T16:40:35.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: expand_channel_meta expand map: {default=false} 2023-11-16T16:40:35.519 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: expand_channel_meta from 'false' to 'false' 2023-11-16T16:40:35.519 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: expand_channel_meta expanded map: {default=false} 2023-11-16T16:40:35.519 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: expand_channel_meta expand map: {default=info} 2023-11-16T16:40:35.519 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: expand_channel_meta from 'info' to 'info' 2023-11-16T16:40:35.519 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: expand_channel_meta expanded map: {default=info} 2023-11-16T16:40:35.519 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: expand_channel_meta expand map: {default=daemon} 2023-11-16T16:40:35.519 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-16T16:40:35.519 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: expand_channel_meta expanded map: {default=daemon} 2023-11-16T16:40:35.519 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: expand_channel_meta expand map: {default=debug} 2023-11-16T16:40:35.519 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: expand_channel_meta from 'debug' to 'debug' 2023-11-16T16:40:35.519 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 ceph-mon[131528]: expand_channel_meta expanded map: {default=debug} 2023-11-16T16:40:35.519 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:35 smithi159 systemd[1]: Started Ceph mon.c for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:40:36.121 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-16T16:40:36.121 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph mon dump -f json 2023-11-16T16:40:36.771 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:40:40.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:40.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:40:40.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:40.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: mon.a calling monitor election 2023-11-16T16:40:40.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:40.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:40.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:40.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: mon.c calling monitor election 2023-11-16T16:40:40.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:40.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:40.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:40.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:40.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-16T16:40:40.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: monmap e2: 2 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} 2023-11-16T16:40:40.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: fsmap 2023-11-16T16:40:40.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:40:40.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: mgrmap e13: x(active, since 49s) 2023-11-16T16:40:40.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: overall HEALTH_OK 2023-11-16T16:40:40.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:40.488 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:40 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:40.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:40.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:40:40.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:40.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: mon.a calling monitor election 2023-11-16T16:40:40.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:40.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:40.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:40.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: mon.c calling monitor election 2023-11-16T16:40:40.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:40.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:40.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:40.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:40.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-16T16:40:40.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: monmap e2: 2 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} 2023-11-16T16:40:40.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: fsmap 2023-11-16T16:40:40.653 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:40:40.653 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: mgrmap e13: x(active, since 49s) 2023-11-16T16:40:40.653 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: overall HEALTH_OK 2023-11-16T16:40:40.653 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:40.653 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:40 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:41.001 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.000+0000 7f2a070e5700 1 -- 172.21.15.159:0/2393352473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29f00057d0 msgr2=0x7f29f0007c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:41.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.000+0000 7f2a070e5700 1 --2- 172.21.15.159:0/2393352473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29f00057d0 0x7f29f0007c50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2a00076150 tx=0x7f29f400fe60 comp rx=0 tx=0).stop 2023-11-16T16:40:41.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.000+0000 7f2a070e5700 1 -- 172.21.15.159:0/2393352473 shutdown_connections 2023-11-16T16:40:41.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.000+0000 7f2a070e5700 1 --2- 172.21.15.159:0/2393352473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29f00057d0 0x7f29f0007c50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:41.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.000+0000 7f2a070e5700 1 -- 172.21.15.159:0/2393352473 >> 172.21.15.159:0/2393352473 conn(0x7f2a00074c40 msgr2=0x7f2a00075040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:41.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.000+0000 7f2a070e5700 1 -- 172.21.15.159:0/2393352473 shutdown_connections 2023-11-16T16:40:41.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.000+0000 7f2a070e5700 1 -- 172.21.15.159:0/2393352473 wait complete. 2023-11-16T16:40:41.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.001+0000 7f2a070e5700 1 Processor -- start 2023-11-16T16:40:41.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.001+0000 7f2a070e5700 1 -- start start 2023-11-16T16:40:41.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.001+0000 7f2a070e5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29f00057d0 0x7f2a001930e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:41.005 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.001+0000 7f2a070e5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a00193620 0x7f2a00197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:41.005 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.001+0000 7f2a070e5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a00193ba0 con 0x7f29f00057d0 2023-11-16T16:40:41.005 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.001+0000 7f2a070e5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a00193d10 con 0x7f2a00193620 2023-11-16T16:40:41.005 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.001+0000 7f29fffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a00193620 0x7f2a00197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:41.005 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.001+0000 7f29fffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a00193620 0x7f2a00197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53994/0 (socket says 172.21.15.159:53994) 2023-11-16T16:40:41.005 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.001+0000 7f29fffff700 1 -- 172.21.15.159:0/4147631328 learned_addr learned my addr 172.21.15.159:0/4147631328 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-11-16T16:40:41.006 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.001+0000 7f29fffff700 1 -- 172.21.15.159:0/4147631328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a00193620 msgr2=0x7f2a00197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-11-16T16:40:41.006 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.001+0000 7f29fffff700 1 -- 172.21.15.159:0/4147631328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a00193620 msgr2=0x7f2a00197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-16T16:40:41.006 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.001+0000 7f29fffff700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a00193620 0x7f2a00197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-16T16:40:41.006 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.002+0000 7f29fffff700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a00193620 0x7f2a00197a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:40:41.006 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.002+0000 7f2a04e81700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29f00057d0 0x7f2a001930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:41.006 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.002+0000 7f2a04e81700 1 -- 172.21.15.159:0/4147631328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a00193620 msgr2=0x7f2a00197a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:40:41.006 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.002+0000 7f2a04e81700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a00193620 0x7f2a00197a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:41.007 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.002+0000 7f2a04e81700 1 -- 172.21.15.159:0/4147631328 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f400f8b0 con 0x7f29f00057d0 2023-11-16T16:40:41.007 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.002+0000 7f2a04e81700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29f00057d0 0x7f2a001930e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29f401b000 tx=0x7f29f4013c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:41.007 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.003+0000 7f29fdffb700 1 -- 172.21.15.159:0/4147631328 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f29f4023070 con 0x7f29f00057d0 2023-11-16T16:40:41.007 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.003+0000 7f2a070e5700 1 -- 172.21.15.159:0/4147631328 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a00105bf0 con 0x7f29f00057d0 2023-11-16T16:40:41.008 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.003+0000 7f2a070e5700 1 -- 172.21.15.159:0/4147631328 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a00069120 con 0x7f29f00057d0 2023-11-16T16:40:41.008 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.003+0000 7f29fdffb700 1 -- 172.21.15.159:0/4147631328 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29f4004bd0 con 0x7f29f00057d0 2023-11-16T16:40:41.008 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.003+0000 7f29fdffb700 1 -- 172.21.15.159:0/4147631328 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f29f4003d60 con 0x7f29f00057d0 2023-11-16T16:40:41.008 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.004+0000 7f29fdffb700 1 -- 172.21.15.159:0/4147631328 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f29f4004710 con 0x7f29f00057d0 2023-11-16T16:40:41.009 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.004+0000 7f2a070e5700 1 -- 172.21.15.159:0/4147631328 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a00105d80 con 0x7f29f00057d0 2023-11-16T16:40:41.009 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.004+0000 7f29fdffb700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f29e40386b0 0x7f29e403ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:41.009 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.004+0000 7f29fdffb700 1 -- 172.21.15.159:0/4147631328 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f29f404d640 con 0x7f29f00057d0 2023-11-16T16:40:41.009 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.005+0000 7f29fffff700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f29e40386b0 0x7f29e403ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:41.009 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.005+0000 7f29fffff700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f29e40386b0 0x7f29e403ab60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29ec009180 tx=0x7f29ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:41.012 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.010+0000 7f29fdffb700 1 -- 172.21.15.159:0/4147631328 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f29f4005400 con 0x7f29f00057d0 2023-11-16T16:40:41.280 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.279+0000 7f2a070e5700 1 -- 172.21.15.159:0/4147631328 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2a001945c0 con 0x7f29f00057d0 2023-11-16T16:40:41.281 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.280+0000 7f29fdffb700 1 -- 172.21.15.159:0/4147631328 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f29f4030810 con 0x7f29f00057d0 2023-11-16T16:40:41.281 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:40:41.282 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":2,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","modified":"2023-11-16T16:40:35.095346Z","created":"2023-11-16T16:38:51.851994Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:3300","nonce":0},{"type":"v1","addr":"172.21.15.93:6789","nonce":0}]},"addr":"172.21.15.93:6789/0","public_addr":"172.21.15.93:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-11-16T16:40:41.284 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.282+0000 7f2a070e5700 1 -- 172.21.15.159:0/4147631328 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f29e40386b0 msgr2=0x7f29e403ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:41.284 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.282+0000 7f2a070e5700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f29e40386b0 0x7f29e403ab60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29ec009180 tx=0x7f29ec008040 comp rx=0 tx=0).stop 2023-11-16T16:40:41.284 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.282+0000 7f2a070e5700 1 -- 172.21.15.159:0/4147631328 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29f00057d0 msgr2=0x7f2a001930e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:41.284 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.282+0000 7f2a070e5700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29f00057d0 0x7f2a001930e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29f401b000 tx=0x7f29f4013c40 comp rx=0 tx=0).stop 2023-11-16T16:40:41.284 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.282+0000 7f2a070e5700 1 -- 172.21.15.159:0/4147631328 shutdown_connections 2023-11-16T16:40:41.284 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.282+0000 7f2a070e5700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f29e40386b0 0x7f29e403ab60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:41.284 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.282+0000 7f2a070e5700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29f00057d0 0x7f2a001930e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:41.284 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.282+0000 7f2a070e5700 1 --2- 172.21.15.159:0/4147631328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a00193620 0x7f2a00197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:41.284 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.282+0000 7f2a070e5700 1 -- 172.21.15.159:0/4147631328 >> 172.21.15.159:0/4147631328 conn(0x7f2a00074c40 msgr2=0x7f2a0018bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:41.285 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.282+0000 7f2a070e5700 1 -- 172.21.15.159:0/4147631328 shutdown_connections 2023-11-16T16:40:41.285 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:41.282+0000 7f2a070e5700 1 -- 172.21.15.159:0/4147631328 wait complete. 2023-11-16T16:40:41.285 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 2 2023-11-16T16:40:41.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:41 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:41.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:41 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:41.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:41 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:41.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:41 smithi159 ceph-mon[131528]: Deploying daemon mon.b on smithi125 2023-11-16T16:40:41.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:41 smithi159 ceph-mon[131528]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:41.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:41 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:41.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:41 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:41.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:41 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:41.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:41 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:41.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:41 smithi093 ceph-mon[97863]: Deploying daemon mon.b on smithi125 2023-11-16T16:40:41.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:41 smithi093 ceph-mon[97863]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:41.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:41 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:42.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:42 smithi093 ceph-mon[97863]: from='client.? 172.21.15.159:0/4147631328' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T16:40:42.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:40:42 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:40:42.095+0000 7f18caae3700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-16T16:40:42.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:42 smithi159 ceph-mon[131528]: from='client.? 172.21.15.159:0/4147631328' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T16:40:42.720 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-16T16:40:42.721 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph mon dump -f json 2023-11-16T16:40:43.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:42 smithi125 systemd[1]: Starting Ceph mon.b for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:40:43.366 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:40:43.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 podman[131036]: 2023-11-16T16:40:43.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: set uid:gid to 167:167 (ceph:ceph) 2023-11-16T16:40:43.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev), process ceph-mon, pid 7 2023-11-16T16:40:43.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pidfile_write: ignore empty --pid-file 2023-11-16T16:40:43.389 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: load: jerasure load: lrc 2023-11-16T16:40:43.389 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: RocksDB version: 7.9.2 2023-11-16T16:40:43.389 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Git sha 0 2023-11-16T16:40:43.389 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Compile date 2023-11-15 23:53:54 2023-11-16T16:40:43.389 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: DB SUMMARY 2023-11-16T16:40:43.389 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: DB Session ID: OM41VQOPM04CF5NXSCM4 2023-11-16T16:40:43.390 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: CURRENT file: CURRENT 2023-11-16T16:40:43.390 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: IDENTITY file: IDENTITY 2023-11-16T16:40:43.390 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-16T16:40:43.390 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-11-16T16:40:43.390 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-11-16T16:40:43.390 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.error_if_exists: 0 2023-11-16T16:40:43.390 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.create_if_missing: 0 2023-11-16T16:40:43.391 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.paranoid_checks: 1 2023-11-16T16:40:43.391 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-16T16:40:43.391 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-16T16:40:43.391 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-16T16:40:43.391 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.env: 0x55da30f2f720 2023-11-16T16:40:43.391 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.fs: PosixFileSystem 2023-11-16T16:40:43.391 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.info_log: 0x55da33a833c0 2023-11-16T16:40:43.391 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_file_opening_threads: 16 2023-11-16T16:40:43.391 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.statistics: (nil) 2023-11-16T16:40:43.392 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.use_fsync: 0 2023-11-16T16:40:43.392 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_log_file_size: 0 2023-11-16T16:40:43.392 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-16T16:40:43.392 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-16T16:40:43.392 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.keep_log_file_num: 1000 2023-11-16T16:40:43.392 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.recycle_log_file_num: 0 2023-11-16T16:40:43.392 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.allow_fallocate: 1 2023-11-16T16:40:43.392 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.allow_mmap_reads: 0 2023-11-16T16:40:43.392 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.allow_mmap_writes: 0 2023-11-16T16:40:43.393 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.use_direct_reads: 0 2023-11-16T16:40:43.393 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-16T16:40:43.393 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.create_missing_column_families: 0 2023-11-16T16:40:43.393 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.db_log_dir: 2023-11-16T16:40:43.393 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.wal_dir: 2023-11-16T16:40:43.393 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-16T16:40:43.393 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-16T16:40:43.393 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-16T16:40:43.394 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-16T16:40:43.394 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-16T16:40:43.394 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-16T16:40:43.394 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.advise_random_on_open: 1 2023-11-16T16:40:43.394 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.db_write_buffer_size: 0 2023-11-16T16:40:43.394 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.write_buffer_manager: 0x55da32cf43c0 2023-11-16T16:40:43.394 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-16T16:40:43.394 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-16T16:40:43.395 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-16T16:40:43.395 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.rate_limiter: (nil) 2023-11-16T16:40:43.395 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-16T16:40:43.395 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.wal_recovery_mode: 2 2023-11-16T16:40:43.395 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.enable_thread_tracking: 0 2023-11-16T16:40:43.395 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.enable_pipelined_write: 0 2023-11-16T16:40:43.395 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.unordered_write: 0 2023-11-16T16:40:43.395 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-16T16:40:43.395 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-16T16:40:43.396 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-16T16:40:43.396 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-16T16:40:43.396 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.row_cache: None 2023-11-16T16:40:43.396 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.wal_filter: None 2023-11-16T16:40:43.396 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-16T16:40:43.396 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.allow_ingest_behind: 0 2023-11-16T16:40:43.396 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.two_write_queues: 0 2023-11-16T16:40:43.397 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.manual_wal_flush: 0 2023-11-16T16:40:43.397 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.wal_compression: 0 2023-11-16T16:40:43.397 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.atomic_flush: 0 2023-11-16T16:40:43.397 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-16T16:40:43.397 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-16T16:40:43.397 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-16T16:40:43.397 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.log_readahead_size: 0 2023-11-16T16:40:43.397 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-16T16:40:43.398 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.best_efforts_recovery: 0 2023-11-16T16:40:43.398 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-16T16:40:43.398 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-16T16:40:43.398 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.allow_data_in_errors: 0 2023-11-16T16:40:43.398 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.db_host_id: __hostname__ 2023-11-16T16:40:43.399 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-16T16:40:43.400 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_background_jobs: 2 2023-11-16T16:40:43.400 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_background_compactions: -1 2023-11-16T16:40:43.400 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_subcompactions: 1 2023-11-16T16:40:43.400 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-16T16:40:43.400 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-16T16:40:43.400 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-16T16:40:43.400 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_total_wal_size: 0 2023-11-16T16:40:43.400 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-16T16:40:43.400 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-16T16:40:43.400 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-16T16:40:43.400 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-16T16:40:43.400 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_open_files: -1 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bytes_per_sync: 0 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_readahead_size: 0 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_background_flushes: -1 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Compression algorithms supported: 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: kZSTD supported: 0 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: kXpressCompression supported: 0 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: kLZ4HCCompression supported: 1 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: kZlibCompression supported: 1 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: kSnappyCompression supported: 1 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: kLZ4Compression supported: 1 2023-11-16T16:40:43.401 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: kBZip2Compression supported: 0 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.merge_operator: 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_filter: None 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_filter_factory: None 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.sst_partitioner_factory: None 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55da33a834e0) 2023-11-16T16:40:43.402 INFO:journalctl@ceph.mon.b.smithi125.stdout: cache_index_and_filter_blocks: 1 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: pin_top_level_index_and_filter: 1 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: index_type: 0 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: data_block_index_type: 0 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: index_shortening: 1 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: checksum: 4 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: no_block_cache: 0 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: block_cache: 0x55da32d951f0 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: block_cache_name: BinnedLRUCache 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: block_cache_options: 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: capacity : 536870912 2023-11-16T16:40:43.403 INFO:journalctl@ceph.mon.b.smithi125.stdout: num_shard_bits : 4 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: strict_capacity_limit : 0 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: high_pri_pool_ratio: 0.000 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: block_cache_compressed: (nil) 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: persistent_cache: (nil) 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: block_size: 4096 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: block_size_deviation: 10 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: block_restart_interval: 16 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: index_block_restart_interval: 1 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: metadata_block_size: 4096 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: partition_filters: 0 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: use_delta_encoding: 1 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: filter_policy: bloomfilter 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: whole_key_filtering: 1 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: verify_compression: 0 2023-11-16T16:40:43.404 INFO:journalctl@ceph.mon.b.smithi125.stdout: read_amp_bytes_per_bit: 0 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout: format_version: 5 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout: enable_index_compression: 1 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout: block_align: 0 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout: max_auto_readahead_size: 262144 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout: prepopulate_block_cache: 0 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout: initial_auto_readahead_size: 8192 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout: num_file_reads_for_auto_readahead: 2 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.write_buffer_size: 33554432 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_write_buffer_number: 2 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compression: NoCompression 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bottommost_compression: Disabled 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.prefix_extractor: nullptr 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.num_levels: 7 2023-11-16T16:40:43.405 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-16T16:40:43.406 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compression_opts.level: 32767 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compression_opts.strategy: 0 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compression_opts.enabled: false 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.target_file_size_base: 67108864 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-16T16:40:43.407 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.arena_block_size: 1048576 2023-11-16T16:40:43.408 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.disable_auto_compactions: 0 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-16T16:40:43.409 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.table_properties_collectors: 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.inplace_update_support: 0 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.bloom_locality: 0 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.max_successive_merges: 0 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.paranoid_file_checks: 0 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.force_consistency_checks: 1 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.report_bg_io_stats: 0 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.ttl: 2592000 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-16T16:40:43.410 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.enable_blob_files: false 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.min_blob_size: 0 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.blob_file_size: 268435456 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.blob_file_starting_level: 0 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1a840f4e-041e-4ea0-b422-d38e3cd37cf4 2023-11-16T16:40:43.411 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152843293286, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152843293698, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1700152843, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1a840f4e-041e-4ea0-b422-d38e3cd37cf4", "db_session_id": "OM41VQOPM04CF5NXSCM4", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152843293764, "job": 1, "event": "recovery_finished"} 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55da32e32000 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: DB pointer 0x55da32e1e000 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout: ** DB Stats ** 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T16:40:43.412 INFO:journalctl@ceph.mon.b.smithi125.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: ** Compaction Stats [default] ** 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: ** Compaction Stats [default] ** 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T16:40:43.413 INFO:journalctl@ceph.mon.b.smithi125.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T16:40:43.414 INFO:journalctl@ceph.mon.b.smithi125.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout: Block cache BinnedLRUCache@0x55da32d951f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout: 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: using public_addr v2:172.21.15.125:0/0 -> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: starting mon.b rank -1 at public addrs [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] at bind addrs [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(???) e0 preinit fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).mds e1 new map 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).mds e1 print_map 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout: e1 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T16:40:43.415 INFO:journalctl@ceph.mon.b.smithi125.stdout: legacy client fscid: -1 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout: 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout: No filesystems configured 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:40:43.416 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Updating smithi125:/etc/ceph/ceph.conf 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Updating smithi125:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Updating smithi125:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Updating smithi125:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Deploying cephadm binary to smithi159 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Added host smithi159 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:43.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Updating smithi159:/etc/ceph/ceph.conf 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Updating smithi159:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Updating smithi159:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/740505949' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/740505949' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi093:172.21.15.93=a;smithi125:172.21.15.125=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:40:43.419 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Saving service mon spec with placement smithi093:172.21.15.93=a;smithi125:172.21.15.125=b;smithi159:172.21.15.159=c;count:3 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Deploying daemon mon.c on smithi159 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:40:43.420 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.a calling monitor election 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.c calling monitor election 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: monmap e2: 2 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: fsmap 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:40:43.421 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mgrmap e13: x(active, since 49s) 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: overall HEALTH_OK 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: Deploying daemon mon.b on smithi125 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: from='client.? 172.21.15.159:0/4147631328' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 ceph-mon[131076]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-11-16T16:40:43.422 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 bash[131036]: f3eb6aa64d25c56857b805007266651dab14c53c7af6660c9d56f43b47e28feb 2023-11-16T16:40:43.423 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:43 smithi125 systemd[1]: Started Ceph mon.b for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:40:43.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:43 smithi159 ceph-mon[131528]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:43.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:43 smithi093 ceph-mon[97863]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:48.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:40:48.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: mon.a calling monitor election 2023-11-16T16:40:48.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:48.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: mon.c calling monitor election 2023-11-16T16:40:48.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:48.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: mon.b calling monitor election 2023-11-16T16:40:48.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:48.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-16T16:40:48.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: monmap e3: 3 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],b=[v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} 2023-11-16T16:40:48.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: fsmap 2023-11-16T16:40:48.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:40:48.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: mgrmap e13: x(active, since 57s) 2023-11-16T16:40:48.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: overall HEALTH_OK 2023-11-16T16:40:48.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:48.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:48.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:40:48.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: mon.a calling monitor election 2023-11-16T16:40:48.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:48.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: mon.c calling monitor election 2023-11-16T16:40:48.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:48.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: mon.b calling monitor election 2023-11-16T16:40:48.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:48.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.831 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.831 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-16T16:40:48.831 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: monmap e3: 3 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],b=[v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} 2023-11-16T16:40:48.831 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: fsmap 2023-11-16T16:40:48.831 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:40:48.831 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: mgrmap e13: x(active, since 57s) 2023-11-16T16:40:48.831 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: overall HEALTH_OK 2023-11-16T16:40:48.831 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:48.831 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:48.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:40:48.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: mon.a calling monitor election 2023-11-16T16:40:48.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:40:48.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: mon.c calling monitor election 2023-11-16T16:40:48.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:48.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: mon.b calling monitor election 2023-11-16T16:40:48.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:48.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:48.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-16T16:40:48.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: monmap e3: 3 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],b=[v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} 2023-11-16T16:40:48.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: fsmap 2023-11-16T16:40:48.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:40:48.903 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: mgrmap e13: x(active, since 57s) 2023-11-16T16:40:48.903 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: overall HEALTH_OK 2023-11-16T16:40:48.903 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:48.903 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:48.997 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.996+0000 7fc009fa0700 1 -- 172.21.15.159:0/3750930872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbfec0057d0 msgr2=0x7fbfec007c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:48.997 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.996+0000 7fc009fa0700 1 --2- 172.21.15.159:0/3750930872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbfec0057d0 0x7fbfec007c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc004075f20 tx=0x7fbff400fe10 comp rx=0 tx=0).stop 2023-11-16T16:40:48.997 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.996+0000 7fc009fa0700 1 -- 172.21.15.159:0/3750930872 shutdown_connections 2023-11-16T16:40:48.998 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.996+0000 7fc009fa0700 1 --2- 172.21.15.159:0/3750930872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbfec0057d0 0x7fbfec007c50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:48.998 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.996+0000 7fc009fa0700 1 -- 172.21.15.159:0/3750930872 >> 172.21.15.159:0/3750930872 conn(0x7fc004074c40 msgr2=0x7fc004075040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:48.998 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.996+0000 7fc009fa0700 1 -- 172.21.15.159:0/3750930872 shutdown_connections 2023-11-16T16:40:48.998 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.996+0000 7fc009fa0700 1 -- 172.21.15.159:0/3750930872 wait complete. 2023-11-16T16:40:49.001 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc009fa0700 1 Processor -- start 2023-11-16T16:40:49.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc009fa0700 1 -- start start 2023-11-16T16:40:49.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc009fa0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfec0057d0 0x7fc0041932e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:49.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc009fa0700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc004193820 0x7fc004197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:49.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc009fa0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0041981d0 0x7fc004198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:49.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc009fa0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc004193ef0 con 0x7fc0041981d0 2023-11-16T16:40:49.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc009fa0700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00419ff40 con 0x7fc004193820 2023-11-16T16:40:49.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc009fa0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0041a0080 con 0x7fbfec0057d0 2023-11-16T16:40:49.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc0037fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfec0057d0 0x7fc0041932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:49.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc0037fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfec0057d0 0x7fc0041932e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54020/0 (socket says 172.21.15.159:54020) 2023-11-16T16:40:49.002 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc0037fe700 1 -- 172.21.15.159:0/3345019943 learned_addr learned my addr 172.21.15.159:0/3345019943 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-11-16T16:40:49.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc0037fe700 1 -- 172.21.15.159:0/3345019943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfec0057d0 msgr2=0x7fc0041932e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-11-16T16:40:49.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc0037fe700 1 -- 172.21.15.159:0/3345019943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfec0057d0 msgr2=0x7fc0041932e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-16T16:40:49.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc0037fe700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfec0057d0 0x7fc0041932e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-16T16:40:49.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.997+0000 7fc0037fe700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfec0057d0 0x7fc0041932e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:40:49.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc002ffd700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc004193820 0x7fc004197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:49.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc002ffd700 1 -- 172.21.15.159:0/3345019943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc004193820 msgr2=0x7fc004197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-11-16T16:40:49.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc002ffd700 1 -- 172.21.15.159:0/3345019943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc004193820 msgr2=0x7fc004197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-16T16:40:49.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc002ffd700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc004193820 0x7fc004197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-16T16:40:49.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc003fff700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0041981d0 0x7fc004198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:49.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc002ffd700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc004193820 0x7fc004197c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:40:49.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc003fff700 1 -- 172.21.15.159:0/3345019943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfec0057d0 msgr2=0x7fc0041932e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:40:49.003 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc003fff700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfec0057d0 0x7fc0041932e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:49.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc003fff700 1 -- 172.21.15.159:0/3345019943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc004193820 msgr2=0x7fc004197c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:40:49.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc003fff700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc004193820 0x7fc004197c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:49.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc003fff700 1 -- 172.21.15.159:0/3345019943 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff400f860 con 0x7fc0041981d0 2023-11-16T16:40:49.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc003fff700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0041981d0 0x7fc004198670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00be30 tx=0x7fbffc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:49.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.998+0000 7fc000ff9700 1 -- 172.21.15.159:0/3345019943 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc00cc90 con 0x7fc0041981d0 2023-11-16T16:40:49.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.999+0000 7fc000ff9700 1 -- 172.21.15.159:0/3345019943 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbffc00cdf0 con 0x7fc0041981d0 2023-11-16T16:40:49.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.999+0000 7fc009fa0700 1 -- 172.21.15.159:0/3345019943 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc004105bf0 con 0x7fc0041981d0 2023-11-16T16:40:49.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.999+0000 7fc000ff9700 1 -- 172.21.15.159:0/3345019943 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0127e0 con 0x7fc0041981d0 2023-11-16T16:40:49.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.999+0000 7fc009fa0700 1 -- 172.21.15.159:0/3345019943 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0040690c0 con 0x7fc0041981d0 2023-11-16T16:40:49.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:48.999+0000 7fc000ff9700 1 -- 172.21.15.159:0/3345019943 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fbffc012960 con 0x7fc0041981d0 2023-11-16T16:40:49.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.000+0000 7fc009fa0700 1 -- 172.21.15.159:0/3345019943 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc004105d80 con 0x7fc0041981d0 2023-11-16T16:40:49.004 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.000+0000 7fc000ff9700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbff0038d00 0x7fbff003b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:49.005 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.000+0000 7fc000ff9700 1 -- 172.21.15.159:0/3345019943 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbffc04ada0 con 0x7fc0041981d0 2023-11-16T16:40:49.005 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.000+0000 7fc0037fe700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbff0038d00 0x7fbff003b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:49.005 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.001+0000 7fc0037fe700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbff0038d00 0x7fbff003b1b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbff4010010 tx=0x7fbff4001e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:49.006 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.005+0000 7fc000ff9700 1 -- 172.21.15.159:0/3345019943 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbffc01d3b0 con 0x7fc0041981d0 2023-11-16T16:40:49.278 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.277+0000 7fc009fa0700 1 -- 172.21.15.159:0/3345019943 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc0041a0b60 con 0x7fc0041981d0 2023-11-16T16:40:49.279 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.277+0000 7fc000ff9700 1 -- 172.21.15.159:0/3345019943 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1276 (secure 0 0 0) 0x7fbffc019030 con 0x7fc0041981d0 2023-11-16T16:40:49.279 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-11-16T16:40:49.279 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":3,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","modified":"2023-11-16T16:40:43.317576Z","created":"2023-11-16T16:38:51.851994Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:3300","nonce":0},{"type":"v1","addr":"172.21.15.93:6789","nonce":0}]},"addr":"172.21.15.93:6789/0","public_addr":"172.21.15.93:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:3300","nonce":0},{"type":"v1","addr":"172.21.15.125:6789","nonce":0}]},"addr":"172.21.15.125:6789/0","public_addr":"172.21.15.125:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-11-16T16:40:49.281 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.280+0000 7fc009fa0700 1 -- 172.21.15.159:0/3345019943 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbff0038d00 msgr2=0x7fbff003b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:49.282 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.280+0000 7fc009fa0700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbff0038d00 0x7fbff003b1b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbff4010010 tx=0x7fbff4001e50 comp rx=0 tx=0).stop 2023-11-16T16:40:49.282 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.280+0000 7fc009fa0700 1 -- 172.21.15.159:0/3345019943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0041981d0 msgr2=0x7fc004198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:49.282 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.280+0000 7fc009fa0700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0041981d0 0x7fc004198670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00be30 tx=0x7fbffc00bf10 comp rx=0 tx=0).stop 2023-11-16T16:40:49.282 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.280+0000 7fc009fa0700 1 -- 172.21.15.159:0/3345019943 shutdown_connections 2023-11-16T16:40:49.282 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.280+0000 7fc009fa0700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbff0038d00 0x7fbff003b1b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:49.282 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.280+0000 7fc009fa0700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0041981d0 0x7fc004198670 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:49.282 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.280+0000 7fc009fa0700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfec0057d0 0x7fc0041932e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:49.282 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.280+0000 7fc009fa0700 1 --2- 172.21.15.159:0/3345019943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc004193820 0x7fc004197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:49.282 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.280+0000 7fc009fa0700 1 -- 172.21.15.159:0/3345019943 >> 172.21.15.159:0/3345019943 conn(0x7fc004074c40 msgr2=0x7fc004100940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:49.282 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.280+0000 7fc009fa0700 1 -- 172.21.15.159:0/3345019943 shutdown_connections 2023-11-16T16:40:49.283 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:49.280+0000 7fc009fa0700 1 -- 172.21.15.159:0/3345019943 wait complete. 2023-11-16T16:40:49.283 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 3 2023-11-16T16:40:49.732 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:49 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:49.732 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:49 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:49.732 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:49 smithi159 ceph-mon[131528]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:49.732 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:49 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:49.732 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:49 smithi159 ceph-mon[131528]: from='client.? 172.21.15.159:0/3345019943' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T16:40:49.732 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:49 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:49.734 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-11-16T16:40:49.734 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph config generate-minimal-conf 2023-11-16T16:40:49.805 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:49 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:49.805 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:49 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:49.805 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:49 smithi093 ceph-mon[97863]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:49.805 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:49 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:49.805 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:49 smithi093 ceph-mon[97863]: from='client.? 172.21.15.159:0/3345019943' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T16:40:49.805 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:49 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:49.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:49 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:49.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:49 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:49.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:49 smithi125 ceph-mon[131076]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:49.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:49 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:49.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:49 smithi125 ceph-mon[131076]: from='client.? 172.21.15.159:0/3345019943' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T16:40:49.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:49 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:40:50.602 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:40:50.650 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:40:50 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:40:50.317+0000 7f18caae3700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-16T16:40:51.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:51.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:51.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.246 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:51.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:51.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:51.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.117+0000 7f5771259700 1 -- 172.21.15.93:0/3307844721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f576c0fff90 msgr2=0x7f576c100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.117+0000 7f5771259700 1 --2- 172.21.15.93:0/3307844721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f576c0fff90 0x7f576c100360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f575c009910 tx=0x7f575c009c20 comp rx=0 tx=0).stop 2023-11-16T16:40:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.118+0000 7f5771259700 1 -- 172.21.15.93:0/3307844721 shutdown_connections 2023-11-16T16:40:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.118+0000 7f5771259700 1 --2- 172.21.15.93:0/3307844721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f576c0fff90 0x7f576c100360 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.118+0000 7f5771259700 1 -- 172.21.15.93:0/3307844721 >> 172.21.15.93:0/3307844721 conn(0x7f576c0fb870 msgr2=0x7f576c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.118+0000 7f5771259700 1 -- 172.21.15.93:0/3307844721 shutdown_connections 2023-11-16T16:40:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.118+0000 7f5771259700 1 -- 172.21.15.93:0/3307844721 wait complete. 2023-11-16T16:40:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.119+0000 7f5771259700 1 Processor -- start 2023-11-16T16:40:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.119+0000 7f5771259700 1 -- start start 2023-11-16T16:40:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.119+0000 7f5771259700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f576c0fff90 0x7f576c197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.119+0000 7f5771259700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f576c197a40 0x7f576c19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.119+0000 7f5771259700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f576c19c3f0 0x7f576c19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.119+0000 7f5771259700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c198110 con 0x7f576c197a40 2023-11-16T16:40:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.119+0000 7f5771259700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c1a4160 con 0x7f576c19c3f0 2023-11-16T16:40:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.119+0000 7f5771259700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c1a42a0 con 0x7f576c0fff90 2023-11-16T16:40:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.120+0000 7f576a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f576c197a40 0x7f576c19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.120+0000 7f576a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f576c197a40 0x7f576c19beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:52078/0 (socket says 172.21.15.93:52078) 2023-11-16T16:40:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.120+0000 7f576a59c700 1 -- 172.21.15.93:0/3969118785 learned_addr learned my addr 172.21.15.93:0/3969118785 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:40:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.120+0000 7f576ad9d700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f576c0fff90 0x7f576c197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.120+0000 7f576b59e700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f576c19c3f0 0x7f576c19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.120+0000 7f576a59c700 1 -- 172.21.15.93:0/3969118785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f576c0fff90 msgr2=0x7f576c197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.120+0000 7f576a59c700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f576c0fff90 0x7f576c197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.120+0000 7f576a59c700 1 -- 172.21.15.93:0/3969118785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f576c19c3f0 msgr2=0x7f576c19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.120+0000 7f576a59c700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f576c19c3f0 0x7f576c19c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.120+0000 7f576a59c700 1 -- 172.21.15.93:0/3969118785 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c0095f0 con 0x7f576c197a40 2023-11-16T16:40:52.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.120+0000 7f576a59c700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f576c197a40 0x7f576c19beb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f576000b7b0 tx=0x7f576000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:52.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.121+0000 7f5753fff700 1 -- 172.21.15.93:0/3969118785 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760004230 con 0x7f576c197a40 2023-11-16T16:40:52.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.121+0000 7f5753fff700 1 -- 172.21.15.93:0/3969118785 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5760004390 con 0x7f576c197a40 2023-11-16T16:40:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.121+0000 7f5753fff700 1 -- 172.21.15.93:0/3969118785 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760011af0 con 0x7f576c197a40 2023-11-16T16:40:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.121+0000 7f5771259700 1 -- 172.21.15.93:0/3969118785 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576c10a170 con 0x7f576c197a40 2023-11-16T16:40:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.121+0000 7f5771259700 1 -- 172.21.15.93:0/3969118785 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576c109fe0 con 0x7f576c197a40 2023-11-16T16:40:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.122+0000 7f5753fff700 1 -- 172.21.15.93:0/3969118785 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f5760011c50 con 0x7f576c197a40 2023-11-16T16:40:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.122+0000 7f5771259700 1 -- 172.21.15.93:0/3969118785 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576c10a300 con 0x7f576c197a40 2023-11-16T16:40:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.122+0000 7f5753fff700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5754038d40 0x7f575403b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.122+0000 7f5753fff700 1 -- 172.21.15.93:0/3969118785 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f576004ac90 con 0x7f576c197a40 2023-11-16T16:40:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.123+0000 7f576ad9d700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5754038d40 0x7f575403b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.123+0000 7f576ad9d700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5754038d40 0x7f575403b1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f575c009fd0 tx=0x7f575c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:52.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.128+0000 7f5753fff700 1 -- 172.21.15.93:0/3969118785 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f576001a400 con 0x7f576c197a40 2023-11-16T16:40:52.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:40:52.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: Updating smithi093:/etc/ceph/ceph.conf 2023-11-16T16:40:52.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: Updating smithi125:/etc/ceph/ceph.conf 2023-11-16T16:40:52.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: Updating smithi159:/etc/ceph/ceph.conf 2023-11-16T16:40:52.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:52.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: Updating smithi125:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:52.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: Updating smithi159:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:52.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: Updating smithi093:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:52.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:52.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:52.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:52.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:52.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:40:52.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:52.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:40:52.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: Updating smithi093:/etc/ceph/ceph.conf 2023-11-16T16:40:52.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: Updating smithi125:/etc/ceph/ceph.conf 2023-11-16T16:40:52.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: Updating smithi159:/etc/ceph/ceph.conf 2023-11-16T16:40:52.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:52.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: Updating smithi125:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:52.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: Updating smithi159:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:52.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: Updating smithi093:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:52.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:52.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:52.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:52.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:52.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:40:52.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:52.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.333+0000 7f5771259700 1 -- 172.21.15.93:0/3969118785 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f576c062120 con 0x7f576c197a40 2023-11-16T16:40:52.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.333+0000 7f5753fff700 1 -- 172.21.15.93:0/3969118785 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f576000f2e0 con 0x7f576c197a40 2023-11-16T16:40:52.335 INFO:teuthology.orchestra.run.smithi093.stdout:# minimal ceph.conf for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:40:52.335 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-11-16T16:40:52.335 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:40:52.335 INFO:teuthology.orchestra.run.smithi093.stdout: mon_host = [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] 2023-11-16T16:40:52.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.335+0000 7f5771259700 1 -- 172.21.15.93:0/3969118785 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5754038d40 msgr2=0x7f575403b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.335+0000 7f5771259700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5754038d40 0x7f575403b1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f575c009fd0 tx=0x7f575c0058e0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.335+0000 7f5771259700 1 -- 172.21.15.93:0/3969118785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f576c197a40 msgr2=0x7f576c19beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.335+0000 7f5771259700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f576c197a40 0x7f576c19beb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f576000b7b0 tx=0x7f576000bac0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.336+0000 7f5771259700 1 -- 172.21.15.93:0/3969118785 shutdown_connections 2023-11-16T16:40:52.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.336+0000 7f5771259700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5754038d40 0x7f575403b1f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.336+0000 7f5771259700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f576c19c3f0 0x7f576c19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.336+0000 7f5771259700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f576c0fff90 0x7f576c197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.336+0000 7f5771259700 1 --2- 172.21.15.93:0/3969118785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f576c197a40 0x7f576c19beb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.336+0000 7f5771259700 1 -- 172.21.15.93:0/3969118785 >> 172.21.15.93:0/3969118785 conn(0x7f576c0fb870 msgr2=0x7f576c1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:52.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.336+0000 7f5771259700 1 -- 172.21.15.93:0/3969118785 shutdown_connections 2023-11-16T16:40:52.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:40:52.336+0000 7f5771259700 1 -- 172.21.15.93:0/3969118785 wait complete. 2023-11-16T16:40:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:40:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: Updating smithi093:/etc/ceph/ceph.conf 2023-11-16T16:40:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: Updating smithi125:/etc/ceph/ceph.conf 2023-11-16T16:40:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: Updating smithi159:/etc/ceph/ceph.conf 2023-11-16T16:40:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: Updating smithi125:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: Updating smithi159:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: Updating smithi093:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:40:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:52.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:52.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:52.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:52.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:40:52.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:52.995 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-11-16T16:40:52.995 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:40:52.995 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.conf 2023-11-16T16:40:53.072 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:40:53.072 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:53.144 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:53 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:40:53.145 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:53 smithi093 ceph-mon[97863]: Reconfiguring mon.a (unknown last config time)... 2023-11-16T16:40:53.145 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:53 smithi093 ceph-mon[97863]: Reconfiguring daemon mon.a on smithi093 2023-11-16T16:40:53.145 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:53 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3969118785' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:53.182 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-11-16T16:40:53.182 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/ceph/ceph.conf 2023-11-16T16:40:53.220 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-11-16T16:40:53.220 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:53.293 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-11-16T16:40:53.293 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.conf 2023-11-16T16:40:53.324 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:53 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:40:53.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:53 smithi159 ceph-mon[131528]: Reconfiguring mon.a (unknown last config time)... 2023-11-16T16:40:53.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:53 smithi159 ceph-mon[131528]: Reconfiguring daemon mon.a on smithi093 2023-11-16T16:40:53.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:53 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3969118785' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:53.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:53 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:40:53.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:53 smithi125 ceph-mon[131076]: Reconfiguring mon.a (unknown last config time)... 2023-11-16T16:40:53.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:53 smithi125 ceph-mon[131076]: Reconfiguring daemon mon.a on smithi093 2023-11-16T16:40:53.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:53 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3969118785' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:53.330 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-11-16T16:40:53.330 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:40:53.401 INFO:tasks.cephadm:Adding mgr.x on smithi093 2023-11-16T16:40:53.401 INFO:tasks.cephadm:Adding mgr.y on smithi125 2023-11-16T16:40:53.401 INFO:tasks.cephadm:Adding mgr.z on smithi159 2023-11-16T16:40:53.401 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch apply mgr '3;smithi093=x;smithi125=y;smithi159=z' 2023-11-16T16:40:54.079 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:40:54.307 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:54 smithi159 ceph-mon[131528]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:54.308 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:54 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:54.308 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:54 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:54.308 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:54 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:54.308 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:54 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:40:54.308 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:54 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:54.310 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:54 smithi125 ceph-mon[131076]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:54.310 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:54 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:54.310 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:54 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:54.311 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:54 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:54.311 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:54 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:40:54.311 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:54 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:54.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:54 smithi093 ceph-mon[97863]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:54.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:54 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:54.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:54 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:54.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:54 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:54.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:54 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:40:54.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:54 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:55.114 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:55 smithi159 ceph-mon[131528]: Reconfiguring mon.b (monmap changed)... 2023-11-16T16:40:55.115 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:55 smithi159 ceph-mon[131528]: Reconfiguring daemon mon.b on smithi125 2023-11-16T16:40:55.115 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:55.115 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:55.115 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:55.115 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:40:55.115 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:55.287 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.284+0000 7f76516fe700 1 -- 172.21.15.159:0/771626588 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f764c0ffe50 msgr2=0x7f764c100260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:55.287 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.284+0000 7f76516fe700 1 --2- 172.21.15.159:0/771626588 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f764c0ffe50 0x7f764c100260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7640009910 tx=0x7f7640009c20 comp rx=0 tx=0).stop 2023-11-16T16:40:55.287 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.284+0000 7f76516fe700 1 -- 172.21.15.159:0/771626588 shutdown_connections 2023-11-16T16:40:55.288 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.284+0000 7f76516fe700 1 --2- 172.21.15.159:0/771626588 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f764c0ffe50 0x7f764c100260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.288 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.284+0000 7f76516fe700 1 -- 172.21.15.159:0/771626588 >> 172.21.15.159:0/771626588 conn(0x7f764c0fb420 msgr2=0x7f764c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:55.288 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.285+0000 7f76516fe700 1 -- 172.21.15.159:0/771626588 shutdown_connections 2023-11-16T16:40:55.288 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.285+0000 7f76516fe700 1 -- 172.21.15.159:0/771626588 wait complete. 2023-11-16T16:40:55.291 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.285+0000 7f76516fe700 1 Processor -- start 2023-11-16T16:40:55.291 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f76516fe700 1 -- start start 2023-11-16T16:40:55.291 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f76516fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f764c0ffe50 0x7f764c195090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:55.291 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f76516fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f764c1955d0 0x7f764c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:55.292 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f76516fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c195a40 0x7f764c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:55.292 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f76516fe700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c19d350 con 0x7f764c0ffe50 2023-11-16T16:40:55.292 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f76516fe700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c19d4c0 con 0x7f764c1955d0 2023-11-16T16:40:55.292 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f76516fe700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c19d680 con 0x7f764c195a40 2023-11-16T16:40:55.292 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f764b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c195a40 0x7f764c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:55.292 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f764b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c195a40 0x7f764c19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54034/0 (socket says 172.21.15.159:54034) 2023-11-16T16:40:55.292 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f764b7fe700 1 -- 172.21.15.159:0/656946027 learned_addr learned my addr 172.21.15.159:0/656946027 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-11-16T16:40:55.292 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f764affd700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f764c0ffe50 0x7f764c195090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:55.292 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f764a7fc700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f764c1955d0 0x7f764c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:55.293 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f764b7fe700 1 -- 172.21.15.159:0/656946027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f764c1955d0 msgr2=0x7f764c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:55.293 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f764b7fe700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f764c1955d0 0x7f764c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.293 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f764b7fe700 1 -- 172.21.15.159:0/656946027 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f764c0ffe50 msgr2=0x7f764c195090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:55.293 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f764b7fe700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f764c0ffe50 0x7f764c195090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.293 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f764b7fe700 1 -- 172.21.15.159:0/656946027 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400095f0 con 0x7f764c195a40 2023-11-16T16:40:55.293 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.286+0000 7f764a7fc700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f764c1955d0 0x7f764c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:40:55.293 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.287+0000 7f764b7fe700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c195a40 0x7f764c19cbb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f764400ba50 tx=0x7f764400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:55.293 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.287+0000 7f7633fff700 1 -- 172.21.15.159:0/656946027 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400c9b0 con 0x7f764c195a40 2023-11-16T16:40:55.293 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.287+0000 7f76516fe700 1 -- 172.21.15.159:0/656946027 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c105cb0 con 0x7f764c195a40 2023-11-16T16:40:55.294 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.287+0000 7f7633fff700 1 -- 172.21.15.159:0/656946027 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7644014440 con 0x7f764c195a40 2023-11-16T16:40:55.294 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.287+0000 7f7633fff700 1 -- 172.21.15.159:0/656946027 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644007590 con 0x7f764c195a40 2023-11-16T16:40:55.294 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.287+0000 7f76516fe700 1 -- 172.21.15.159:0/656946027 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c105b20 con 0x7f764c195a40 2023-11-16T16:40:55.294 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.288+0000 7f7633fff700 1 -- 172.21.15.159:0/656946027 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f76440077b0 con 0x7f764c195a40 2023-11-16T16:40:55.294 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.288+0000 7f7633fff700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7634038d40 0x7f763403b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:55.294 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.288+0000 7f76516fe700 1 -- 172.21.15.159:0/656946027 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c18e9e0 con 0x7f764c195a40 2023-11-16T16:40:55.294 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.288+0000 7f7633fff700 1 -- 172.21.15.159:0/656946027 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f764404aa80 con 0x7f764c195a40 2023-11-16T16:40:55.295 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.288+0000 7f764affd700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7634038d40 0x7f763403b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:55.295 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.289+0000 7f764affd700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7634038d40 0x7f763403b1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7640005950 tx=0x7f76400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:55.296 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.293+0000 7f7633fff700 1 -- 172.21.15.159:0/656946027 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f764401b8b0 con 0x7f764c195a40 2023-11-16T16:40:55.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:55 smithi125 ceph-mon[131076]: Reconfiguring mon.b (monmap changed)... 2023-11-16T16:40:55.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:55 smithi125 ceph-mon[131076]: Reconfiguring daemon mon.b on smithi125 2023-11-16T16:40:55.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:55.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:55.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:55.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:40:55.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:55.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:55 smithi093 ceph-mon[97863]: Reconfiguring mon.b (monmap changed)... 2023-11-16T16:40:55.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:55 smithi093 ceph-mon[97863]: Reconfiguring daemon mon.b on smithi125 2023-11-16T16:40:55.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:55.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:55.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:40:55.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:40:55.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:55.510 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.507+0000 7f76516fe700 1 -- 172.21.15.159:0/656946027 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi093=x;smithi125=y;smithi159=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f764c061bc0 con 0x7f7634038d40 2023-11-16T16:40:55.517 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.514+0000 7f7633fff700 1 -- 172.21.15.159:0/656946027 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f764c061bc0 con 0x7f7634038d40 2023-11-16T16:40:55.517 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mgr update... 2023-11-16T16:40:55.519 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.517+0000 7f76516fe700 1 -- 172.21.15.159:0/656946027 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7634038d40 msgr2=0x7f763403b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:55.519 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.517+0000 7f76516fe700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7634038d40 0x7f763403b1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7640005950 tx=0x7f76400058e0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.520 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.517+0000 7f76516fe700 1 -- 172.21.15.159:0/656946027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c195a40 msgr2=0x7f764c19cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:55.520 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.517+0000 7f76516fe700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c195a40 0x7f764c19cbb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f764400ba50 tx=0x7f764400bd60 comp rx=0 tx=0).stop 2023-11-16T16:40:55.520 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.517+0000 7f76516fe700 1 -- 172.21.15.159:0/656946027 shutdown_connections 2023-11-16T16:40:55.520 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.517+0000 7f76516fe700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7634038d40 0x7f763403b1f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.520 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.517+0000 7f76516fe700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f764c0ffe50 0x7f764c195090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.520 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.517+0000 7f76516fe700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c195a40 0x7f764c19cbb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.520 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.517+0000 7f76516fe700 1 --2- 172.21.15.159:0/656946027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f764c1955d0 0x7f764c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.520 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.517+0000 7f76516fe700 1 -- 172.21.15.159:0/656946027 >> 172.21.15.159:0/656946027 conn(0x7f764c0fb420 msgr2=0x7f764c0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:55.521 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.517+0000 7f76516fe700 1 -- 172.21.15.159:0/656946027 shutdown_connections 2023-11-16T16:40:55.521 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:40:55.517+0000 7f76516fe700 1 -- 172.21.15.159:0/656946027 wait complete. 2023-11-16T16:40:56.127 DEBUG:teuthology.orchestra.run.smithi125:mgr.y> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.y.service 2023-11-16T16:40:56.130 DEBUG:teuthology.orchestra.run.smithi159:mgr.z> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.z.service 2023-11-16T16:40:56.132 INFO:tasks.cephadm:Deploying OSDs... 2023-11-16T16:40:56.133 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:40:56.133 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/scratch_devs of=/dev/stdout 2023-11-16T16:40:56.161 INFO:journalctl@ceph.mgr.z.smithi159.stdout:-- Logs begin at Thu 2023-11-16 16:15:38 UTC. -- 2023-11-16T16:40:56.165 INFO:journalctl@ceph.mgr.y.smithi125.stdout:-- Logs begin at Thu 2023-11-16 16:14:41 UTC. -- 2023-11-16T16:40:56.182 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-16T16:40:56.182 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_1 2023-11-16T16:40:56.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:56 smithi159 ceph-mon[131528]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:56.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:56 smithi159 ceph-mon[131528]: Reconfiguring mon.c (monmap changed)... 2023-11-16T16:40:56.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:56 smithi159 ceph-mon[131528]: Reconfiguring daemon mon.c on smithi159 2023-11-16T16:40:56.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:56 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:56.272 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-16T16:40:56.273 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:40:56.273 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-11-16T16:40:56.273 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:40:56.273 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:40:56.273 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-11-16 16:39:59.881658577 +0000 2023-11-16T16:40:56.273 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-11-16 16:33:13.286502540 +0000 2023-11-16T16:40:56.273 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-11-16 16:33:13.286502540 +0000 2023-11-16T16:40:56.273 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: 2023-11-16 16:33:13.256503171 +0000 2023-11-16T16:40:56.274 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-16T16:40:56.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:56 smithi125 ceph-mon[131076]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:56.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:56 smithi125 ceph-mon[131076]: Reconfiguring mon.c (monmap changed)... 2023-11-16T16:40:56.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:56 smithi125 ceph-mon[131076]: Reconfiguring daemon mon.c on smithi159 2023-11-16T16:40:56.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:56 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:56.339 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:56 smithi093 ceph-mon[97863]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:56.339 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:56 smithi093 ceph-mon[97863]: Reconfiguring mon.c (monmap changed)... 2023-11-16T16:40:56.340 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:56 smithi093 ceph-mon[97863]: Reconfiguring daemon mon.c on smithi159 2023-11-16T16:40:56.340 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:56 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:56.347 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-11-16T16:40:56.347 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-11-16T16:40:56.347 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.00031623 s, 1.6 MB/s 2023-11-16T16:40:56.350 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-16T16:40:56.400 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_2 2023-11-16T16:40:56.444 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-16T16:40:56.445 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:40:56.445 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-11-16T16:40:56.445 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:40:56.445 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:40:56.445 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-11-16 16:39:59.881658577 +0000 2023-11-16T16:40:56.445 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-11-16 16:33:14.078485883 +0000 2023-11-16T16:40:56.445 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-11-16 16:33:14.078485883 +0000 2023-11-16T16:40:56.445 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: 2023-11-16 16:33:14.049486493 +0000 2023-11-16T16:40:56.446 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-16T16:40:56.552 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-11-16T16:40:56.553 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-11-16T16:40:56.553 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000377322 s, 1.4 MB/s 2023-11-16T16:40:56.556 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-16T16:40:56.602 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_3 2023-11-16T16:40:56.647 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-16T16:40:56.647 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:40:56.647 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-11-16T16:40:56.647 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:40:56.647 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:40:56.647 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-11-16 16:39:59.881658577 +0000 2023-11-16T16:40:56.648 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-11-16 16:33:14.876469100 +0000 2023-11-16T16:40:56.648 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-11-16 16:33:14.876469100 +0000 2023-11-16T16:40:56.648 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: 2023-11-16 16:33:14.849469668 +0000 2023-11-16T16:40:56.648 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-16T16:40:56.753 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-11-16T16:40:56.753 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-11-16T16:40:56.753 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000307938 s, 1.7 MB/s 2023-11-16T16:40:56.756 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-16T16:40:56.802 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_4 2023-11-16T16:40:56.887 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-16T16:40:56.887 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:40:56.887 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-11-16T16:40:56.887 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:40:56.887 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:40:56.887 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-11-16 16:39:59.881658577 +0000 2023-11-16T16:40:56.887 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-11-16 16:33:15.683452128 +0000 2023-11-16T16:40:56.887 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-11-16 16:33:15.683452128 +0000 2023-11-16T16:40:56.888 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: 2023-11-16 16:33:15.656452696 +0000 2023-11-16T16:40:56.888 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-16T16:40:56.993 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-11-16T16:40:56.993 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-11-16T16:40:56.994 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000294773 s, 1.7 MB/s 2023-11-16T16:40:56.996 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-16T16:40:57.041 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-11-16T16:40:57.042 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/scratch_devs of=/dev/stdout 2023-11-16T16:40:57.063 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-16T16:40:57.063 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_1 2023-11-16T16:40:57.125 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-16T16:40:57.125 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:40:57.125 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 6h/6d Inode: 387011 Links: 1 2023-11-16T16:40:57.125 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:40:57.125 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:40:57.125 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2023-11-16 16:40:42.755968475 +0000 2023-11-16T16:40:57.125 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2023-11-16 16:30:07.869362431 +0000 2023-11-16T16:40:57.126 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2023-11-16 16:30:07.869362431 +0000 2023-11-16T16:40:57.126 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: - 2023-11-16T16:40:57.126 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-16T16:40:57.196 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2023-11-16T16:40:57.196 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2023-11-16T16:40:57.197 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000243781 s, 2.1 MB/s 2023-11-16T16:40:57.198 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-16T16:40:57.260 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_2 2023-11-16T16:40:57.321 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-16T16:40:57.322 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:40:57.322 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 6h/6d Inode: 389901 Links: 1 2023-11-16T16:40:57.322 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:40:57.322 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:40:57.322 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2023-11-16 16:40:42.755968475 +0000 2023-11-16T16:40:57.322 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2023-11-16 16:30:08.369354396 +0000 2023-11-16T16:40:57.322 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2023-11-16 16:30:08.369354396 +0000 2023-11-16T16:40:57.322 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: - 2023-11-16T16:40:57.323 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-16T16:40:57.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:57 smithi125 ceph-mon[131076]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi093=x;smithi125=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:40:57.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:57 smithi125 ceph-mon[131076]: Saving service mgr spec with placement smithi093=x;smithi125=y;smithi159=z;count:3 2023-11-16T16:40:57.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:57 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:57.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:57 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:57.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:57 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:57.359 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2023-11-16T16:40:57.359 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2023-11-16T16:40:57.359 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000249553 s, 2.1 MB/s 2023-11-16T16:40:57.360 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-16T16:40:57.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:57 smithi093 ceph-mon[97863]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi093=x;smithi125=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:40:57.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:57 smithi093 ceph-mon[97863]: Saving service mgr spec with placement smithi093=x;smithi125=y;smithi159=z;count:3 2023-11-16T16:40:57.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:57 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:57.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:57 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:57.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:57 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:57.422 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_3 2023-11-16T16:40:57.483 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-16T16:40:57.483 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:40:57.483 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 6h/6d Inode: 385954 Links: 1 2023-11-16T16:40:57.483 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:40:57.484 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:40:57.484 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2023-11-16 16:40:42.755968475 +0000 2023-11-16T16:40:57.484 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2023-11-16 16:30:08.854346601 +0000 2023-11-16T16:40:57.484 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2023-11-16 16:30:08.854346601 +0000 2023-11-16T16:40:57.484 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: - 2023-11-16T16:40:57.484 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-16T16:40:57.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:57 smithi159 ceph-mon[131528]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi093=x;smithi125=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:40:57.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:57 smithi159 ceph-mon[131528]: Saving service mgr spec with placement smithi093=x;smithi125=y;smithi159=z;count:3 2023-11-16T16:40:57.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:57 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:57.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:57 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:57.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:57 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:57.554 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2023-11-16T16:40:57.554 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2023-11-16T16:40:57.554 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000251606 s, 2.0 MB/s 2023-11-16T16:40:57.555 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-16T16:40:57.617 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_4 2023-11-16T16:40:57.678 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-16T16:40:57.678 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:40:57.679 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 6h/6d Inode: 394260 Links: 1 2023-11-16T16:40:57.679 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:40:57.679 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:40:57.679 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2023-11-16 16:40:42.755968475 +0000 2023-11-16T16:40:57.679 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2023-11-16 16:30:09.342338759 +0000 2023-11-16T16:40:57.679 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2023-11-16 16:30:09.342338759 +0000 2023-11-16T16:40:57.679 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: - 2023-11-16T16:40:57.679 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-16T16:40:57.750 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2023-11-16T16:40:57.750 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2023-11-16T16:40:57.750 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000211897 s, 2.4 MB/s 2023-11-16T16:40:57.751 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-16T16:40:57.814 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-11-16T16:40:57.814 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2023-11-16T16:40:57.832 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-16T16:40:57.832 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2023-11-16T16:40:57.890 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-16T16:40:57.890 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:40:57.890 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 388720 Links: 1 2023-11-16T16:40:57.890 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:40:57.890 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:40:57.890 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-11-16 16:40:34.260330269 +0000 2023-11-16T16:40:57.891 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-11-16 16:30:07.872296005 +0000 2023-11-16T16:40:57.891 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-11-16 16:30:07.872296005 +0000 2023-11-16T16:40:57.891 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-11-16T16:40:57.891 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-16T16:40:57.956 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-11-16T16:40:57.957 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-11-16T16:40:57.957 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000242708 s, 2.1 MB/s 2023-11-16T16:40:57.958 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-16T16:40:58.016 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2023-11-16T16:40:58.075 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-16T16:40:58.076 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:40:58.076 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 391516 Links: 1 2023-11-16T16:40:58.076 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:40:58.076 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:40:58.076 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-11-16 16:40:34.261330250 +0000 2023-11-16T16:40:58.076 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-11-16 16:30:08.369286324 +0000 2023-11-16T16:40:58.076 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-11-16 16:30:08.369286324 +0000 2023-11-16T16:40:58.076 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-11-16T16:40:58.077 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-16T16:40:58.141 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-11-16T16:40:58.141 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-11-16T16:40:58.141 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000244914 s, 2.1 MB/s 2023-11-16T16:40:58.142 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-16T16:40:58.199 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2023-11-16T16:40:58.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:58 smithi159 ceph-mon[131528]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:58.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:58.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:58.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:58.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:40:58.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-16T16:40:58.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:40:58.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:58.252 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-16T16:40:58.252 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:40:58.253 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 388917 Links: 1 2023-11-16T16:40:58.253 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:40:58.253 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:40:58.253 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-11-16 16:40:34.261330250 +0000 2023-11-16T16:40:58.253 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-11-16 16:30:08.858276799 +0000 2023-11-16T16:40:58.253 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-11-16 16:30:08.858276799 +0000 2023-11-16T16:40:58.253 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-11-16T16:40:58.253 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-16T16:40:58.320 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-11-16T16:40:58.320 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-11-16T16:40:58.320 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000243294 s, 2.1 MB/s 2023-11-16T16:40:58.321 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-16T16:40:58.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:58 smithi125 ceph-mon[131076]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:58.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:58.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:58.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:58.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:40:58.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-16T16:40:58.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:40:58.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:58.379 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2023-11-16T16:40:58.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:58 smithi093 ceph-mon[97863]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:40:58.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:58.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:58.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:40:58.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:40:58.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-16T16:40:58.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:40:58.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:58.436 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-16T16:40:58.436 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:40:58.436 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 389459 Links: 1 2023-11-16T16:40:58.437 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:40:58.437 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:40:58.437 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-11-16 16:40:34.261330250 +0000 2023-11-16T16:40:58.437 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-11-16 16:30:09.388266476 +0000 2023-11-16T16:40:58.437 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-11-16 16:30:09.388266476 +0000 2023-11-16T16:40:58.437 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-11-16T16:40:58.437 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-16T16:40:58.501 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-11-16T16:40:58.501 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-11-16T16:40:58.501 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000198776 s, 2.6 MB/s 2023-11-16T16:40:58.503 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-16T16:40:58.561 INFO:tasks.cephadm:Deploying osd.0 on smithi093 with /dev/vg_nvme/lv_4... 2023-11-16T16:40:58.561 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-16T16:40:58.985 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:40:58 smithi159 systemd[1]: Starting Ceph mgr.z for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:40:59.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:59 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:40:59.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:59 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:40:59.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:40:59 smithi125 ceph-mon[131076]: Deploying daemon mgr.z on smithi159 2023-11-16T16:40:59.349 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:59 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:40:59.349 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:59 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:40:59.349 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:40:59 smithi159 ceph-mon[131528]: Deploying daemon mgr.z on smithi159 2023-11-16T16:40:59.349 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:40:59 smithi159 podman[133627]: 2023-11-16T16:40:59.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:59 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:40:59.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:59 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:40:59.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:40:59 smithi093 ceph-mon[97863]: Deploying daemon mgr.z on smithi159 2023-11-16T16:40:59.411 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:40:59.735 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:40:59 smithi159 bash[133627]: 85d0ba0609d7d5158e3fa490ac5a2f85d3143831c4c6ada0d165addd636c3d2a 2023-11-16T16:40:59.735 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:40:59 smithi159 ceph-mgr[133667]: -- 172.21.15.159:0/3872726407 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c6297ef520 con 0x55c6297c4800 2023-11-16T16:40:59.736 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:40:59 smithi159 systemd[1]: Started Ceph mgr.z for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:40:59.736 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:40:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:40:59.608+0000 7f70105a8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:40:59.736 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:40:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:40:59.686+0000 7f70105a8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:41:00.038 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:40:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:40:59.776+0000 7f70105a8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:41:00.038 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:40:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:40:59.868+0000 7f70105a8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:41:00.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:00 smithi125 ceph-mon[131076]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:00.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:00.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:00.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:00.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:41:00.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-16T16:41:00.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:41:00.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:00.348 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:00.036+0000 7f70105a8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:41:00.348 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-mon[131528]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:00.348 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:00.349 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:00.349 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:00.349 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:41:00.349 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-16T16:41:00.349 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:41:00.349 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:00.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:00 smithi093 ceph-mon[97863]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:00.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:00.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:00.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:41:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-16T16:41:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:41:00.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:00.666 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:00.347+0000 7f70105a8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:41:00.667 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:00.436+0000 7f70105a8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:41:00.667 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:00.571+0000 7f70105a8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:41:00.985 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:00.665+0000 7f70105a8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:41:01.284 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-16T16:41:01.284 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-16T16:41:01.284 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-11-16T16:41:01.284 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-11-16T16:41:01.284 INFO:teuthology.orchestra.run.smithi093.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0488313 s, 215 MB/s 2023-11-16T16:41:01.284 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-11-16T16:41:01.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:01 smithi125 ceph-mon[131076]: Deploying daemon mgr.y on smithi125 2023-11-16T16:41:01.328 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:01 smithi125 systemd[1]: Starting Ceph mgr.y for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:41:01.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:01 smithi093 ceph-mon[97863]: Deploying daemon mgr.y on smithi125 2023-11-16T16:41:01.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:01 smithi159 ceph-mon[131528]: Deploying daemon mgr.y on smithi125 2023-11-16T16:41:01.485 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:01 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:01.087+0000 7f70105a8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:41:01.612 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:01 smithi125 podman[132519]: 2023-11-16T16:41:01.612 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:01 smithi125 bash[132519]: 1feb1e3a87b3f070df759598c4b44a26a47dc3ae3f2376d630944d1acd51248a 2023-11-16T16:41:01.791 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi093:vg_nvme/lv_4 2023-11-16T16:41:01.905 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:01 smithi125 ceph-mgr[132559]: -- 172.21.15.125:0/622230213 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55761026d520 con 0x557610243000 2023-11-16T16:41:01.905 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:01 smithi125 systemd[1]: Started Ceph mgr.y for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:41:01.905 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:01 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:01.733+0000 7f5743e02200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:41:01.905 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:01 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:01.810+0000 7f5743e02200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:41:01.905 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:01 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:01.903+0000 7f5743e02200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:41:01.985 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:01 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:01.663+0000 7f70105a8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:41:01.985 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:01 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:01.835+0000 7f70105a8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:41:02.184 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:02.007+0000 7f5743e02200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:41:02.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:02 smithi125 ceph-mon[131076]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:02.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:02 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:02.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:02 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:02.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:02 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:02.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:02 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:02.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:02 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:02.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:02 smithi093 ceph-mon[97863]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:02.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:02 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:02.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:02 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:02.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:02 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:02.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:02 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:02.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:02 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:02.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:02 smithi159 ceph-mon[131528]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:02.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:02 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:02.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:02 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:02.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:02 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:02.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:02 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:02.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:02 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:02.507 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:02.182+0000 7f5743e02200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:41:02.507 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:02.505+0000 7f5743e02200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:41:02.638 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:41:02.825 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:02.593+0000 7f5743e02200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:41:02.826 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:02.726+0000 7f5743e02200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:41:02.915 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:02.642+0000 7f70105a8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:41:02.916 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:02.736+0000 7f70105a8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:41:03.077 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:02.824+0000 7f5743e02200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:41:03.214 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:02.914+0000 7f70105a8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:41:03.215 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:03 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:03.259 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:03 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:03.485 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:03.213+0000 7f70105a8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:41:03.485 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:03.450+0000 7f70105a8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:41:03.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:03 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:03.578 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:03 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:03.267+0000 7f5743e02200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:41:03.867 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:03.602+0000 7f70105a8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:41:04.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.147+0000 7f826417c700 1 -- 172.21.15.93:0/1768084835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c074b20 msgr2=0x7f825c072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:04.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.147+0000 7f826417c700 1 --2- 172.21.15.93:0/1768084835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c074b20 0x7f825c072f80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f82580099c0 tx=0x7f8258009cd0 comp rx=0 tx=0).stop 2023-11-16T16:41:04.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.147+0000 7f826417c700 1 -- 172.21.15.93:0/1768084835 shutdown_connections 2023-11-16T16:41:04.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.148+0000 7f826417c700 1 --2- 172.21.15.93:0/1768084835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c074b20 0x7f825c072f80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:04.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.148+0000 7f826417c700 1 --2- 172.21.15.93:0/1768084835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f825c10c950 0x7f825c10ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:04.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.148+0000 7f826417c700 1 --2- 172.21.15.93:0/1768084835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f825c073550 0x7f825c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:04.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.148+0000 7f826417c700 1 -- 172.21.15.93:0/1768084835 >> 172.21.15.93:0/1768084835 conn(0x7f825c0fb9a0 msgr2=0x7f825c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:04.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.148+0000 7f826417c700 1 -- 172.21.15.93:0/1768084835 shutdown_connections 2023-11-16T16:41:04.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.148+0000 7f826417c700 1 -- 172.21.15.93:0/1768084835 wait complete. 2023-11-16T16:41:04.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:04 smithi093 ceph-mon[97863]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:04.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:04.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:04.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:41:04.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:41:04.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:04.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.149+0000 7f826417c700 1 Processor -- start 2023-11-16T16:41:04.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.149+0000 7f826417c700 1 -- start start 2023-11-16T16:41:04.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.149+0000 7f826417c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c073550 0x7f825c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:04.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.149+0000 7f826417c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f825c074b20 0x7f825c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:04.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.149+0000 7f826417c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f825c10c950 0x7f825c1a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:04.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.149+0000 7f826417c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c1a41c0 con 0x7f825c073550 2023-11-16T16:41:04.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.149+0000 7f826417c700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c1a4300 con 0x7f825c10c950 2023-11-16T16:41:04.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.149+0000 7f826417c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c1a4490 con 0x7f825c074b20 2023-11-16T16:41:04.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.149+0000 7f8261717700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f825c074b20 0x7f825c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:04.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.149+0000 7f8261717700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f825c074b20 0x7f825c19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:38706/0 (socket says 172.21.15.93:38706) 2023-11-16T16:41:04.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.149+0000 7f8261717700 1 -- 172.21.15.93:0/2121212555 learned_addr learned my addr 172.21.15.93:0/2121212555 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:41:04.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.150+0000 7f8261f18700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c073550 0x7f825c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:04.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.150+0000 7f8262719700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f825c10c950 0x7f825c1a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:04.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.150+0000 7f8261717700 1 -- 172.21.15.93:0/2121212555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f825c10c950 msgr2=0x7f825c1a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:04.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.150+0000 7f8261717700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f825c10c950 0x7f825c1a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:04.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.150+0000 7f8261717700 1 -- 172.21.15.93:0/2121212555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c073550 msgr2=0x7f825c19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:04.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.150+0000 7f8261717700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c073550 0x7f825c19c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:04.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.150+0000 7f8261717700 1 -- 172.21.15.93:0/2121212555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82580095f0 con 0x7f825c074b20 2023-11-16T16:41:04.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.150+0000 7f8262719700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f825c10c950 0x7f825c1a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:41:04.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.150+0000 7f8261f18700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c073550 0x7f825c19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:41:04.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.150+0000 7f8261717700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f825c074b20 0x7f825c19cb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f824c00e9d0 tx=0x7f824c00ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:04.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.151+0000 7f8252ffd700 1 -- 172.21.15.93:0/2121212555 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c009790 con 0x7f825c074b20 2023-11-16T16:41:04.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.151+0000 7f8252ffd700 1 -- 172.21.15.93:0/2121212555 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f824c00cbd0 con 0x7f825c074b20 2023-11-16T16:41:04.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.151+0000 7f8252ffd700 1 -- 172.21.15.93:0/2121212555 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c018680 con 0x7f825c074b20 2023-11-16T16:41:04.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.151+0000 7f826417c700 1 -- 172.21.15.93:0/2121212555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825c112320 con 0x7f825c074b20 2023-11-16T16:41:04.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.151+0000 7f826417c700 1 -- 172.21.15.93:0/2121212555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825c112190 con 0x7f825c074b20 2023-11-16T16:41:04.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.152+0000 7f8252ffd700 1 -- 172.21.15.93:0/2121212555 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f824c0187e0 con 0x7f825c074b20 2023-11-16T16:41:04.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.152+0000 7f826417c700 1 -- 172.21.15.93:0/2121212555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825c1024a0 con 0x7f825c074b20 2023-11-16T16:41:04.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.152+0000 7f8252ffd700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f82480389e0 0x7f824803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:04.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.152+0000 7f8252ffd700 1 -- 172.21.15.93:0/2121212555 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f824c014070 con 0x7f825c074b20 2023-11-16T16:41:04.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.153+0000 7f8261f18700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f82480389e0 0x7f824803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:04.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.153+0000 7f8261f18700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f82480389e0 0x7f824803ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8258006010 tx=0x7f8258000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:04.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.157+0000 7f8252ffd700 1 -- 172.21.15.93:0/2121212555 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f824c01f8b0 con 0x7f825c074b20 2023-11-16T16:41:04.235 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:03.865+0000 7f70105a8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:41:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-mon[131528]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:04.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:04.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:41:04.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:41:04.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:04.328 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:03 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:03.850+0000 7f5743e02200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:41:04.328 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:04.017+0000 7f5743e02200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:41:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-mon[131076]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:04.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:04.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:41:04.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:41:04.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:04.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:04.370+0000 7f826417c700 1 -- 172.21.15.93:0/2121212555 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f825c061cb0 con 0x7f82480389e0 2023-11-16T16:41:04.890 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:04.571+0000 7f70105a8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:41:04.890 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:04.699+0000 7f70105a8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:41:05.034 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:04.776+0000 7f5743e02200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:41:05.034 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:04 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:04.865+0000 7f5743e02200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:41:05.220 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:04 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:04.889+0000 7f70105a8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:41:05.220 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:05 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:05.002+0000 7f70105a8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:41:05.220 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:05 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:05.218+0000 7f70105a8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:41:05.220 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:05 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:05.220 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:05 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:41:05.220 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:05 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:05.220 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:05 smithi159 ceph-mon[131528]: Reconfiguring mgr.x (unknown last config time)... 2023-11-16T16:41:05.220 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:05 smithi159 ceph-mon[131528]: Reconfiguring daemon mgr.x on smithi093 2023-11-16T16:41:05.221 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:05 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:41:05.221 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:05 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:41:05.221 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:05 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:05.308 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:05.032+0000 7f5743e02200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:41:05.308 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:05.308 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:41:05.308 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:05.308 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-mon[131076]: Reconfiguring mgr.x (unknown last config time)... 2023-11-16T16:41:05.308 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-mon[131076]: Reconfiguring daemon mgr.x on smithi093 2023-11-16T16:41:05.308 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:41:05.309 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:41:05.309 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:05.362 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:05 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:05.362 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:05 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:41:05.362 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:05 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:05.362 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:05 smithi093 ceph-mon[97863]: Reconfiguring mgr.x (unknown last config time)... 2023-11-16T16:41:05.363 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:05 smithi093 ceph-mon[97863]: Reconfiguring daemon mgr.x on smithi093 2023-11-16T16:41:05.363 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:05 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:41:05.363 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:05 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:41:05.363 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:05 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:05.578 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:05.306+0000 7f5743e02200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:41:05.578 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:05.530+0000 7f5743e02200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:41:05.938 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:05.679+0000 7f5743e02200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:41:06.329 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:05.936+0000 7f5743e02200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:41:06.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:41:06.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:06.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:06.330 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:06.330 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:06.330 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:06.330 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:06.330 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:06.330 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:06.330 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: Standby manager daemon z started 2023-11-16T16:41:06.331 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: from='mgr.? 172.21.15.159:0/478437545' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-16T16:41:06.331 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: from='mgr.? 172.21.15.159:0/478437545' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:41:06.331 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: from='mgr.? 172.21.15.159:0/478437545' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-16T16:41:06.331 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-mon[131076]: from='mgr.? 172.21.15.159:0/478437545' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:41:06.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:41:06.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:06.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:06.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:06.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:06.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:06.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:06.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:06.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:06.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: Standby manager daemon z started 2023-11-16T16:41:06.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: from='mgr.? 172.21.15.159:0/478437545' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-16T16:41:06.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: from='mgr.? 172.21.15.159:0/478437545' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:41:06.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: from='mgr.? 172.21.15.159:0/478437545' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-16T16:41:06.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:06 smithi093 ceph-mon[97863]: from='mgr.? 172.21.15.159:0/478437545' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:41:06.485 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z[133643]: 2023-11-16T16:41:06.076+0000 7f70105a8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:41:06.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:41:06.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:06.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:06.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:06.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:06.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:06.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:06.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:06.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:06.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: Standby manager daemon z started 2023-11-16T16:41:06.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: from='mgr.? 172.21.15.159:0/478437545' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-16T16:41:06.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: from='mgr.? 172.21.15.159:0/478437545' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:41:06.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: from='mgr.? 172.21.15.159:0/478437545' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-16T16:41:06.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:06 smithi159 ceph-mon[131528]: from='mgr.? 172.21.15.159:0/478437545' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:41:06.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:06.506+0000 7f8252ffd700 1 -- 172.21.15.93:0/2121212555 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7f824c017750 con 0x7f825c074b20 2023-11-16T16:41:06.948 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:06.630+0000 7f5743e02200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:41:06.948 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:06.759+0000 7f5743e02200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:41:07.272 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:07 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:07.272 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:07 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:41:07.272 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:07 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:07.272 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:07 smithi125 ceph-mon[131076]: mgrmap e14: x(active, since 76s), standbys: z 2023-11-16T16:41:07.273 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:07 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-16T16:41:07.273 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:06 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:06.946+0000 7f5743e02200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:41:07.273 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:07 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:07.057+0000 7f5743e02200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:41:07.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:07 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:07.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:07 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:41:07.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:07 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:07.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:07 smithi093 ceph-mon[97863]: mgrmap e14: x(active, since 76s), standbys: z 2023-11-16T16:41:07.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:07 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-16T16:41:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:07 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:07 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:41:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:07 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:07 smithi159 ceph-mon[131528]: mgrmap e14: x(active, since 76s), standbys: z 2023-11-16T16:41:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:07 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-16T16:41:07.578 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:07 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:07.270+0000 7f5743e02200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:41:08.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:08 smithi093 ceph-mon[97863]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:08.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:08 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3099438982' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "329eef05-603a-4f45-be62-d6317cc25e60"}]: dispatch 2023-11-16T16:41:08.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:08 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "329eef05-603a-4f45-be62-d6317cc25e60"}]: dispatch 2023-11-16T16:41:08.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:08 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "329eef05-603a-4f45-be62-d6317cc25e60"}]': finished 2023-11-16T16:41:08.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:08 smithi093 ceph-mon[97863]: osdmap e5: 1 total, 0 up, 1 in 2023-11-16T16:41:08.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:08 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:08.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:08 smithi159 ceph-mon[131528]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:08.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:08 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3099438982' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "329eef05-603a-4f45-be62-d6317cc25e60"}]: dispatch 2023-11-16T16:41:08.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:08 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "329eef05-603a-4f45-be62-d6317cc25e60"}]: dispatch 2023-11-16T16:41:08.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:08 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "329eef05-603a-4f45-be62-d6317cc25e60"}]': finished 2023-11-16T16:41:08.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:08 smithi159 ceph-mon[131528]: osdmap e5: 1 total, 0 up, 1 in 2023-11-16T16:41:08.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:08 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:08.578 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:41:08 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y[132535]: 2023-11-16T16:41:08.111+0000 7f5743e02200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:41:08.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:08 smithi125 ceph-mon[131076]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:08.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:08 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3099438982' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "329eef05-603a-4f45-be62-d6317cc25e60"}]: dispatch 2023-11-16T16:41:08.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:08 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "329eef05-603a-4f45-be62-d6317cc25e60"}]: dispatch 2023-11-16T16:41:08.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:08 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "329eef05-603a-4f45-be62-d6317cc25e60"}]': finished 2023-11-16T16:41:08.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:08 smithi125 ceph-mon[131076]: osdmap e5: 1 total, 0 up, 1 in 2023-11-16T16:41:08.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:08 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:09.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:09.117+0000 7f8252ffd700 1 -- 172.21.15.93:0/2121212555 <== mon.1 v2:172.21.15.159:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f824c00fad0 con 0x7f825c074b20 2023-11-16T16:41:09.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:09 smithi093 ceph-mon[97863]: Standby manager daemon y started 2023-11-16T16:41:09.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:09 smithi093 ceph-mon[97863]: from='mgr.? 172.21.15.125:0/1611643452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-16T16:41:09.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:09 smithi093 ceph-mon[97863]: from='mgr.? 172.21.15.125:0/1611643452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:41:09.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:09 smithi093 ceph-mon[97863]: from='mgr.? 172.21.15.125:0/1611643452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-16T16:41:09.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:09 smithi093 ceph-mon[97863]: from='mgr.? 172.21.15.125:0/1611643452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:41:09.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:09 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2343894786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:41:09.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:09 smithi159 ceph-mon[131528]: Standby manager daemon y started 2023-11-16T16:41:09.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:09 smithi159 ceph-mon[131528]: from='mgr.? 172.21.15.125:0/1611643452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-16T16:41:09.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:09 smithi159 ceph-mon[131528]: from='mgr.? 172.21.15.125:0/1611643452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:41:09.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:09 smithi159 ceph-mon[131528]: from='mgr.? 172.21.15.125:0/1611643452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-16T16:41:09.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:09 smithi159 ceph-mon[131528]: from='mgr.? 172.21.15.125:0/1611643452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:41:09.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:09 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2343894786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:41:09.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:09 smithi125 ceph-mon[131076]: Standby manager daemon y started 2023-11-16T16:41:09.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:09 smithi125 ceph-mon[131076]: from='mgr.? 172.21.15.125:0/1611643452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-16T16:41:09.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:09 smithi125 ceph-mon[131076]: from='mgr.? 172.21.15.125:0/1611643452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:41:09.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:09 smithi125 ceph-mon[131076]: from='mgr.? 172.21.15.125:0/1611643452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-16T16:41:09.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:09 smithi125 ceph-mon[131076]: from='mgr.? 172.21.15.125:0/1611643452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:41:09.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:09 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2343894786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:41:10.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:10 smithi093 ceph-mon[97863]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:10.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:10 smithi093 ceph-mon[97863]: mgrmap e15: x(active, since 78s), standbys: z, y 2023-11-16T16:41:10.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:10 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-16T16:41:10.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:10 smithi159 ceph-mon[131528]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:10.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:10 smithi159 ceph-mon[131528]: mgrmap e15: x(active, since 78s), standbys: z, y 2023-11-16T16:41:10.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:10 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-16T16:41:10.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:10 smithi125 ceph-mon[131076]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:10.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:10 smithi125 ceph-mon[131076]: mgrmap e15: x(active, since 78s), standbys: z, y 2023-11-16T16:41:10.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:10 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-16T16:41:11.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:11 smithi159 ceph-mon[131528]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:11.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:11 smithi125 ceph-mon[131076]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:11.596 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:11 smithi093 ceph-mon[97863]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:13.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:13 smithi125 ceph-mon[131076]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:13 smithi093 ceph-mon[97863]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:13.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:13 smithi159 ceph-mon[131528]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:15.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:15 smithi125 ceph-mon[131076]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:15.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:15 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-16T16:41:15.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:15 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:15.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:15 smithi125 ceph-mon[131076]: Deploying daemon osd.0 on smithi093 2023-11-16T16:41:15.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:15 smithi093 ceph-mon[97863]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:15.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:15 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-16T16:41:15.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:15 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:15.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:15 smithi093 ceph-mon[97863]: Deploying daemon osd.0 on smithi093 2023-11-16T16:41:15.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:15 smithi159 ceph-mon[131528]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:15.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:15 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-16T16:41:15.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:15 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:15.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:15 smithi159 ceph-mon[131528]: Deploying daemon osd.0 on smithi093 2023-11-16T16:41:17.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:17 smithi125 ceph-mon[131076]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:17 smithi093 ceph-mon[97863]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:17 smithi159 ceph-mon[131528]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:19 smithi125 ceph-mon[131076]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:19 smithi093 ceph-mon[97863]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:19.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:19 smithi159 ceph-mon[131528]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:21.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:21 smithi159 ceph-mon[131528]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:21.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:21 smithi125 ceph-mon[131076]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:21.626 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:21 smithi093 ceph-mon[97863]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:22.494 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:22 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:22.494 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:22 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:22.494 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:22 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:22.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:22 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:22.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:22 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:22.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:22 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:22.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:22 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:22.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:22 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:22.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:22 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:23.560 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:23 smithi093 ceph-mon[97863]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:23.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:23 smithi125 ceph-mon[131076]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:23.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:23 smithi159 ceph-mon[131528]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:24.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.832+0000 7f8252ffd700 1 -- 172.21.15.93:0/2121212555 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f825c061cb0 con 0x7f82480389e0 2023-11-16T16:41:24.835 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 0 on host 'smithi093' 2023-11-16T16:41:24.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.835+0000 7f826417c700 1 -- 172.21.15.93:0/2121212555 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f82480389e0 msgr2=0x7f824803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:24.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.835+0000 7f826417c700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f82480389e0 0x7f824803ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8258006010 tx=0x7f8258000bc0 comp rx=0 tx=0).stop 2023-11-16T16:41:24.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.835+0000 7f826417c700 1 -- 172.21.15.93:0/2121212555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f825c074b20 msgr2=0x7f825c19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:24.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.835+0000 7f826417c700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f825c074b20 0x7f825c19cb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f824c00e9d0 tx=0x7f824c00ed90 comp rx=0 tx=0).stop 2023-11-16T16:41:24.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.835+0000 7f826417c700 1 -- 172.21.15.93:0/2121212555 shutdown_connections 2023-11-16T16:41:24.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.835+0000 7f826417c700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f82480389e0 0x7f824803ae90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:24.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.835+0000 7f826417c700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c073550 0x7f825c19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:24.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.835+0000 7f826417c700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f825c10c950 0x7f825c1a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:24.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.835+0000 7f826417c700 1 --2- 172.21.15.93:0/2121212555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f825c074b20 0x7f825c19cb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:24.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.835+0000 7f826417c700 1 -- 172.21.15.93:0/2121212555 >> 172.21.15.93:0/2121212555 conn(0x7f825c0fb9a0 msgr2=0x7f825c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:24.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.835+0000 7f826417c700 1 -- 172.21.15.93:0/2121212555 shutdown_connections 2023-11-16T16:41:24.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:41:24.835+0000 7f826417c700 1 -- 172.21.15.93:0/2121212555 wait complete. 2023-11-16T16:41:24.996 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:24.996 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:24.996 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:24.996 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:24.996 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:24 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:24.996 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:24.997 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:24 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:41:24.997 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:24 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:24.997 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:24.997 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:24.997 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:25.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:25.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:25.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:25.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:25.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:24 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:25.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:25.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:24 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:41:25.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:24 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:25.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:25.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:25.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:25.284 DEBUG:teuthology.orchestra.run.smithi093:osd.0> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.0.service 2023-11-16T16:41:25.288 INFO:tasks.cephadm:Deploying osd.1 on smithi125 with /dev/vg_nvme/lv_4... 2023-11-16T16:41:25.289 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-16T16:41:25.321 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:25.321 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:25.322 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:25.322 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:25.322 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:24 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:25.322 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:25.322 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:24 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:41:25.322 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:24 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:25.322 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:25.322 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:25.322 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:25.368 INFO:journalctl@ceph.osd.0.smithi093.stdout:-- Logs begin at Thu 2023-11-16 16:19:38 UTC. -- 2023-11-16T16:41:25.937 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:41:26.131 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:25 smithi093 ceph-mon[97863]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:26.131 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:25 smithi093 ceph-mon[97863]: from='osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-16T16:41:26.164 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:25 smithi125 ceph-mon[131076]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:26.164 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:25 smithi125 ceph-mon[131076]: from='osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-16T16:41:26.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:25 smithi159 ceph-mon[131528]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:26.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:25 smithi159 ceph-mon[131528]: from='osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-16T16:41:27.201 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:27 smithi093 ceph-mon[97863]: from='osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-16T16:41:27.201 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:27 smithi093 ceph-mon[97863]: osdmap e6: 1 total, 0 up, 1 in 2023-11-16T16:41:27.201 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:27 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:27.202 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:27 smithi093 ceph-mon[97863]: from='osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:41:27.202 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:27 smithi093 ceph-mon[97863]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:27.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:27 smithi125 ceph-mon[131076]: from='osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-16T16:41:27.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:27 smithi125 ceph-mon[131076]: osdmap e6: 1 total, 0 up, 1 in 2023-11-16T16:41:27.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:27 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:27.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:27 smithi125 ceph-mon[131076]: from='osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:41:27.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:27 smithi125 ceph-mon[131076]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:27.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:27 smithi159 ceph-mon[131528]: from='osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-16T16:41:27.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:27 smithi159 ceph-mon[131528]: osdmap e6: 1 total, 0 up, 1 in 2023-11-16T16:41:27.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:27 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:27.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:27 smithi159 ceph-mon[131528]: from='osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:41:27.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:27 smithi159 ceph-mon[131528]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:41:27.571 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-16T16:41:27.572 INFO:teuthology.orchestra.run.smithi125.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-16T16:41:27.572 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10+0 records in 2023-11-16T16:41:27.572 INFO:teuthology.orchestra.run.smithi125.stderr:10+0 records out 2023-11-16T16:41:27.572 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112393 s, 933 MB/s 2023-11-16T16:41:27.572 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping successful for: 2023-11-16T16:41:27.650 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:41:27 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-0[110160]: 2023-11-16T16:41:27.198+0000 7ff27c68a700 -1 osd.0 0 waiting for initial osdmap 2023-11-16T16:41:27.650 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:41:27 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-0[110160]: 2023-11-16T16:41:27.203+0000 7ff27647b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:41:27.990 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi125:vg_nvme/lv_4 2023-11-16T16:41:28.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:28 smithi125 ceph-mon[131076]: purged_snaps scrub starts 2023-11-16T16:41:28.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:28 smithi125 ceph-mon[131076]: purged_snaps scrub ok 2023-11-16T16:41:28.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:28 smithi125 ceph-mon[131076]: from='osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-11-16T16:41:28.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:28 smithi125 ceph-mon[131076]: osdmap e7: 1 total, 0 up, 1 in 2023-11-16T16:41:28.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:28.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:28.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:28 smithi093 ceph-mon[97863]: purged_snaps scrub starts 2023-11-16T16:41:28.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:28 smithi093 ceph-mon[97863]: purged_snaps scrub ok 2023-11-16T16:41:28.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:28 smithi093 ceph-mon[97863]: from='osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-11-16T16:41:28.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:28 smithi093 ceph-mon[97863]: osdmap e7: 1 total, 0 up, 1 in 2023-11-16T16:41:28.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:28.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:28.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:28 smithi159 ceph-mon[131528]: purged_snaps scrub starts 2023-11-16T16:41:28.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:28 smithi159 ceph-mon[131528]: purged_snaps scrub ok 2023-11-16T16:41:28.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:28 smithi159 ceph-mon[131528]: from='osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-11-16T16:41:28.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:28 smithi159 ceph-mon[131528]: osdmap e7: 1 total, 0 up, 1 in 2023-11-16T16:41:28.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:28.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:28.670 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:41:29.451 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748] boot 2023-11-16T16:41:29.451 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: osdmap e8: 1 total, 1 up, 1 in 2023-11-16T16:41:29.451 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:29.451 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: Detected new or changed devices on smithi093 2023-11-16T16:41:29.451 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:29.451 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:29.452 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:41:29.452 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi093 to 17271M 2023-11-16T16:41:29.452 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:29.452 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:29.452 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:29.452 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:29.452 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:29.452 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:41:29.452 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:29.453 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:29 smithi125 ceph-mon[131076]: pgmap v47: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:29.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748] boot 2023-11-16T16:41:29.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: osdmap e8: 1 total, 1 up, 1 in 2023-11-16T16:41:29.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:29.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: Detected new or changed devices on smithi093 2023-11-16T16:41:29.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:29.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:29.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:41:29.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi093 to 17271M 2023-11-16T16:41:29.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:29.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:29.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:29.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:29.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:29.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:41:29.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:29.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:29 smithi159 ceph-mon[131528]: pgmap v47: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:29.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: osd.0 [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748] boot 2023-11-16T16:41:29.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: osdmap e8: 1 total, 1 up, 1 in 2023-11-16T16:41:29.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:41:29.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: Detected new or changed devices on smithi093 2023-11-16T16:41:29.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:29.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:29.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:41:29.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi093 to 17271M 2023-11-16T16:41:29.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:29.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:29.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:29.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:29.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:29.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:41:29.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:29.653 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:29 smithi093 ceph-mon[97863]: pgmap v47: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:29.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.909+0000 7fe857efe700 1 -- 172.21.15.125:0/2335676769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8501003e0 msgr2=0x7fe850100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:29.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.909+0000 7fe857efe700 1 --2- 172.21.15.125:0/2335676769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8501003e0 0x7fe850100830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009990 tx=0x7fe84c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:41:29.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.909+0000 7fe857efe700 1 -- 172.21.15.125:0/2335676769 shutdown_connections 2023-11-16T16:41:29.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.909+0000 7fe857efe700 1 --2- 172.21.15.125:0/2335676769 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8500ff1e0 0x7fe8500ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:29.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.909+0000 7fe857efe700 1 --2- 172.21.15.125:0/2335676769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe850100d70 0x7fe850107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:29.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.909+0000 7fe857efe700 1 --2- 172.21.15.125:0/2335676769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8501003e0 0x7fe850100830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:29.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.909+0000 7fe857efe700 1 -- 172.21.15.125:0/2335676769 >> 172.21.15.125:0/2335676769 conn(0x7fe8500fa7f0 msgr2=0x7fe8500fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:29.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.909+0000 7fe857efe700 1 -- 172.21.15.125:0/2335676769 shutdown_connections 2023-11-16T16:41:29.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.909+0000 7fe857efe700 1 -- 172.21.15.125:0/2335676769 wait complete. 2023-11-16T16:41:29.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.909+0000 7fe857efe700 1 Processor -- start 2023-11-16T16:41:29.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.909+0000 7fe857efe700 1 -- start start 2023-11-16T16:41:29.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe857efe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8500ff1e0 0x7fe850196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:29.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe857efe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8501003e0 0x7fe8501965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:29.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe857efe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe850100d70 0x7fe85019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:29.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe857efe700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85019dc00 con 0x7fe850100d70 2023-11-16T16:41:29.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe857efe700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85019dd40 con 0x7fe8500ff1e0 2023-11-16T16:41:29.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe857efe700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85019ded0 con 0x7fe8501003e0 2023-11-16T16:41:29.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe855c9a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8500ff1e0 0x7fe850196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:29.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe855c9a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8500ff1e0 0x7fe850196070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:34552/0 (socket says 172.21.15.125:34552) 2023-11-16T16:41:29.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe855c9a700 1 -- 172.21.15.125:0/11219940 learned_addr learned my addr 172.21.15.125:0/11219940 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-11-16T16:41:29.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe85649b700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe850100d70 0x7fe85019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:29.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe855499700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8501003e0 0x7fe8501965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:29.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe855c9a700 1 -- 172.21.15.125:0/11219940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8501003e0 msgr2=0x7fe8501965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:29.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe855c9a700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8501003e0 0x7fe8501965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:29.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe855c9a700 1 -- 172.21.15.125:0/11219940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe850100d70 msgr2=0x7fe85019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:29.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe855c9a700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe850100d70 0x7fe85019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:29.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe855c9a700 1 -- 172.21.15.125:0/11219940 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c0095f0 con 0x7fe8500ff1e0 2023-11-16T16:41:29.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe85649b700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe850100d70 0x7fe85019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:41:29.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe855499700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8501003e0 0x7fe8501965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:41:29.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe855c9a700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8500ff1e0 0x7fe850196070 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe84000b500 tx=0x7fe84000b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:29.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe846ffd700 1 -- 172.21.15.125:0/11219940 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe840011830 con 0x7fe8500ff1e0 2023-11-16T16:41:29.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe846ffd700 1 -- 172.21.15.125:0/11219940 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe840011e50 con 0x7fe8500ff1e0 2023-11-16T16:41:29.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe846ffd700 1 -- 172.21.15.125:0/11219940 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84000f2e0 con 0x7fe8500ff1e0 2023-11-16T16:41:29.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe857efe700 1 -- 172.21.15.125:0/11219940 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85010abd0 con 0x7fe8500ff1e0 2023-11-16T16:41:29.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.910+0000 7fe857efe700 1 -- 172.21.15.125:0/11219940 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85010aa40 con 0x7fe8500ff1e0 2023-11-16T16:41:29.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.912+0000 7fe857efe700 1 -- 172.21.15.125:0/11219940 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe850066ea0 con 0x7fe8500ff1e0 2023-11-16T16:41:29.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.912+0000 7fe846ffd700 1 -- 172.21.15.125:0/11219940 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fe840010580 con 0x7fe8500ff1e0 2023-11-16T16:41:29.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.913+0000 7fe846ffd700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe83c0a1210 0x7fe83c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:29.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.913+0000 7fe846ffd700 1 -- 172.21.15.125:0/11219940 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fe8400ca820 con 0x7fe8500ff1e0 2023-11-16T16:41:29.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.913+0000 7fe855499700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe83c0a1210 0x7fe83c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:29.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.914+0000 7fe855499700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe83c0a1210 0x7fe83c0a36c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00b5c0 tx=0x7fe84c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:29.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:29.917+0000 7fe846ffd700 1 -- 172.21.15.125:0/11219940 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe8400959e0 con 0x7fe8500ff1e0 2023-11-16T16:41:30.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:30.111+0000 7fe857efe700 1 -- 172.21.15.125:0/11219940 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe85019ea30 con 0x7fe83c0a1210 2023-11-16T16:41:30.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:30 smithi159 ceph-mon[131528]: osdmap e9: 1 total, 1 up, 1 in 2023-11-16T16:41:30.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:30 smithi159 ceph-mon[131528]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:41:30.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:30 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:41:30.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:30 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:41:30.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:30 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:30.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:30 smithi125 ceph-mon[131076]: osdmap e9: 1 total, 1 up, 1 in 2023-11-16T16:41:30.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:30 smithi125 ceph-mon[131076]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:41:30.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:30 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:41:30.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:30 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:41:30.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:30 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:30 smithi093 ceph-mon[97863]: osdmap e9: 1 total, 1 up, 1 in 2023-11-16T16:41:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:30 smithi093 ceph-mon[97863]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:41:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:30 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:41:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:30 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:41:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:30 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:31.282 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:31 smithi125 ceph-mon[131076]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:31.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:31 smithi159 ceph-mon[131528]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:31 smithi093 ceph-mon[97863]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:33.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:33 smithi125 ceph-mon[131076]: pgmap v50: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:33.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:33 smithi125 ceph-mon[131076]: from='client.? 172.21.15.125:0/1471261841' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b72319a2-0c12-4b08-b78f-85d3af107bca"}]: dispatch 2023-11-16T16:41:33.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:33 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b72319a2-0c12-4b08-b78f-85d3af107bca"}]: dispatch 2023-11-16T16:41:33.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:33 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b72319a2-0c12-4b08-b78f-85d3af107bca"}]': finished 2023-11-16T16:41:33.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:33 smithi125 ceph-mon[131076]: osdmap e10: 2 total, 1 up, 2 in 2023-11-16T16:41:33.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:33.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:33 smithi125 ceph-mon[131076]: from='client.? 172.21.15.125:0/740175518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:41:33.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:33 smithi093 ceph-mon[97863]: pgmap v50: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:33.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:33 smithi093 ceph-mon[97863]: from='client.? 172.21.15.125:0/1471261841' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b72319a2-0c12-4b08-b78f-85d3af107bca"}]: dispatch 2023-11-16T16:41:33.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:33 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b72319a2-0c12-4b08-b78f-85d3af107bca"}]: dispatch 2023-11-16T16:41:33.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:33 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b72319a2-0c12-4b08-b78f-85d3af107bca"}]': finished 2023-11-16T16:41:33.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:33 smithi093 ceph-mon[97863]: osdmap e10: 2 total, 1 up, 2 in 2023-11-16T16:41:33.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:33.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:33 smithi093 ceph-mon[97863]: from='client.? 172.21.15.125:0/740175518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:41:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:33 smithi159 ceph-mon[131528]: pgmap v50: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:33 smithi159 ceph-mon[131528]: from='client.? 172.21.15.125:0/1471261841' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b72319a2-0c12-4b08-b78f-85d3af107bca"}]: dispatch 2023-11-16T16:41:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:33 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b72319a2-0c12-4b08-b78f-85d3af107bca"}]: dispatch 2023-11-16T16:41:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:33 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b72319a2-0c12-4b08-b78f-85d3af107bca"}]': finished 2023-11-16T16:41:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:33 smithi159 ceph-mon[131528]: osdmap e10: 2 total, 1 up, 2 in 2023-11-16T16:41:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:33 smithi159 ceph-mon[131528]: from='client.? 172.21.15.125:0/740175518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:41:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:35 smithi125 ceph-mon[131076]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:35.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:35 smithi093 ceph-mon[97863]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:35.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:35 smithi159 ceph-mon[131528]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:37.814 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:37 smithi125 ceph-mon[131076]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:37 smithi093 ceph-mon[97863]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:37 smithi159 ceph-mon[131528]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:39.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:39 smithi125 ceph-mon[131076]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:39.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-16T16:41:39.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:39 smithi093 ceph-mon[97863]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:39.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-16T16:41:39.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:39.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:39 smithi159 ceph-mon[131528]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:39.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-16T16:41:39.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:40.820 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:40 smithi125 ceph-mon[131076]: Deploying daemon osd.1 on smithi125 2023-11-16T16:41:40.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:40 smithi093 ceph-mon[97863]: Deploying daemon osd.1 on smithi125 2023-11-16T16:41:40.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:40 smithi159 ceph-mon[131528]: Deploying daemon osd.1 on smithi125 2023-11-16T16:41:41.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:41 smithi125 ceph-mon[131076]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:41.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:41 smithi093 ceph-mon[97863]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:41.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:41 smithi159 ceph-mon[131528]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:43.780 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:43 smithi125 ceph-mon[131076]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:43.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:43 smithi093 ceph-mon[97863]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:43.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:43 smithi159 ceph-mon[131528]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:45.062 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:44 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:45.063 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:44 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:45.063 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:44 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:45.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:44 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:45.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:44 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:45.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:44 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:45.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:44 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:45.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:44 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:45.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:44 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:46.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:45 smithi125 ceph-mon[131076]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:46.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:45 smithi093 ceph-mon[97863]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:46.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:45 smithi159 ceph-mon[131528]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:46.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.724+0000 7fe846ffd700 1 -- 172.21.15.125:0/11219940 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe85019ea30 con 0x7fe83c0a1210 2023-11-16T16:41:46.727 INFO:teuthology.orchestra.run.smithi125.stdout:Created osd(s) 1 on host 'smithi125' 2023-11-16T16:41:46.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.726+0000 7fe857efe700 1 -- 172.21.15.125:0/11219940 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe83c0a1210 msgr2=0x7fe83c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:46.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.726+0000 7fe857efe700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe83c0a1210 0x7fe83c0a36c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00b5c0 tx=0x7fe84c005fb0 comp rx=0 tx=0).stop 2023-11-16T16:41:46.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.726+0000 7fe857efe700 1 -- 172.21.15.125:0/11219940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8500ff1e0 msgr2=0x7fe850196070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:46.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.726+0000 7fe857efe700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8500ff1e0 0x7fe850196070 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe84000b500 tx=0x7fe84000b8c0 comp rx=0 tx=0).stop 2023-11-16T16:41:46.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.726+0000 7fe857efe700 1 -- 172.21.15.125:0/11219940 shutdown_connections 2023-11-16T16:41:46.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.726+0000 7fe857efe700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe83c0a1210 0x7fe83c0a36c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:46.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.726+0000 7fe857efe700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe850100d70 0x7fe85019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:46.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.726+0000 7fe857efe700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8500ff1e0 0x7fe850196070 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:46.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.726+0000 7fe857efe700 1 --2- 172.21.15.125:0/11219940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8501003e0 0x7fe8501965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:46.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.727+0000 7fe857efe700 1 -- 172.21.15.125:0/11219940 >> 172.21.15.125:0/11219940 conn(0x7fe8500fa7f0 msgr2=0x7fe850068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:46.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.727+0000 7fe857efe700 1 -- 172.21.15.125:0/11219940 shutdown_connections 2023-11-16T16:41:46.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:41:46.727+0000 7fe857efe700 1 -- 172.21.15.125:0/11219940 wait complete. 2023-11-16T16:41:47.158 DEBUG:teuthology.orchestra.run.smithi125:osd.1> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.1.service 2023-11-16T16:41:47.160 INFO:tasks.cephadm:Deploying osd.2 on smithi159 with /dev/vg_nvme/lv_4... 2023-11-16T16:41:47.160 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-16T16:41:47.191 INFO:journalctl@ceph.osd.1.smithi125.stdout:-- Logs begin at Thu 2023-11-16 16:14:41 UTC. -- 2023-11-16T16:41:47.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:47.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:47.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:47 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:47.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:47.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:47.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:47.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:47 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:47.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:47.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.543 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.543 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.543 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:47.543 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:47.544 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:47 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:47.544 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.544 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.544 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:47.544 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:47.809 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:41:47.828 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:41:47 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-1[136185]: 2023-11-16T16:41:47.540+0000 7f3e5f365640 -1 osd.1 0 log_to_monitors true 2023-11-16T16:41:48.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:48 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:41:48.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:48 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:48.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:48 smithi093 ceph-mon[97863]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:48.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:48 smithi093 ceph-mon[97863]: from='osd.1 [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-16T16:41:48.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:48 smithi093 ceph-mon[97863]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-16T16:41:48.437 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:48 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:41:48.437 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:48 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:48.437 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:48 smithi159 ceph-mon[131528]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:48.437 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:48 smithi159 ceph-mon[131528]: from='osd.1 [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-16T16:41:48.437 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:48 smithi159 ceph-mon[131528]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-16T16:41:48.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:48 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:41:48.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:48 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:48.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:48 smithi125 ceph-mon[131076]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:48.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:48 smithi125 ceph-mon[131076]: from='osd.1 [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-16T16:41:48.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:48 smithi125 ceph-mon[131076]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-16T16:41:49.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:49 smithi159 ceph-mon[131528]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-16T16:41:49.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:49 smithi159 ceph-mon[131528]: osdmap e11: 2 total, 1 up, 2 in 2023-11-16T16:41:49.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:49 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:49.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:49 smithi159 ceph-mon[131528]: from='osd.1 [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:41:49.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:49 smithi159 ceph-mon[131528]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:41:49.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:49 smithi159 ceph-mon[131528]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:49.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:49 smithi093 ceph-mon[97863]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-16T16:41:49.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:49 smithi093 ceph-mon[97863]: osdmap e11: 2 total, 1 up, 2 in 2023-11-16T16:41:49.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:49 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:49.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:49 smithi093 ceph-mon[97863]: from='osd.1 [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:41:49.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:49 smithi093 ceph-mon[97863]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:41:49.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:49 smithi093 ceph-mon[97863]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:49.403 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-16T16:41:49.403 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-16T16:41:49.403 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2023-11-16T16:41:49.403 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2023-11-16T16:41:49.403 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110813 s, 946 MB/s 2023-11-16T16:41:49.403 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2023-11-16T16:41:49.404 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:49 smithi125 ceph-mon[131076]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-16T16:41:49.404 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:49 smithi125 ceph-mon[131076]: osdmap e11: 2 total, 1 up, 2 in 2023-11-16T16:41:49.404 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:49 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:49.405 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:49 smithi125 ceph-mon[131076]: from='osd.1 [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:41:49.405 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:49 smithi125 ceph-mon[131076]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:41:49.405 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:49 smithi125 ceph-mon[131076]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:49.405 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:41:49 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-1[136185]: 2023-11-16T16:41:49.139+0000 7f3e549dc700 -1 osd.1 0 waiting for initial osdmap 2023-11-16T16:41:49.405 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:41:49 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-1[136185]: 2023-11-16T16:41:49.142+0000 7f3e4d7cb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:41:49.765 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi159:vg_nvme/lv_4 2023-11-16T16:41:50.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2023-11-16T16:41:50.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: osdmap e12: 2 total, 1 up, 2 in 2023-11-16T16:41:50.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:50.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:50.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: Detected new or changed devices on smithi125 2023-11-16T16:41:50.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:50.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:50.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:41:50.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi125 to 17168M 2023-11-16T16:41:50.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:50.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:50.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:50.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:50.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:50.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:41:50.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:50 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:41:50.405 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:41:50.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2023-11-16T16:41:50.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: osdmap e12: 2 total, 1 up, 2 in 2023-11-16T16:41:50.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:50.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:50.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: Detected new or changed devices on smithi125 2023-11-16T16:41:50.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:50.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:50.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:41:50.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi125 to 17168M 2023-11-16T16:41:50.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:50.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:50.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:50.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:50.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:50.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:41:50.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:50 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:41:50.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2023-11-16T16:41:50.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: osdmap e12: 2 total, 1 up, 2 in 2023-11-16T16:41:50.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:50.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:50.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: Detected new or changed devices on smithi125 2023-11-16T16:41:50.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:50.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:50.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:41:50.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi125 to 17168M 2023-11-16T16:41:50.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:50.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:50.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:50.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:50.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:41:50.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:41:50.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:50 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:41:51.181 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:51 smithi159 ceph-mon[131528]: purged_snaps scrub starts 2023-11-16T16:41:51.181 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:51 smithi159 ceph-mon[131528]: purged_snaps scrub ok 2023-11-16T16:41:51.181 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:51 smithi159 ceph-mon[131528]: osd.1 [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115] boot 2023-11-16T16:41:51.181 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:51 smithi159 ceph-mon[131528]: osdmap e13: 2 total, 2 up, 2 in 2023-11-16T16:41:51.182 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:51.182 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:51 smithi159 ceph-mon[131528]: pgmap v63: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:51.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:51 smithi093 ceph-mon[97863]: purged_snaps scrub starts 2023-11-16T16:41:51.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:51 smithi093 ceph-mon[97863]: purged_snaps scrub ok 2023-11-16T16:41:51.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:51 smithi093 ceph-mon[97863]: osd.1 [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115] boot 2023-11-16T16:41:51.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:51 smithi093 ceph-mon[97863]: osdmap e13: 2 total, 2 up, 2 in 2023-11-16T16:41:51.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:51.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:51 smithi093 ceph-mon[97863]: pgmap v63: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:51 smithi125 ceph-mon[131076]: purged_snaps scrub starts 2023-11-16T16:41:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:51 smithi125 ceph-mon[131076]: purged_snaps scrub ok 2023-11-16T16:41:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:51 smithi125 ceph-mon[131076]: osd.1 [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115] boot 2023-11-16T16:41:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:51 smithi125 ceph-mon[131076]: osdmap e13: 2 total, 2 up, 2 in 2023-11-16T16:41:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:41:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:51 smithi125 ceph-mon[131076]: pgmap v63: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:41:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.627+0000 7fb390b5d700 1 -- 172.21.15.159:0/2695712301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb38c0ffb60 msgr2=0x7fb38c0fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.627+0000 7fb390b5d700 1 --2- 172.21.15.159:0/2695712301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb38c0ffb60 0x7fb38c0fffb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009990 tx=0x7fb37c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:41:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.627+0000 7fb390b5d700 1 -- 172.21.15.159:0/2695712301 shutdown_connections 2023-11-16T16:41:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.627+0000 7fb390b5d700 1 --2- 172.21.15.159:0/2695712301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb38c0fe960 0x7fb38c0fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.627+0000 7fb390b5d700 1 --2- 172.21.15.159:0/2695712301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb38c1004f0 0x7fb38c106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.627+0000 7fb390b5d700 1 --2- 172.21.15.159:0/2695712301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb38c0ffb60 0x7fb38c0fffb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.627+0000 7fb390b5d700 1 -- 172.21.15.159:0/2695712301 >> 172.21.15.159:0/2695712301 conn(0x7fb38c0f9f70 msgr2=0x7fb38c0fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:51.630 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.627+0000 7fb390b5d700 1 -- 172.21.15.159:0/2695712301 shutdown_connections 2023-11-16T16:41:51.630 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.627+0000 7fb390b5d700 1 -- 172.21.15.159:0/2695712301 wait complete. 2023-11-16T16:41:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.628+0000 7fb390b5d700 1 Processor -- start 2023-11-16T16:41:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.628+0000 7fb390b5d700 1 -- start start 2023-11-16T16:41:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb390b5d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb38c0fe960 0x7fb38c194ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb390b5d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb38c0ffb60 0x7fb38c195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb390b5d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb38c1004f0 0x7fb38c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb390b5d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c19ca80 con 0x7fb38c0fe960 2023-11-16T16:41:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb390b5d700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c19cbc0 con 0x7fb38c1004f0 2023-11-16T16:41:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb390b5d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c19cd50 con 0x7fb38c0ffb60 2023-11-16T16:41:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb389d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb38c0ffb60 0x7fb38c195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb389d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb38c0ffb60 0x7fb38c195430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54064/0 (socket says 172.21.15.159:54064) 2023-11-16T16:41:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb389d9b700 1 -- 172.21.15.159:0/2499123115 learned_addr learned my addr 172.21.15.159:0/2499123115 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-11-16T16:41:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb38ad9d700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb38c1004f0 0x7fb38c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb389d9b700 1 -- 172.21.15.159:0/2499123115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb38c1004f0 msgr2=0x7fb38c19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb389d9b700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb38c1004f0 0x7fb38c19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb389d9b700 1 -- 172.21.15.159:0/2499123115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb38c0fe960 msgr2=0x7fb38c194ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb389d9b700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb38c0fe960 0x7fb38c194ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb389d9b700 1 -- 172.21.15.159:0/2499123115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c0095f0 con 0x7fb38c0ffb60 2023-11-16T16:41:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb389d9b700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb38c0ffb60 0x7fb38c195430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb37c005910 tx=0x7fb37c01a9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb37b7fe700 1 -- 172.21.15.159:0/2499123115 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c01f070 con 0x7fb38c0ffb60 2023-11-16T16:41:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb37b7fe700 1 -- 172.21.15.159:0/2499123115 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb37c01acb0 con 0x7fb38c0ffb60 2023-11-16T16:41:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb390b5d700 1 -- 172.21.15.159:0/2499123115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c10ab30 con 0x7fb38c0ffb60 2023-11-16T16:41:51.634 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.629+0000 7fb37b7fe700 1 -- 172.21.15.159:0/2499123115 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c025680 con 0x7fb38c0ffb60 2023-11-16T16:41:51.634 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.630+0000 7fb390b5d700 1 -- 172.21.15.159:0/2499123115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c10a9a0 con 0x7fb38c0ffb60 2023-11-16T16:41:51.634 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.631+0000 7fb390b5d700 1 -- 172.21.15.159:0/2499123115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c1082c0 con 0x7fb38c0ffb60 2023-11-16T16:41:51.634 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.631+0000 7fb37b7fe700 1 -- 172.21.15.159:0/2499123115 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fb37c023070 con 0x7fb38c0ffb60 2023-11-16T16:41:51.635 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.632+0000 7fb37b7fe700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb3740a1140 0x7fb3740a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:51.635 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.632+0000 7fb37b7fe700 1 -- 172.21.15.159:0/2499123115 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fb37c0ce3a0 con 0x7fb38c0ffb60 2023-11-16T16:41:51.635 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.632+0000 7fb38a59c700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb3740a1140 0x7fb3740a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:51.635 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.633+0000 7fb38a59c700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb3740a1140 0x7fb3740a35f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0ff9c0 tx=0x7fb380005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:51.637 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.636+0000 7fb37b7fe700 1 -- 172.21.15.159:0/2499123115 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb37c0993d0 con 0x7fb38c0ffb60 2023-11-16T16:41:51.831 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:41:51.830+0000 7fb390b5d700 1 -- 172.21.15.159:0/2499123115 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb38c105ed0 con 0x7fb3740a1140 2023-11-16T16:41:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:52 smithi159 ceph-mon[131528]: osdmap e14: 2 total, 2 up, 2 in 2023-11-16T16:41:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:52 smithi159 ceph-mon[131528]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:41:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:41:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:41:52.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:52 smithi125 ceph-mon[131076]: osdmap e14: 2 total, 2 up, 2 in 2023-11-16T16:41:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:52 smithi125 ceph-mon[131076]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:41:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:41:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:41:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:52 smithi093 ceph-mon[97863]: osdmap e14: 2 total, 2 up, 2 in 2023-11-16T16:41:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:52 smithi093 ceph-mon[97863]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:41:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:41:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:41:52.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:53.281 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:53 smithi159 ceph-mon[131528]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:41:53.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:53 smithi125 ceph-mon[131076]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:41:53.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:53 smithi093 ceph-mon[97863]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:41:54.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:54 smithi093 ceph-mon[97863]: from='client.? 172.21.15.159:0/367027717' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d17be2a-b82b-4a45-9015-c451f7884bc5"}]: dispatch 2023-11-16T16:41:54.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:54 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d17be2a-b82b-4a45-9015-c451f7884bc5"}]: dispatch 2023-11-16T16:41:54.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:54 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d17be2a-b82b-4a45-9015-c451f7884bc5"}]': finished 2023-11-16T16:41:54.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:54 smithi093 ceph-mon[97863]: osdmap e15: 3 total, 2 up, 3 in 2023-11-16T16:41:54.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:54 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:41:54.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:54 smithi159 ceph-mon[131528]: from='client.? 172.21.15.159:0/367027717' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d17be2a-b82b-4a45-9015-c451f7884bc5"}]: dispatch 2023-11-16T16:41:54.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:54 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d17be2a-b82b-4a45-9015-c451f7884bc5"}]: dispatch 2023-11-16T16:41:54.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:54 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d17be2a-b82b-4a45-9015-c451f7884bc5"}]': finished 2023-11-16T16:41:54.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:54 smithi159 ceph-mon[131528]: osdmap e15: 3 total, 2 up, 3 in 2023-11-16T16:41:54.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:54 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:41:54.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:54 smithi125 ceph-mon[131076]: from='client.? 172.21.15.159:0/367027717' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d17be2a-b82b-4a45-9015-c451f7884bc5"}]: dispatch 2023-11-16T16:41:54.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:54 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d17be2a-b82b-4a45-9015-c451f7884bc5"}]: dispatch 2023-11-16T16:41:54.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:54 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d17be2a-b82b-4a45-9015-c451f7884bc5"}]': finished 2023-11-16T16:41:54.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:54 smithi125 ceph-mon[131076]: osdmap e15: 3 total, 2 up, 3 in 2023-11-16T16:41:54.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:54 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:41:55.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:55 smithi093 ceph-mon[97863]: pgmap v67: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:41:55.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:55 smithi093 ceph-mon[97863]: from='client.? 172.21.15.159:0/4053263201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:41:55.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:55 smithi159 ceph-mon[131528]: pgmap v67: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:41:55.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:55 smithi159 ceph-mon[131528]: from='client.? 172.21.15.159:0/4053263201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:41:55.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:55 smithi125 ceph-mon[131076]: pgmap v67: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:41:55.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:55 smithi125 ceph-mon[131076]: from='client.? 172.21.15.159:0/4053263201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:41:57.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:57 smithi125 ceph-mon[131076]: pgmap v68: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:41:57.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:57 smithi093 ceph-mon[97863]: pgmap v68: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:41:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:57 smithi159 ceph-mon[131528]: pgmap v68: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:41:59.660 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:41:59 smithi159 ceph-mon[131528]: pgmap v69: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:41:59.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:41:59 smithi125 ceph-mon[131076]: pgmap v69: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:41:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:41:59 smithi093 ceph-mon[97863]: pgmap v69: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:01.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:01 smithi159 ceph-mon[131528]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:01.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-16T16:42:01.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:01.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:01 smithi159 ceph-mon[131528]: Deploying daemon osd.2 on smithi159 2023-11-16T16:42:01.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:01 smithi125 ceph-mon[131076]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:01.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-16T16:42:01.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:01.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:01 smithi125 ceph-mon[131076]: Deploying daemon osd.2 on smithi159 2023-11-16T16:42:01.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:01 smithi093 ceph-mon[97863]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:01.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-16T16:42:01.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:01.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:01 smithi093 ceph-mon[97863]: Deploying daemon osd.2 on smithi159 2023-11-16T16:42:03.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:03 smithi159 ceph-mon[131528]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:03.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:03 smithi125 ceph-mon[131076]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:03.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:03 smithi093 ceph-mon[97863]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:05.806 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:05 smithi159 ceph-mon[131528]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:05.806 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:05 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:05.806 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:05 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:05.806 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:05 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:05.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:05 smithi125 ceph-mon[131076]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:05.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:05 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:05.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:05 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:05.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:05 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:05.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:05 smithi093 ceph-mon[97863]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:05.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:05 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:05.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:05 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:05.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:05 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:07.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:07 smithi159 ceph-mon[131528]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:07.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:07 smithi125 ceph-mon[131076]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:07.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:07 smithi093 ceph-mon[97863]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:08.299 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.297+0000 7fb37b7fe700 1 -- 172.21.15.159:0/2499123115 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb38c105ed0 con 0x7fb3740a1140 2023-11-16T16:42:08.299 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 2 on host 'smithi159' 2023-11-16T16:42:08.301 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.300+0000 7fb390b5d700 1 -- 172.21.15.159:0/2499123115 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb3740a1140 msgr2=0x7fb3740a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.301 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.300+0000 7fb390b5d700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb3740a1140 0x7fb3740a35f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0ff9c0 tx=0x7fb380005ca0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.302 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.300+0000 7fb390b5d700 1 -- 172.21.15.159:0/2499123115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb38c0ffb60 msgr2=0x7fb38c195430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.302 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.300+0000 7fb390b5d700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb38c0ffb60 0x7fb38c195430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb37c005910 tx=0x7fb37c01a9e0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.302 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.300+0000 7fb390b5d700 1 -- 172.21.15.159:0/2499123115 shutdown_connections 2023-11-16T16:42:08.302 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.300+0000 7fb390b5d700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb3740a1140 0x7fb3740a35f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.302 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.300+0000 7fb390b5d700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb38c0fe960 0x7fb38c194ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.302 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.300+0000 7fb390b5d700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb38c1004f0 0x7fb38c19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.302 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.300+0000 7fb390b5d700 1 --2- 172.21.15.159:0/2499123115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb38c0ffb60 0x7fb38c195430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.302 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.300+0000 7fb390b5d700 1 -- 172.21.15.159:0/2499123115 >> 172.21.15.159:0/2499123115 conn(0x7fb38c0f9f70 msgr2=0x7fb38c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.302 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.300+0000 7fb390b5d700 1 -- 172.21.15.159:0/2499123115 shutdown_connections 2023-11-16T16:42:08.302 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:42:08.300+0000 7fb390b5d700 1 -- 172.21.15.159:0/2499123115 wait complete. 2023-11-16T16:42:08.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:08 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:08.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:08 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:08.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:08 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:08.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:08 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:08.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:08 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:42:08.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:08 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:08.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:08 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:42:08.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:08 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:42:08.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:08 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:08.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:08 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:08.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:08 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:08.786 DEBUG:teuthology.orchestra.run.smithi159:osd.2> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.2.service 2023-11-16T16:42:08.788 INFO:tasks.cephadm:Deploying osd.3 on smithi093 with /dev/vg_nvme/lv_3... 2023-11-16T16:42:08.789 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-16T16:42:08.819 INFO:journalctl@ceph.osd.2.smithi159.stdout:-- Logs begin at Thu 2023-11-16 16:15:38 UTC. -- 2023-11-16T16:42:09.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:08 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:09.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:08 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:09.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:08 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:09.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:08 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:09.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:08 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:42:09.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:08 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:09.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:08 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:42:09.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:08 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:42:09.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:08 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:09.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:08 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:09.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:08 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:09.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:08 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:09.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:08 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:09.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:08 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:09.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:08 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:09.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:08 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:42:09.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:08 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:09.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:08 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:42:09.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:08 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:42:09.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:08 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:09.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:08 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:09.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:08 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:09.575 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:42:09 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-2[137311]: 2023-11-16T16:42:09.352+0000 7fdadcdef640 -1 osd.2 0 log_to_monitors true 2023-11-16T16:42:09.646 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:42:09.853 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:09 smithi159 ceph-mon[131528]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:09.853 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:09 smithi159 ceph-mon[131528]: from='osd.2 [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-16T16:42:09.853 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:09 smithi159 ceph-mon[131528]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-16T16:42:09.914 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:09 smithi093 ceph-mon[97863]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:09.914 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:09 smithi093 ceph-mon[97863]: from='osd.2 [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-16T16:42:09.914 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:09 smithi093 ceph-mon[97863]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-16T16:42:10.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:09 smithi125 ceph-mon[131076]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:10.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:09 smithi125 ceph-mon[131076]: from='osd.2 [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-16T16:42:10.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:09 smithi125 ceph-mon[131076]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-16T16:42:11.485 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-2[137311]: 2023-11-16T16:42:11.300+0000 7fdad2466700 -1 osd.2 0 waiting for initial osdmap 2023-11-16T16:42:11.486 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-2[137311]: 2023-11-16T16:42:11.304+0000 7fdacd259700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:42:11.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-16T16:42:11.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: osdmap e16: 3 total, 2 up, 3 in 2023-11-16T16:42:11.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:42:11.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: from='osd.2 [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:42:11.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:42:11.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: pgmap v76: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:11.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:11.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:11.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:42:11.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:11.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:11.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:11.487 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:11.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-16T16:42:11.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: osdmap e16: 3 total, 2 up, 3 in 2023-11-16T16:42:11.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:42:11.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: from='osd.2 [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:42:11.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:42:11.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: pgmap v76: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:11.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:11.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:11.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:42:11.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:11.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:11.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:11.580 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:11.617 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-16T16:42:11.618 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-16T16:42:11.618 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-11-16T16:42:11.618 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-11-16T16:42:11.618 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0305368 s, 343 MB/s 2023-11-16T16:42:11.618 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-11-16T16:42:11.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-16T16:42:11.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: osdmap e16: 3 total, 2 up, 3 in 2023-11-16T16:42:11.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:42:11.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: from='osd.2 [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:42:11.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:42:11.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: pgmap v76: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:11.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:11.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:11.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:42:11.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:11.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:11.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:11.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:12.222 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi093:vg_nvme/lv_3 2023-11-16T16:42:12.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:12 smithi125 ceph-mon[131076]: Detected new or changed devices on smithi159 2023-11-16T16:42:12.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:12 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi159 to 17168M 2023-11-16T16:42:12.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:12 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:42:12.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:12 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:42:12.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:12 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:42:12.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:12 smithi125 ceph-mon[131076]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-11-16T16:42:12.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:12 smithi125 ceph-mon[131076]: osdmap e17: 3 total, 2 up, 3 in 2023-11-16T16:42:12.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:42:12.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:42:12.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:12 smithi093 ceph-mon[97863]: Detected new or changed devices on smithi159 2023-11-16T16:42:12.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:12 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi159 to 17168M 2023-11-16T16:42:12.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:12 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:42:12.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:12 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:42:12.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:12 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:42:12.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:12 smithi093 ceph-mon[97863]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-11-16T16:42:12.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:12 smithi093 ceph-mon[97863]: osdmap e17: 3 total, 2 up, 3 in 2023-11-16T16:42:12.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:42:12.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:42:12.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:12 smithi159 ceph-mon[131528]: Detected new or changed devices on smithi159 2023-11-16T16:42:12.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:12 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi159 to 17168M 2023-11-16T16:42:12.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:12 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:42:12.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:12 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:42:12.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:12 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:42:12.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:12 smithi159 ceph-mon[131528]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-11-16T16:42:12.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:12 smithi159 ceph-mon[131528]: osdmap e17: 3 total, 2 up, 3 in 2023-11-16T16:42:12.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:42:12.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:42:13.088 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:42:13.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:13 smithi125 ceph-mon[131076]: purged_snaps scrub starts 2023-11-16T16:42:13.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:13 smithi125 ceph-mon[131076]: purged_snaps scrub ok 2023-11-16T16:42:13.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:13 smithi125 ceph-mon[131076]: osd.2 [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992] boot 2023-11-16T16:42:13.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:13 smithi125 ceph-mon[131076]: osdmap e18: 3 total, 3 up, 3 in 2023-11-16T16:42:13.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:13 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:42:13.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:13 smithi125 ceph-mon[131076]: pgmap v79: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:13.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:13 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:42:13.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:13 smithi093 ceph-mon[97863]: purged_snaps scrub starts 2023-11-16T16:42:13.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:13 smithi093 ceph-mon[97863]: purged_snaps scrub ok 2023-11-16T16:42:13.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:13 smithi093 ceph-mon[97863]: osd.2 [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992] boot 2023-11-16T16:42:13.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:13 smithi093 ceph-mon[97863]: osdmap e18: 3 total, 3 up, 3 in 2023-11-16T16:42:13.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:13 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:42:13.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:13 smithi093 ceph-mon[97863]: pgmap v79: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:13.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:13 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:42:13.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:13 smithi159 ceph-mon[131528]: purged_snaps scrub starts 2023-11-16T16:42:13.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:13 smithi159 ceph-mon[131528]: purged_snaps scrub ok 2023-11-16T16:42:13.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:13 smithi159 ceph-mon[131528]: osd.2 [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992] boot 2023-11-16T16:42:13.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:13 smithi159 ceph-mon[131528]: osdmap e18: 3 total, 3 up, 3 in 2023-11-16T16:42:13.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:13 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:42:13.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:13 smithi159 ceph-mon[131528]: pgmap v79: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:42:13.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:13 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:42:14.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:14 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-16T16:42:14.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:14 smithi125 ceph-mon[131076]: osdmap e19: 3 total, 3 up, 3 in 2023-11-16T16:42:14.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:14 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:42:14.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.628+0000 7fb45f3b6700 1 -- 172.21.15.93:0/291235024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb458074b20 msgr2=0x7fb458072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:14.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.628+0000 7fb45f3b6700 1 --2- 172.21.15.93:0/291235024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb458074b20 0x7fb458072f80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb454009990 tx=0x7fb454009ca0 comp rx=0 tx=0).stop 2023-11-16T16:42:14.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.628+0000 7fb45f3b6700 1 -- 172.21.15.93:0/291235024 shutdown_connections 2023-11-16T16:42:14.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.628+0000 7fb45f3b6700 1 --2- 172.21.15.93:0/291235024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb45810c8d0 0x7fb45810ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:14.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.628+0000 7fb45f3b6700 1 --2- 172.21.15.93:0/291235024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb458074b20 0x7fb458072f80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:14.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.628+0000 7fb45f3b6700 1 --2- 172.21.15.93:0/291235024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb458073550 0x7fb4580739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:14.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.628+0000 7fb45f3b6700 1 -- 172.21.15.93:0/291235024 >> 172.21.15.93:0/291235024 conn(0x7fb4580fb900 msgr2=0x7fb4580fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:14.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.628+0000 7fb45f3b6700 1 -- 172.21.15.93:0/291235024 shutdown_connections 2023-11-16T16:42:14.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.628+0000 7fb45f3b6700 1 -- 172.21.15.93:0/291235024 wait complete. 2023-11-16T16:42:14.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.629+0000 7fb45f3b6700 1 Processor -- start 2023-11-16T16:42:14.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.629+0000 7fb45f3b6700 1 -- start start 2023-11-16T16:42:14.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45f3b6700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb458073550 0x7fb45819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:14.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45f3b6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074b20 0x7fb45819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:14.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45f3b6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb45810c8d0 0x7fb4581a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:14.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45f3b6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581a4200 con 0x7fb45810c8d0 2023-11-16T16:42:14.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45f3b6700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581a4340 con 0x7fb458073550 2023-11-16T16:42:14.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45f3b6700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581a44d0 con 0x7fb458074b20 2023-11-16T16:42:14.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45d953700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb45810c8d0 0x7fb4581a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:14.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45d152700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb458073550 0x7fb45819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:14.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45d953700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb45810c8d0 0x7fb4581a3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43430/0 (socket says 172.21.15.93:43430) 2023-11-16T16:42:14.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45d152700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb458073550 0x7fb45819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:50714/0 (socket says 172.21.15.93:50714) 2023-11-16T16:42:14.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45d953700 1 -- 172.21.15.93:0/3506384663 learned_addr learned my addr 172.21.15.93:0/3506384663 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:42:14.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45c951700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074b20 0x7fb45819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:14.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45d152700 1 -- 172.21.15.93:0/3506384663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074b20 msgr2=0x7fb45819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:14.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45d152700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074b20 0x7fb45819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:14.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45d152700 1 -- 172.21.15.93:0/3506384663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb45810c8d0 msgr2=0x7fb4581a3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:14.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45d152700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb45810c8d0 0x7fb4581a3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:14.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45d152700 1 -- 172.21.15.93:0/3506384663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4540095f0 con 0x7fb458073550 2023-11-16T16:42:14.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.630+0000 7fb45c951700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074b20 0x7fb45819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:14.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.631+0000 7fb45d152700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb458073550 0x7fb45819c670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb45400b680 tx=0x7fb454009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:14.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.631+0000 7fb44e7fc700 1 -- 172.21.15.93:0/3506384663 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45401d070 con 0x7fb458073550 2023-11-16T16:42:14.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.631+0000 7fb45f3b6700 1 -- 172.21.15.93:0/3506384663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4581122a0 con 0x7fb458073550 2023-11-16T16:42:14.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.631+0000 7fb45f3b6700 1 -- 172.21.15.93:0/3506384663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb458112110 con 0x7fb458073550 2023-11-16T16:42:14.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.631+0000 7fb44e7fc700 1 -- 172.21.15.93:0/3506384663 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb454021410 con 0x7fb458073550 2023-11-16T16:42:14.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.631+0000 7fb44e7fc700 1 -- 172.21.15.93:0/3506384663 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454017650 con 0x7fb458073550 2023-11-16T16:42:14.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.632+0000 7fb45f3b6700 1 -- 172.21.15.93:0/3506384663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb458102420 con 0x7fb458073550 2023-11-16T16:42:14.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.634+0000 7fb44e7fc700 1 -- 172.21.15.93:0/3506384663 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fb4540217f0 con 0x7fb458073550 2023-11-16T16:42:14.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.636+0000 7fb44e7fc700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb4440a1140 0x7fb4440a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:14.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.637+0000 7fb44e7fc700 1 -- 172.21.15.93:0/3506384663 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7fb4540cc610 con 0x7fb458073550 2023-11-16T16:42:14.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.637+0000 7fb45c951700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb4440a1140 0x7fb4440a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:14.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.637+0000 7fb45c951700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb4440a1140 0x7fb4440a35f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb448011490 tx=0x7fb448009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:14.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.640+0000 7fb44e7fc700 1 -- 172.21.15.93:0/3506384663 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb454097200 con 0x7fb458073550 2023-11-16T16:42:14.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:14 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-16T16:42:14.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:14 smithi093 ceph-mon[97863]: osdmap e19: 3 total, 3 up, 3 in 2023-11-16T16:42:14.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:14 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:42:14.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:14 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-16T16:42:14.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:14 smithi159 ceph-mon[131528]: osdmap e19: 3 total, 3 up, 3 in 2023-11-16T16:42:14.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:14 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:42:14.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:14.845+0000 7fb45f3b6700 1 -- 172.21.15.93:0/3506384663 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb458061c70 con 0x7fb4440a1140 2023-11-16T16:42:15.327 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:42:14 smithi093 sudo[113214]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-16T16:42:15.327 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:42:14 smithi093 sudo[113214]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:42:15.327 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:42:14 smithi093 sudo[113214]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:42:15.327 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:42:14 smithi093 sudo[113214]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:42:15.334 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:15.331+0000 7fb44e7fc700 1 -- 172.21.15.93:0/3506384663 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb4540975d0 con 0x7fb458073550 2023-11-16T16:42:15.650 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:42:15 smithi093 sudo[113272]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-16T16:42:15.651 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:42:15 smithi093 sudo[113272]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:42:15.651 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:42:15 smithi093 sudo[113272]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:42:15.651 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:42:15 smithi093 sudo[113272]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:42:15.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:15 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-16T16:42:15.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:15 smithi093 ceph-mon[97863]: osdmap e20: 3 total, 3 up, 3 in 2023-11-16T16:42:15.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:15 smithi093 ceph-mon[97863]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:15.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:15 smithi093 ceph-mon[97863]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:42:15.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:15 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:42:15.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:15 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:42:15.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:15 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:15.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:15 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-16T16:42:15.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:15 smithi159 ceph-mon[131528]: osdmap e20: 3 total, 3 up, 3 in 2023-11-16T16:42:15.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:15 smithi159 ceph-mon[131528]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:15.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:15 smithi159 ceph-mon[131528]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:42:15.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:15 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:42:15.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:15 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:42:15.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:15 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:15.784 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:15 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-16T16:42:15.784 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:15 smithi125 ceph-mon[131076]: osdmap e20: 3 total, 3 up, 3 in 2023-11-16T16:42:15.784 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:15 smithi125 ceph-mon[131076]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:15.784 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:15 smithi125 ceph-mon[131076]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:42:15.785 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:15 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:42:15.785 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:15 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:42:15.785 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:15 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:16.038 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:42:15 smithi125 sudo[138398]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-16T16:42:16.039 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:42:15 smithi125 sudo[138398]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:42:16.039 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:42:15 smithi125 sudo[138398]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:42:16.039 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:42:15 smithi125 sudo[138398]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:42:16.328 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:42:16 smithi125 sudo[138401]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-16T16:42:16.328 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:42:16 smithi125 sudo[138401]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:42:16.328 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:42:16 smithi125 sudo[138401]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:42:16.328 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:42:16 smithi125 sudo[138401]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:42:16.557 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:16 smithi159 ceph-mon[131528]: osdmap e21: 3 total, 3 up, 3 in 2023-11-16T16:42:16.558 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:16 smithi159 ceph-mon[131528]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-16T16:42:16.558 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:16 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:16.558 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:16 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:16.558 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:42:16 smithi159 sudo[139559]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-16T16:42:16.558 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:42:16 smithi159 sudo[139559]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:42:16.558 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:42:16 smithi159 sudo[139559]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:42:16.559 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:42:16 smithi159 sudo[139559]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:42:16.626 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:16 smithi093 ceph-mon[97863]: osdmap e21: 3 total, 3 up, 3 in 2023-11-16T16:42:16.626 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:16 smithi093 ceph-mon[97863]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-16T16:42:16.627 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:16 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:16.627 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:16 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:16.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:16 smithi125 ceph-mon[131076]: osdmap e21: 3 total, 3 up, 3 in 2023-11-16T16:42:16.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:16 smithi125 ceph-mon[131076]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-16T16:42:16.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:16 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:16.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:16 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:16.985 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:42:16 smithi159 sudo[139562]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-16T16:42:16.985 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:42:16 smithi159 sudo[139562]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:42:16.985 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:42:16 smithi159 sudo[139562]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:42:16.985 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:42:16 smithi159 sudo[139562]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:42:17.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:17 smithi093 sudo[113398]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-16T16:42:17.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:17 smithi093 sudo[113398]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:42:17.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:17 smithi093 sudo[113398]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:42:17.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:17 smithi093 sudo[113398]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:42:18.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:18.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:17 smithi125 ceph-mon[131076]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:42:18.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:17 smithi093 sudo[113424]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-16T16:42:18.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:17 smithi093 sudo[113424]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:42:18.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:17 smithi093 sudo[113424]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:42:18.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:17 smithi093 sudo[113424]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:42:18.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:18.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:17 smithi093 ceph-mon[97863]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:42:18.164 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:17 smithi159 sudo[139565]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-16T16:42:18.164 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:17 smithi159 sudo[139565]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:42:18.164 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:17 smithi159 sudo[139565]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:42:18.164 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:17 smithi159 sudo[139565]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:42:18.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 sudo[139568]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-16T16:42:18.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 sudo[139568]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:42:18.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 sudo[139568]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:42:18.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 sudo[139568]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:42:18.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:18.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:42:18.658 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 sudo[138404]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-16T16:42:18.658 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 sudo[138404]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:42:18.658 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 sudo[138404]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:42:18.658 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 sudo[138404]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:42:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 16 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:42:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:42:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:42:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:42:18.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:42:18.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:42:18.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:42:18.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:42:18.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2692968820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71e6b247-710a-4bbb-9f6e-6333b3ea25bd"}]: dispatch 2023-11-16T16:42:18.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:42:18.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:42:18.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:42:18.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:42:18.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:42:18.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2692968820' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71e6b247-710a-4bbb-9f6e-6333b3ea25bd"}]': finished 2023-11-16T16:42:18.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: osdmap e22: 4 total, 3 up, 4 in 2023-11-16T16:42:18.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:18.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:18 smithi159 ceph-mon[131528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:42:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 sudo[138407]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-16T16:42:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 sudo[138407]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:42:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 sudo[138407]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:42:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 sudo[138407]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:42:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 16 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:42:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:42:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:42:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:42:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:42:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:42:19.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:42:19.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:42:19.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2692968820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71e6b247-710a-4bbb-9f6e-6333b3ea25bd"}]: dispatch 2023-11-16T16:42:19.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:42:19.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:42:19.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:42:19.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:42:19.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:42:19.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2692968820' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71e6b247-710a-4bbb-9f6e-6333b3ea25bd"}]': finished 2023-11-16T16:42:19.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: osdmap e22: 4 total, 3 up, 4 in 2023-11-16T16:42:19.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:19.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:18 smithi125 ceph-mon[131076]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:42:19.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 16 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:19.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:42:19.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:42:19.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:42:19.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:42:19.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:42:19.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:42:19.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:42:19.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:42:19.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2692968820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71e6b247-710a-4bbb-9f6e-6333b3ea25bd"}]: dispatch 2023-11-16T16:42:19.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:42:19.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:42:19.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:42:19.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:42:19.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:42:19.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2692968820' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71e6b247-710a-4bbb-9f6e-6333b3ea25bd"}]': finished 2023-11-16T16:42:19.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: osdmap e22: 4 total, 3 up, 4 in 2023-11-16T16:42:19.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:19.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:18 smithi093 ceph-mon[97863]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:42:19.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:19 smithi159 ceph-mon[131528]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:19.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:19 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1747473346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:42:20.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:19 smithi125 ceph-mon[131076]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:20.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:19 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1747473346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:42:20.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:19 smithi093 ceph-mon[97863]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:20.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:19 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1747473346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:42:21.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:21 smithi159 ceph-mon[131528]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:21.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:21 smithi125 ceph-mon[131076]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:21.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:21 smithi093 ceph-mon[97863]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:23.773 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:23 smithi093 ceph-mon[97863]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-11-16T16:42:23.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:23 smithi125 ceph-mon[131076]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-11-16T16:42:23.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:23 smithi159 ceph-mon[131528]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-11-16T16:42:25.629 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:25 smithi093 ceph-mon[97863]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-11-16T16:42:25.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:25 smithi125 ceph-mon[131076]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-11-16T16:42:25.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:25 smithi159 ceph-mon[131528]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-11-16T16:42:26.794 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:26 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-16T16:42:26.794 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:26 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:26.794 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:26 smithi093 ceph-mon[97863]: Deploying daemon osd.3 on smithi093 2023-11-16T16:42:26.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:26 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-16T16:42:26.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:26 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:26.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:26 smithi125 ceph-mon[131076]: Deploying daemon osd.3 on smithi093 2023-11-16T16:42:26.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:26 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-16T16:42:26.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:26 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:26.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:26 smithi159 ceph-mon[131528]: Deploying daemon osd.3 on smithi093 2023-11-16T16:42:27.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:27 smithi125 ceph-mon[131076]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-11-16T16:42:27.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:27 smithi093 ceph-mon[97863]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-11-16T16:42:27.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:27 smithi159 ceph-mon[131528]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-11-16T16:42:29.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:29 smithi125 ceph-mon[131076]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 76 KiB/s wr, 5 op/s 2023-11-16T16:42:29.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:29 smithi093 ceph-mon[97863]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 76 KiB/s wr, 5 op/s 2023-11-16T16:42:29.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:29 smithi159 ceph-mon[131528]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 76 KiB/s wr, 5 op/s 2023-11-16T16:42:31.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:31 smithi159 ceph-mon[131528]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-11-16T16:42:31.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:31 smithi125 ceph-mon[131076]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-11-16T16:42:31.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:31 smithi093 ceph-mon[97863]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-11-16T16:42:33.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:32 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:33.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:32 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:33.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:32 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:33.246 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:32 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:33.247 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:32 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:33.247 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:32 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:33.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:32 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:33.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:32 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:33.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:32 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:34.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:33 smithi159 ceph-mon[131528]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-11-16T16:42:34.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:33 smithi125 ceph-mon[131076]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-11-16T16:42:34.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:33 smithi093 ceph-mon[97863]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-11-16T16:42:35.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.767+0000 7fb44e7fc700 1 -- 172.21.15.93:0/3506384663 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb458061c70 con 0x7fb4440a1140 2023-11-16T16:42:35.770 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 3 on host 'smithi093' 2023-11-16T16:42:35.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.770+0000 7fb45f3b6700 1 -- 172.21.15.93:0/3506384663 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb4440a1140 msgr2=0x7fb4440a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:35.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.770+0000 7fb45f3b6700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb4440a1140 0x7fb4440a35f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb448011490 tx=0x7fb448009260 comp rx=0 tx=0).stop 2023-11-16T16:42:35.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.770+0000 7fb45f3b6700 1 -- 172.21.15.93:0/3506384663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb458073550 msgr2=0x7fb45819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:35.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.770+0000 7fb45f3b6700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb458073550 0x7fb45819c670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb45400b680 tx=0x7fb454009580 comp rx=0 tx=0).stop 2023-11-16T16:42:35.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.770+0000 7fb45f3b6700 1 -- 172.21.15.93:0/3506384663 shutdown_connections 2023-11-16T16:42:35.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.770+0000 7fb45f3b6700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb4440a1140 0x7fb4440a35f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:35.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.770+0000 7fb45f3b6700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb45810c8d0 0x7fb4581a3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:35.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.770+0000 7fb45f3b6700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb458073550 0x7fb45819c670 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:35.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.770+0000 7fb45f3b6700 1 --2- 172.21.15.93:0/3506384663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074b20 0x7fb45819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:35.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.770+0000 7fb45f3b6700 1 -- 172.21.15.93:0/3506384663 >> 172.21.15.93:0/3506384663 conn(0x7fb4580fb900 msgr2=0x7fb4580fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:35.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.770+0000 7fb45f3b6700 1 -- 172.21.15.93:0/3506384663 shutdown_connections 2023-11-16T16:42:35.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:42:35.770+0000 7fb45f3b6700 1 -- 172.21.15.93:0/3506384663 wait complete. 2023-11-16T16:42:36.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:35 smithi093 ceph-mon[97863]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:36.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:35 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:35 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:35 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:36.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:35 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:36.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:35 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:42:36.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:35 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:35 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:42:36.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:35 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:42:36.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:35 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:35 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:35 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:36.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:35 smithi159 ceph-mon[131528]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:36.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:36.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:36.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:35 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:42:36.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:35 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:42:36.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:35 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:42:36.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:36.311 DEBUG:teuthology.orchestra.run.smithi093:osd.3> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.3.service 2023-11-16T16:42:36.315 INFO:tasks.cephadm:Deploying osd.4 on smithi125 with /dev/vg_nvme/lv_3... 2023-11-16T16:42:36.316 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-16T16:42:36.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:35 smithi125 ceph-mon[131076]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:36.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:35 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:35 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:35 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:36.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:35 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:36.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:35 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:42:36.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:35 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:35 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:42:36.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:35 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:42:36.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:35 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:35 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:36.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:35 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:36.405 INFO:journalctl@ceph.osd.3.smithi093.stdout:-- Logs begin at Thu 2023-11-16 16:19:38 UTC. -- 2023-11-16T16:42:36.975 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:42:37.177 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:36 smithi093 ceph-mon[97863]: from='osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-16T16:42:37.218 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:36 smithi125 ceph-mon[131076]: from='osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-16T16:42:37.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:36 smithi159 ceph-mon[131528]: from='osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-16T16:42:38.173 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:37 smithi125 ceph-mon[131076]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:38.173 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:37 smithi125 ceph-mon[131076]: from='osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-16T16:42:38.173 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:37 smithi125 ceph-mon[131076]: osdmap e23: 4 total, 3 up, 4 in 2023-11-16T16:42:38.173 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:37 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:38.173 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:37 smithi125 ceph-mon[131076]: from='osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:42:38.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:37 smithi159 ceph-mon[131528]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:38.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:37 smithi159 ceph-mon[131528]: from='osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-16T16:42:38.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:37 smithi159 ceph-mon[131528]: osdmap e23: 4 total, 3 up, 4 in 2023-11-16T16:42:38.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:37 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:38.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:37 smithi159 ceph-mon[131528]: from='osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:42:38.400 INFO:journalctl@ceph.osd.3.smithi093.stdout:Nov 16 16:42:37 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-3[115738]: 2023-11-16T16:42:37.915+0000 7fb70a142700 -1 osd.3 0 waiting for initial osdmap 2023-11-16T16:42:38.401 INFO:journalctl@ceph.osd.3.smithi093.stdout:Nov 16 16:42:37 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-3[115738]: 2023-11-16T16:42:37.922+0000 7fb705736700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:42:38.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:37 smithi093 ceph-mon[97863]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:38.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:37 smithi093 ceph-mon[97863]: from='osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-16T16:42:38.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:37 smithi093 ceph-mon[97863]: osdmap e23: 4 total, 3 up, 4 in 2023-11-16T16:42:38.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:37 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:38.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:37 smithi093 ceph-mon[97863]: from='osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:42:38.607 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-16T16:42:38.607 INFO:teuthology.orchestra.run.smithi125.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-16T16:42:38.607 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10+0 records in 2023-11-16T16:42:38.607 INFO:teuthology.orchestra.run.smithi125.stderr:10+0 records out 2023-11-16T16:42:38.607 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118286 s, 886 MB/s 2023-11-16T16:42:38.607 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping successful for: 2023-11-16T16:42:39.131 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi125:vg_nvme/lv_3 2023-11-16T16:42:39.133 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:38 smithi093 ceph-mon[97863]: from='osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-11-16T16:42:39.133 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:38 smithi093 ceph-mon[97863]: osdmap e24: 4 total, 3 up, 4 in 2023-11-16T16:42:39.133 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:38 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:39.133 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:38 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:39.197 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:38 smithi125 ceph-mon[131076]: from='osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-11-16T16:42:39.197 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:38 smithi125 ceph-mon[131076]: osdmap e24: 4 total, 3 up, 4 in 2023-11-16T16:42:39.197 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:38 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:39.198 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:38 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:39.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:38 smithi159 ceph-mon[131528]: from='osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-11-16T16:42:39.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:38 smithi159 ceph-mon[131528]: osdmap e24: 4 total, 3 up, 4 in 2023-11-16T16:42:39.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:38 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:39.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:38 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:39.812 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:42:40.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: purged_snaps scrub starts 2023-11-16T16:42:40.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: purged_snaps scrub ok 2023-11-16T16:42:40.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:40.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:40.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600] boot 2023-11-16T16:42:40.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: osdmap e25: 4 total, 4 up, 4 in 2023-11-16T16:42:40.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:40.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:40.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:40.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:42:40.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:42:40.237 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:40.237 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:40.237 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:40.237 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:40.259 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: purged_snaps scrub starts 2023-11-16T16:42:40.259 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: purged_snaps scrub ok 2023-11-16T16:42:40.259 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:40.259 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:40.260 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600] boot 2023-11-16T16:42:40.260 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: osdmap e25: 4 total, 4 up, 4 in 2023-11-16T16:42:40.260 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:40.260 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:40.260 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:40.260 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:42:40.260 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:42:40.261 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:40.261 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:40.261 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:40.261 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:40.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: purged_snaps scrub starts 2023-11-16T16:42:40.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: purged_snaps scrub ok 2023-11-16T16:42:40.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:42:40.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:40.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: osd.3 [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600] boot 2023-11-16T16:42:40.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: osdmap e25: 4 total, 4 up, 4 in 2023-11-16T16:42:40.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:42:40.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:40.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:40.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:42:40.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:42:40.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:40.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:40.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:40.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:41.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.174+0000 7f9a0211d700 1 -- 172.21.15.125:0/2957036528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99fc0ff1e0 msgr2=0x7f99fc0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:41.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.174+0000 7f9a0211d700 1 --2- 172.21.15.125:0/2957036528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99fc0ff1e0 0x7f99fc0ff5f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009990 tx=0x7f99ec009ca0 comp rx=0 tx=0).stop 2023-11-16T16:42:41.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.174+0000 7f9a0211d700 1 -- 172.21.15.125:0/2957036528 shutdown_connections 2023-11-16T16:42:41.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.174+0000 7f9a0211d700 1 --2- 172.21.15.125:0/2957036528 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99fc100d70 0x7f99fc107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:41.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.174+0000 7f9a0211d700 1 --2- 172.21.15.125:0/2957036528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99fc0ff1e0 0x7f99fc0ff5f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:41.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.174+0000 7f9a0211d700 1 --2- 172.21.15.125:0/2957036528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99fc1003e0 0x7f99fc100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:41.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.174+0000 7f9a0211d700 1 -- 172.21.15.125:0/2957036528 >> 172.21.15.125:0/2957036528 conn(0x7f99fc0fa7f0 msgr2=0x7f99fc0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:41.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.175+0000 7f9a0211d700 1 -- 172.21.15.125:0/2957036528 shutdown_connections 2023-11-16T16:42:41.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.175+0000 7f9a0211d700 1 -- 172.21.15.125:0/2957036528 wait complete. 2023-11-16T16:42:41.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.175+0000 7f9a0211d700 1 Processor -- start 2023-11-16T16:42:41.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.175+0000 7f9a0211d700 1 -- start start 2023-11-16T16:42:41.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.175+0000 7f9a0211d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99fc0ff1e0 0x7f99fc1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:41.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.175+0000 7f9a0211d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99fc1003e0 0x7f99fc1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:41.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.175+0000 7f9a0211d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99fc100d70 0x7f99fc19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:41.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.175+0000 7f9a0211d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc19fe30 con 0x7f99fc0ff1e0 2023-11-16T16:42:41.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.175+0000 7f9a0211d700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc19ff70 con 0x7f99fc1003e0 2023-11-16T16:42:41.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.175+0000 7f9a0211d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc1a0100 con 0x7f99fc100d70 2023-11-16T16:42:41.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99faffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99fc1003e0 0x7f99fc1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:41.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99faffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99fc1003e0 0x7f99fc1987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:34630/0 (socket says 172.21.15.125:34630) 2023-11-16T16:42:41.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99faffd700 1 -- 172.21.15.125:0/4232071268 learned_addr learned my addr 172.21.15.125:0/4232071268 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-11-16T16:42:41.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99fbfff700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99fc100d70 0x7f99fc19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:41.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99fb7fe700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99fc0ff1e0 0x7f99fc1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:41.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99faffd700 1 -- 172.21.15.125:0/4232071268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99fc100d70 msgr2=0x7f99fc19f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:41.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99faffd700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99fc100d70 0x7f99fc19f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:41.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99faffd700 1 -- 172.21.15.125:0/4232071268 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99fc0ff1e0 msgr2=0x7f99fc1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:41.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99faffd700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99fc0ff1e0 0x7f99fc1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:41.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99faffd700 1 -- 172.21.15.125:0/4232071268 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec0095f0 con 0x7f99fc1003e0 2023-11-16T16:42:41.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99fb7fe700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99fc0ff1e0 0x7f99fc1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:42:41.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99faffd700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99fc1003e0 0x7f99fc1987e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c920 tx=0x7f99f000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:41.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99f8ff9700 1 -- 172.21.15.125:0/4232071268 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0007b00 con 0x7f99fc1003e0 2023-11-16T16:42:41.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f9a0211d700 1 -- 172.21.15.125:0/4232071268 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc10abb0 con 0x7f99fc1003e0 2023-11-16T16:42:41.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99f8ff9700 1 -- 172.21.15.125:0/4232071268 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99f0004cf0 con 0x7f99fc1003e0 2023-11-16T16:42:41.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f99f8ff9700 1 -- 172.21.15.125:0/4232071268 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0005520 con 0x7f99fc1003e0 2023-11-16T16:42:41.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.176+0000 7f9a0211d700 1 -- 172.21.15.125:0/4232071268 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc10aa20 con 0x7f99fc1003e0 2023-11-16T16:42:41.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.177+0000 7f9a0211d700 1 -- 172.21.15.125:0/4232071268 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99fc066ea0 con 0x7f99fc1003e0 2023-11-16T16:42:41.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.178+0000 7f99f8ff9700 1 -- 172.21.15.125:0/4232071268 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f99f0004750 con 0x7f99fc1003e0 2023-11-16T16:42:41.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.179+0000 7f99f8ff9700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f99e40a1150 0x7f99e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:41.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.179+0000 7f99f8ff9700 1 -- 172.21.15.125:0/4232071268 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f99f00cb7c0 con 0x7f99fc1003e0 2023-11-16T16:42:41.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.179+0000 7f99fb7fe700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f99e40a1150 0x7f99e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:41.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.180+0000 7f99fb7fe700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f99e40a1150 0x7f99e40a3600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00b5c0 tx=0x7f99ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:41.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.183+0000 7f99f8ff9700 1 -- 172.21.15.125:0/4232071268 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f99f0096110 con 0x7f99fc1003e0 2023-11-16T16:42:41.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:40 smithi159 ceph-mon[131528]: Detected new or changed devices on smithi093 2023-11-16T16:42:41.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:40 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi093 to 8635M 2023-11-16T16:42:41.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:40 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:42:41.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:40 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:42:41.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:40 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:42:41.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:40 smithi159 ceph-mon[131528]: osdmap e26: 4 total, 4 up, 4 in 2023-11-16T16:42:41.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:40 smithi125 ceph-mon[131076]: Detected new or changed devices on smithi093 2023-11-16T16:42:41.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:40 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi093 to 8635M 2023-11-16T16:42:41.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:40 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:42:41.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:40 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:42:41.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:40 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:42:41.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:40 smithi125 ceph-mon[131076]: osdmap e26: 4 total, 4 up, 4 in 2023-11-16T16:42:41.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:41.378+0000 7f9a0211d700 1 -- 172.21.15.125:0/4232071268 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f99fc1a0cb0 con 0x7f99e40a1150 2023-11-16T16:42:41.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:40 smithi093 ceph-mon[97863]: Detected new or changed devices on smithi093 2023-11-16T16:42:41.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:40 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi093 to 8635M 2023-11-16T16:42:41.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:40 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:42:41.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:40 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:42:41.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:40 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:42:41.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:40 smithi093 ceph-mon[97863]: osdmap e26: 4 total, 4 up, 4 in 2023-11-16T16:42:42.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:41 smithi159 ceph-mon[131528]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:42:42.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:41 smithi159 ceph-mon[131528]: osdmap e27: 4 total, 4 up, 4 in 2023-11-16T16:42:42.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:41 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:42:42.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:41 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:42:42.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:41 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:42.316 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:41 smithi125 ceph-mon[131076]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:42:42.316 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:41 smithi125 ceph-mon[131076]: osdmap e27: 4 total, 4 up, 4 in 2023-11-16T16:42:42.316 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:41 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:42:42.316 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:41 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:42:42.317 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:41 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:42.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:41 smithi093 ceph-mon[97863]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:42:42.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:41 smithi093 ceph-mon[97863]: osdmap e27: 4 total, 4 up, 4 in 2023-11-16T16:42:42.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:41 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:42:42.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:41 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:42:42.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:41 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:43.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:42 smithi125 ceph-mon[131076]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:42:43.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:42 smithi125 ceph-mon[131076]: osdmap e28: 4 total, 4 up, 4 in 2023-11-16T16:42:43.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:42 smithi159 ceph-mon[131528]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:42:43.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:42 smithi159 ceph-mon[131528]: osdmap e28: 4 total, 4 up, 4 in 2023-11-16T16:42:43.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:42 smithi093 ceph-mon[97863]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:42:43.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:42 smithi093 ceph-mon[97863]: osdmap e28: 4 total, 4 up, 4 in 2023-11-16T16:42:44.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:43 smithi159 ceph-mon[131528]: pgmap v104: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 1/6 objects misplaced (16.667%); 64 KiB/s, 0 objects/s recovering 2023-11-16T16:42:44.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:43 smithi159 ceph-mon[131528]: osdmap e29: 4 total, 4 up, 4 in 2023-11-16T16:42:44.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:43 smithi125 ceph-mon[131076]: pgmap v104: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 1/6 objects misplaced (16.667%); 64 KiB/s, 0 objects/s recovering 2023-11-16T16:42:44.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:43 smithi125 ceph-mon[131076]: osdmap e29: 4 total, 4 up, 4 in 2023-11-16T16:42:44.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:43 smithi093 ceph-mon[97863]: pgmap v104: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 1/6 objects misplaced (16.667%); 64 KiB/s, 0 objects/s recovering 2023-11-16T16:42:44.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:43 smithi093 ceph-mon[97863]: osdmap e29: 4 total, 4 up, 4 in 2023-11-16T16:42:45.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:44 smithi159 ceph-mon[131528]: from='client.? 172.21.15.125:0/3423652505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b965a7b2-6838-4b3a-b469-e2165cc42d16"}]: dispatch 2023-11-16T16:42:45.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:44 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b965a7b2-6838-4b3a-b469-e2165cc42d16"}]: dispatch 2023-11-16T16:42:45.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:44 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b965a7b2-6838-4b3a-b469-e2165cc42d16"}]': finished 2023-11-16T16:42:45.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:44 smithi159 ceph-mon[131528]: osdmap e30: 5 total, 4 up, 5 in 2023-11-16T16:42:45.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:44 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:42:45.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:44 smithi159 ceph-mon[131528]: from='client.? 172.21.15.125:0/2096213436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:42:45.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:44 smithi125 ceph-mon[131076]: from='client.? 172.21.15.125:0/3423652505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b965a7b2-6838-4b3a-b469-e2165cc42d16"}]: dispatch 2023-11-16T16:42:45.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:44 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b965a7b2-6838-4b3a-b469-e2165cc42d16"}]: dispatch 2023-11-16T16:42:45.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:44 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b965a7b2-6838-4b3a-b469-e2165cc42d16"}]': finished 2023-11-16T16:42:45.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:44 smithi125 ceph-mon[131076]: osdmap e30: 5 total, 4 up, 5 in 2023-11-16T16:42:45.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:44 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:42:45.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:44 smithi125 ceph-mon[131076]: from='client.? 172.21.15.125:0/2096213436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:42:45.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:44 smithi093 ceph-mon[97863]: from='client.? 172.21.15.125:0/3423652505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b965a7b2-6838-4b3a-b469-e2165cc42d16"}]: dispatch 2023-11-16T16:42:45.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:44 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b965a7b2-6838-4b3a-b469-e2165cc42d16"}]: dispatch 2023-11-16T16:42:45.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:44 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b965a7b2-6838-4b3a-b469-e2165cc42d16"}]': finished 2023-11-16T16:42:45.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:44 smithi093 ceph-mon[97863]: osdmap e30: 5 total, 4 up, 5 in 2023-11-16T16:42:45.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:44 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:42:45.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:44 smithi093 ceph-mon[97863]: from='client.? 172.21.15.125:0/2096213436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:42:46.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:45 smithi159 ceph-mon[131528]: pgmap v107: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 1/6 objects misplaced (16.667%); 64 KiB/s, 0 objects/s recovering 2023-11-16T16:42:46.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:45 smithi125 ceph-mon[131076]: pgmap v107: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 1/6 objects misplaced (16.667%); 64 KiB/s, 0 objects/s recovering 2023-11-16T16:42:46.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:45 smithi093 ceph-mon[97863]: pgmap v107: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 1/6 objects misplaced (16.667%); 64 KiB/s, 0 objects/s recovering 2023-11-16T16:42:48.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:47 smithi125 ceph-mon[131076]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 25 MiB used, 358 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-11-16T16:42:48.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:47 smithi159 ceph-mon[131528]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 25 MiB used, 358 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-11-16T16:42:48.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:47 smithi093 ceph-mon[97863]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 25 MiB used, 358 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-11-16T16:42:50.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:49 smithi159 ceph-mon[131528]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-11-16T16:42:50.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:49 smithi125 ceph-mon[131076]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-11-16T16:42:50.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:49 smithi093 ceph-mon[97863]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-11-16T16:42:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:51 smithi159 ceph-mon[131528]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 40 KiB/s, 0 objects/s recovering 2023-11-16T16:42:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-16T16:42:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:51 smithi159 ceph-mon[131528]: Deploying daemon osd.4 on smithi125 2023-11-16T16:42:51.548 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:51 smithi125 ceph-mon[131076]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 40 KiB/s, 0 objects/s recovering 2023-11-16T16:42:51.548 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-16T16:42:51.548 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:51.548 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:51 smithi125 ceph-mon[131076]: Deploying daemon osd.4 on smithi125 2023-11-16T16:42:51.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:51 smithi093 ceph-mon[97863]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 40 KiB/s, 0 objects/s recovering 2023-11-16T16:42:51.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-16T16:42:51.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:51.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:51 smithi093 ceph-mon[97863]: Deploying daemon osd.4 on smithi125 2023-11-16T16:42:53.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:53 smithi125 ceph-mon[131076]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 35 KiB/s, 0 objects/s recovering 2023-11-16T16:42:53.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:53 smithi093 ceph-mon[97863]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 35 KiB/s, 0 objects/s recovering 2023-11-16T16:42:53.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:53 smithi159 ceph-mon[131528]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 35 KiB/s, 0 objects/s recovering 2023-11-16T16:42:55.792 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:55 smithi125 ceph-mon[131076]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 30 KiB/s, 0 objects/s recovering 2023-11-16T16:42:55.792 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:55.792 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:55.792 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:55 smithi093 ceph-mon[97863]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 30 KiB/s, 0 objects/s recovering 2023-11-16T16:42:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:55.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:55.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:55 smithi159 ceph-mon[131528]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 30 KiB/s, 0 objects/s recovering 2023-11-16T16:42:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:57.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:57 smithi125 ceph-mon[131076]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 27 KiB/s, 0 objects/s recovering 2023-11-16T16:42:57.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:57 smithi093 ceph-mon[97863]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 27 KiB/s, 0 objects/s recovering 2023-11-16T16:42:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:57 smithi159 ceph-mon[131528]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 27 KiB/s, 0 objects/s recovering 2023-11-16T16:42:58.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.390+0000 7f99f8ff9700 1 -- 172.21.15.125:0/4232071268 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f99fc1a0cb0 con 0x7f99e40a1150 2023-11-16T16:42:58.393 INFO:teuthology.orchestra.run.smithi125.stdout:Created osd(s) 4 on host 'smithi125' 2023-11-16T16:42:58.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.393+0000 7f9a0211d700 1 -- 172.21.15.125:0/4232071268 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f99e40a1150 msgr2=0x7f99e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:58.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.393+0000 7f9a0211d700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f99e40a1150 0x7f99e40a3600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00b5c0 tx=0x7f99ec005fb0 comp rx=0 tx=0).stop 2023-11-16T16:42:58.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.393+0000 7f9a0211d700 1 -- 172.21.15.125:0/4232071268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99fc1003e0 msgr2=0x7f99fc1987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.393+0000 7f9a0211d700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99fc1003e0 0x7f99fc1987e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c920 tx=0x7f99f000cce0 comp rx=0 tx=0).stop 2023-11-16T16:42:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.393+0000 7f9a0211d700 1 -- 172.21.15.125:0/4232071268 shutdown_connections 2023-11-16T16:42:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.393+0000 7f9a0211d700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f99e40a1150 0x7f99e40a3600 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.393+0000 7f9a0211d700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99fc0ff1e0 0x7f99fc1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.393+0000 7f9a0211d700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99fc100d70 0x7f99fc19f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.393+0000 7f9a0211d700 1 --2- 172.21.15.125:0/4232071268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99fc1003e0 0x7f99fc1987e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.393+0000 7f9a0211d700 1 -- 172.21.15.125:0/4232071268 >> 172.21.15.125:0/4232071268 conn(0x7f99fc0fa7f0 msgr2=0x7f99fc068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.393+0000 7f9a0211d700 1 -- 172.21.15.125:0/4232071268 shutdown_connections 2023-11-16T16:42:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:42:58.393+0000 7f9a0211d700 1 -- 172.21.15.125:0/4232071268 wait complete. 2023-11-16T16:42:58.812 DEBUG:teuthology.orchestra.run.smithi125:osd.4> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.4.service 2023-11-16T16:42:58.854 INFO:tasks.cephadm:Deploying osd.5 on smithi159 with /dev/vg_nvme/lv_3... 2023-11-16T16:42:58.854 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-16T16:42:58.886 INFO:journalctl@ceph.osd.4.smithi125.stdout:-- Logs begin at Thu 2023-11-16 16:14:41 UTC. -- 2023-11-16T16:42:59.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:59.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:59.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:58 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:42:59.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:58 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:42:59.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:58 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:42:59.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:59.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:59.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:59.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:58 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:42:59.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:58 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:42:59.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:58 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:42:59.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:59.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:59.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:59.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:58 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:42:59.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:58 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:42:59.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:58 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:42:59.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:42:59.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:59.514 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:42:59.675 INFO:journalctl@ceph.osd.4.smithi125.stdout:Nov 16 16:42:59 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-4[141625]: 2023-11-16T16:42:59.346+0000 7f2ef8073640 -1 osd.4 0 log_to_monitors true 2023-11-16T16:42:59.932 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:59 smithi125 ceph-mon[131076]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:42:59.932 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:59 smithi125 ceph-mon[131076]: from='osd.4 [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-16T16:42:59.932 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:42:59 smithi125 ceph-mon[131076]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-16T16:42:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:59 smithi159 ceph-mon[131528]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:42:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:59 smithi159 ceph-mon[131528]: from='osd.4 [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-16T16:42:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:42:59 smithi159 ceph-mon[131528]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-16T16:43:00.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:59 smithi093 ceph-mon[97863]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:43:00.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:59 smithi093 ceph-mon[97863]: from='osd.4 [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-16T16:43:00.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:42:59 smithi093 ceph-mon[97863]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-16T16:43:00.967 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-16T16:43:00.967 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-16T16:43:00.968 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2023-11-16T16:43:00.968 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2023-11-16T16:43:00.968 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110718 s, 947 MB/s 2023-11-16T16:43:00.968 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2023-11-16T16:43:01.078 INFO:journalctl@ceph.osd.4.smithi125.stdout:Nov 16 16:43:00 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-4[141625]: 2023-11-16T16:43:00.883+0000 7f2eed6ea700 -1 osd.4 0 waiting for initial osdmap 2023-11-16T16:43:01.078 INFO:journalctl@ceph.osd.4.smithi125.stdout:Nov 16 16:43:00 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-4[141625]: 2023-11-16T16:43:00.889+0000 7f2ee7cdc700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:43:01.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:00 smithi125 ceph-mon[131076]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-16T16:43:01.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:00 smithi125 ceph-mon[131076]: osdmap e31: 5 total, 4 up, 5 in 2023-11-16T16:43:01.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:43:01.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:00 smithi125 ceph-mon[131076]: from='osd.4 [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:43:01.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:00 smithi125 ceph-mon[131076]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:43:01.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:00 smithi093 ceph-mon[97863]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-16T16:43:01.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:00 smithi093 ceph-mon[97863]: osdmap e31: 5 total, 4 up, 5 in 2023-11-16T16:43:01.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:43:01.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:00 smithi093 ceph-mon[97863]: from='osd.4 [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:43:01.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:00 smithi093 ceph-mon[97863]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:43:01.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:00 smithi159 ceph-mon[131528]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-16T16:43:01.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:00 smithi159 ceph-mon[131528]: osdmap e31: 5 total, 4 up, 5 in 2023-11-16T16:43:01.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:43:01.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:00 smithi159 ceph-mon[131528]: from='osd.4 [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:43:01.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:00 smithi159 ceph-mon[131528]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:43:01.368 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi159:vg_nvme/lv_3 2023-11-16T16:43:02.009 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:43:02.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:43:02.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2023-11-16T16:43:02.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: osdmap e32: 5 total, 4 up, 5 in 2023-11-16T16:43:02.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:43:02.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:43:02.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:02.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:02.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:02.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:02.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:02.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:02.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:02.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:02.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:43:02.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2023-11-16T16:43:02.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: osdmap e32: 5 total, 4 up, 5 in 2023-11-16T16:43:02.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:43:02.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:43:02.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:02.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:02.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:02.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:02.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:02.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:02.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:02.237 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:02.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:43:02.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2023-11-16T16:43:02.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: osdmap e32: 5 total, 4 up, 5 in 2023-11-16T16:43:02.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:43:02.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:43:02.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:02.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:02.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:02.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:02.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:02.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:02.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:02.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:03.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:02 smithi093 ceph-mon[97863]: purged_snaps scrub starts 2023-11-16T16:43:03.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:02 smithi093 ceph-mon[97863]: purged_snaps scrub ok 2023-11-16T16:43:03.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:02 smithi093 ceph-mon[97863]: Detected new or changed devices on smithi125 2023-11-16T16:43:03.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:02 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi125 to 8584M 2023-11-16T16:43:03.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:02 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:43:03.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:02 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:43:03.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:02 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:43:03.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:02 smithi093 ceph-mon[97863]: osd.4 [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088] boot 2023-11-16T16:43:03.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:02 smithi093 ceph-mon[97863]: osdmap e33: 5 total, 5 up, 5 in 2023-11-16T16:43:03.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:02 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:43:03.164 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:02 smithi159 ceph-mon[131528]: purged_snaps scrub starts 2023-11-16T16:43:03.164 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:02 smithi159 ceph-mon[131528]: purged_snaps scrub ok 2023-11-16T16:43:03.165 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:02 smithi159 ceph-mon[131528]: Detected new or changed devices on smithi125 2023-11-16T16:43:03.165 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:02 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi125 to 8584M 2023-11-16T16:43:03.165 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:02 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:43:03.165 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:02 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:43:03.165 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:02 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:43:03.165 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:02 smithi159 ceph-mon[131528]: osd.4 [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088] boot 2023-11-16T16:43:03.165 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:02 smithi159 ceph-mon[131528]: osdmap e33: 5 total, 5 up, 5 in 2023-11-16T16:43:03.165 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:02 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:43:03.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:02 smithi125 ceph-mon[131076]: purged_snaps scrub starts 2023-11-16T16:43:03.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:02 smithi125 ceph-mon[131076]: purged_snaps scrub ok 2023-11-16T16:43:03.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:02 smithi125 ceph-mon[131076]: Detected new or changed devices on smithi125 2023-11-16T16:43:03.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:02 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi125 to 8584M 2023-11-16T16:43:03.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:02 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:43:03.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:02 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:43:03.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:02 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:43:03.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:02 smithi125 ceph-mon[131076]: osd.4 [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088] boot 2023-11-16T16:43:03.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:02 smithi125 ceph-mon[131076]: osdmap e33: 5 total, 5 up, 5 in 2023-11-16T16:43:03.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:02 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:43:03.331 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.327+0000 7fb00bf87700 1 -- 172.21.15.159:0/3252490719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0040ff420 msgr2=0x7fb0040ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.327+0000 7fb00bf87700 1 --2- 172.21.15.159:0/3252490719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0040ff420 0x7fb0040ff830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faff4009990 tx=0x7faff4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:43:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.327+0000 7fb00bf87700 1 -- 172.21.15.159:0/3252490719 shutdown_connections 2023-11-16T16:43:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.327+0000 7fb00bf87700 1 --2- 172.21.15.159:0/3252490719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb004100fb0 0x7fb004107870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.327+0000 7fb00bf87700 1 --2- 172.21.15.159:0/3252490719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0040ff420 0x7fb0040ff830 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.327+0000 7fb00bf87700 1 --2- 172.21.15.159:0/3252490719 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb004100620 0x7fb004100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.327+0000 7fb00bf87700 1 -- 172.21.15.159:0/3252490719 >> 172.21.15.159:0/3252490719 conn(0x7fb0040faa50 msgr2=0x7fb0040fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.327+0000 7fb00bf87700 1 -- 172.21.15.159:0/3252490719 shutdown_connections 2023-11-16T16:43:03.333 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.327+0000 7fb00bf87700 1 -- 172.21.15.159:0/3252490719 wait complete. 2023-11-16T16:43:03.333 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb00bf87700 1 Processor -- start 2023-11-16T16:43:03.333 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb00bf87700 1 -- start start 2023-11-16T16:43:03.333 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb00bf87700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0040ff420 0x7fb004069be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:03.333 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb00bf87700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb004100620 0x7fb00419b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:03.333 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb00bf87700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb004100fb0 0x7fb00419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:03.333 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb00bf87700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00419de10 con 0x7fb0040ff420 2023-11-16T16:43:03.333 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb00bf87700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00419df80 con 0x7fb004100fb0 2023-11-16T16:43:03.334 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb00bf87700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00419e140 con 0x7fb004100620 2023-11-16T16:43:03.334 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb009522700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb004100620 0x7fb00419b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:03.334 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb009522700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb004100620 0x7fb00419b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54154/0 (socket says 172.21.15.159:54154) 2023-11-16T16:43:03.334 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb009522700 1 -- 172.21.15.159:0/3385862586 learned_addr learned my addr 172.21.15.159:0/3385862586 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-11-16T16:43:03.334 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb00a524700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb004100fb0 0x7fb00419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:03.334 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb009d23700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0040ff420 0x7fb004069be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:03.334 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb009522700 1 -- 172.21.15.159:0/3385862586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb004100fb0 msgr2=0x7fb00419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:03.334 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb009522700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb004100fb0 0x7fb00419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:03.335 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb009522700 1 -- 172.21.15.159:0/3385862586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0040ff420 msgr2=0x7fb004069be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:03.335 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb009522700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0040ff420 0x7fb004069be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:03.335 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.328+0000 7fb009522700 1 -- 172.21.15.159:0/3385862586 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff40095f0 con 0x7fb004100620 2023-11-16T16:43:03.335 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.329+0000 7fb009522700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb004100620 0x7fb00419b030 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb00000b530 tx=0x7fb00000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:03.335 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.329+0000 7faffaffd700 1 -- 172.21.15.159:0/3385862586 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000011830 con 0x7fb004100620 2023-11-16T16:43:03.335 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.329+0000 7faffaffd700 1 -- 172.21.15.159:0/3385862586 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb000011e50 con 0x7fb004100620 2023-11-16T16:43:03.335 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.329+0000 7faffaffd700 1 -- 172.21.15.159:0/3385862586 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00000f2e0 con 0x7fb004100620 2023-11-16T16:43:03.335 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.329+0000 7fb00bf87700 1 -- 172.21.15.159:0/3385862586 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00410adf0 con 0x7fb004100620 2023-11-16T16:43:03.336 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.329+0000 7fb00bf87700 1 -- 172.21.15.159:0/3385862586 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00410ac60 con 0x7fb004100620 2023-11-16T16:43:03.336 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.330+0000 7fb00bf87700 1 -- 172.21.15.159:0/3385862586 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb004067b00 con 0x7fb004100620 2023-11-16T16:43:03.337 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.331+0000 7faffaffd700 1 -- 172.21.15.159:0/3385862586 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb000010380 con 0x7fb004100620 2023-11-16T16:43:03.337 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.331+0000 7faffaffd700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7faff00a11d0 0x7faff00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:03.337 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.331+0000 7faffaffd700 1 -- 172.21.15.159:0/3385862586 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fb0000cb920 con 0x7fb004100620 2023-11-16T16:43:03.337 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.332+0000 7fb009d23700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7faff00a11d0 0x7faff00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:03.337 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.332+0000 7fb009d23700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7faff00a11d0 0x7faff00a3680 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faff4006010 tx=0x7faff400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:03.338 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.335+0000 7faffaffd700 1 -- 172.21.15.159:0/3385862586 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb0000960e0 con 0x7fb004100620 2023-11-16T16:43:03.531 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:03.530+0000 7fb00bf87700 1 -- 172.21.15.159:0/3385862586 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb004060f30 con 0x7faff00a11d0 2023-11-16T16:43:04.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:03 smithi093 ceph-mon[97863]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:04.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:03 smithi093 ceph-mon[97863]: osdmap e34: 5 total, 5 up, 5 in 2023-11-16T16:43:04.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:43:04.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:43:04.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:04.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:03 smithi159 ceph-mon[131528]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:04.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:03 smithi159 ceph-mon[131528]: osdmap e34: 5 total, 5 up, 5 in 2023-11-16T16:43:04.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:43:04.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:43:04.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:03 smithi125 ceph-mon[131076]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:03 smithi125 ceph-mon[131076]: osdmap e34: 5 total, 5 up, 5 in 2023-11-16T16:43:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:43:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:43:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:05.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:04 smithi159 ceph-mon[131528]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:43:05.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:04 smithi125 ceph-mon[131076]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:43:05.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:04 smithi093 ceph-mon[97863]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:43:06.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:05 smithi159 ceph-mon[131528]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:06.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:05 smithi125 ceph-mon[131076]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:06.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:05 smithi093 ceph-mon[97863]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:06 smithi159 ceph-mon[131528]: from='client.? 172.21.15.159:0/1540357833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4277ae8-0ebf-4650-a0aa-3bd3f7f96968"}]: dispatch 2023-11-16T16:43:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:06 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4277ae8-0ebf-4650-a0aa-3bd3f7f96968"}]: dispatch 2023-11-16T16:43:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:06 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4277ae8-0ebf-4650-a0aa-3bd3f7f96968"}]': finished 2023-11-16T16:43:07.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:06 smithi159 ceph-mon[131528]: osdmap e35: 6 total, 5 up, 6 in 2023-11-16T16:43:07.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:07.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:06 smithi159 ceph-mon[131528]: from='client.? 172.21.15.159:0/2138175833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:43:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:06 smithi125 ceph-mon[131076]: from='client.? 172.21.15.159:0/1540357833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4277ae8-0ebf-4650-a0aa-3bd3f7f96968"}]: dispatch 2023-11-16T16:43:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:06 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4277ae8-0ebf-4650-a0aa-3bd3f7f96968"}]: dispatch 2023-11-16T16:43:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:06 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4277ae8-0ebf-4650-a0aa-3bd3f7f96968"}]': finished 2023-11-16T16:43:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:06 smithi125 ceph-mon[131076]: osdmap e35: 6 total, 5 up, 6 in 2023-11-16T16:43:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:06 smithi125 ceph-mon[131076]: from='client.? 172.21.15.159:0/2138175833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:43:07.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:06 smithi093 ceph-mon[97863]: from='client.? 172.21.15.159:0/1540357833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4277ae8-0ebf-4650-a0aa-3bd3f7f96968"}]: dispatch 2023-11-16T16:43:07.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:06 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4277ae8-0ebf-4650-a0aa-3bd3f7f96968"}]: dispatch 2023-11-16T16:43:07.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:06 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4277ae8-0ebf-4650-a0aa-3bd3f7f96968"}]': finished 2023-11-16T16:43:07.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:06 smithi093 ceph-mon[97863]: osdmap e35: 6 total, 5 up, 6 in 2023-11-16T16:43:07.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:07.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:06 smithi093 ceph-mon[97863]: from='client.? 172.21.15.159:0/2138175833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:43:08.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:08 smithi125 ceph-mon[131076]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:08.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:08 smithi093 ceph-mon[97863]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:08.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:08 smithi159 ceph-mon[131528]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:10.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:10 smithi125 ceph-mon[131076]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:10.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:10 smithi093 ceph-mon[97863]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:10.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:10 smithi159 ceph-mon[131528]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:11.407 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:11 smithi159 ceph-mon[131528]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:11.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:11 smithi125 ceph-mon[131076]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:11.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:11 smithi093 ceph-mon[97863]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:13.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:13 smithi159 ceph-mon[131528]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:13.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:13 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-16T16:43:13.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:13 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:13.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:13 smithi159 ceph-mon[131528]: Deploying daemon osd.5 on smithi159 2023-11-16T16:43:13.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:13 smithi125 ceph-mon[131076]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:13.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:13 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-16T16:43:13.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:13 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:13.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:13 smithi125 ceph-mon[131076]: Deploying daemon osd.5 on smithi159 2023-11-16T16:43:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:13 smithi093 ceph-mon[97863]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:13 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-16T16:43:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:13 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:13 smithi093 ceph-mon[97863]: Deploying daemon osd.5 on smithi159 2023-11-16T16:43:15.709 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:15 smithi159 ceph-mon[131528]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:15.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:15 smithi125 ceph-mon[131076]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:15.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:15 smithi093 ceph-mon[97863]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:17.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:17 smithi125 ceph-mon[131076]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:17.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:17.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:17.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:17 smithi093 ceph-mon[97863]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:17 smithi159 ceph-mon[131528]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:19.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:19 smithi159 ceph-mon[131528]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:19.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:19.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:19.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:19.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:19.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:19 smithi125 ceph-mon[131076]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:19.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:19 smithi093 ceph-mon[97863]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:20.250 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.249+0000 7faffaffd700 1 -- 172.21.15.159:0/3385862586 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb004060f30 con 0x7faff00a11d0 2023-11-16T16:43:20.251 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 5 on host 'smithi159' 2023-11-16T16:43:20.253 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.251+0000 7fb00bf87700 1 -- 172.21.15.159:0/3385862586 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7faff00a11d0 msgr2=0x7faff00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:20.253 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.251+0000 7fb00bf87700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7faff00a11d0 0x7faff00a3680 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faff4006010 tx=0x7faff400b540 comp rx=0 tx=0).stop 2023-11-16T16:43:20.253 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.251+0000 7fb00bf87700 1 -- 172.21.15.159:0/3385862586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb004100620 msgr2=0x7fb00419b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:20.253 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.251+0000 7fb00bf87700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb004100620 0x7fb00419b030 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb00000b530 tx=0x7fb00000b840 comp rx=0 tx=0).stop 2023-11-16T16:43:20.253 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.251+0000 7fb00bf87700 1 -- 172.21.15.159:0/3385862586 shutdown_connections 2023-11-16T16:43:20.253 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.251+0000 7fb00bf87700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7faff00a11d0 0x7faff00a3680 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:20.253 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.251+0000 7fb00bf87700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0040ff420 0x7fb004069be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:20.253 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.251+0000 7fb00bf87700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb004100fb0 0x7fb00419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:20.253 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.251+0000 7fb00bf87700 1 --2- 172.21.15.159:0/3385862586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb004100620 0x7fb00419b030 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:20.253 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.251+0000 7fb00bf87700 1 -- 172.21.15.159:0/3385862586 >> 172.21.15.159:0/3385862586 conn(0x7fb0040faa50 msgr2=0x7fb0040fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:20.254 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.251+0000 7fb00bf87700 1 -- 172.21.15.159:0/3385862586 shutdown_connections 2023-11-16T16:43:20.254 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:43:20.251+0000 7fb00bf87700 1 -- 172.21.15.159:0/3385862586 wait complete. 2023-11-16T16:43:20.702 DEBUG:teuthology.orchestra.run.smithi159:osd.5> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.5.service 2023-11-16T16:43:20.704 INFO:tasks.cephadm:Deploying osd.6 on smithi093 with /dev/vg_nvme/lv_2... 2023-11-16T16:43:20.705 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-16T16:43:20.736 INFO:journalctl@ceph.osd.5.smithi159.stdout:-- Logs begin at Thu 2023-11-16 16:15:38 UTC. -- 2023-11-16T16:43:20.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:20 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:43:20.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:20 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:43:20.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:20 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:43:20.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:20 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:20.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:20 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:20.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:20 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:20 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:43:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:20 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:43:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:20 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:43:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:20 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:20 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:20 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:20 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:43:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:20 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:43:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:20 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:43:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:20 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:20 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:20 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:21.385 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:43:21 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:43:21.097+0000 7f919a448640 -1 osd.5 0 log_to_monitors true 2023-11-16T16:43:21.570 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:43:21.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:21 smithi159 ceph-mon[131528]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:21.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:21 smithi159 ceph-mon[131528]: from='osd.5 [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-16T16:43:21.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:21 smithi159 ceph-mon[131528]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-16T16:43:21.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:21 smithi125 ceph-mon[131076]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:21.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:21 smithi125 ceph-mon[131076]: from='osd.5 [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-16T16:43:21.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:21 smithi125 ceph-mon[131076]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-16T16:43:21.832 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:21 smithi093 ceph-mon[97863]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:21.832 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:21 smithi093 ceph-mon[97863]: from='osd.5 [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-16T16:43:21.833 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:21 smithi093 ceph-mon[97863]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-16T16:43:22.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:22 smithi093 ceph-mon[97863]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-16T16:43:22.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:22 smithi093 ceph-mon[97863]: osdmap e36: 6 total, 5 up, 6 in 2023-11-16T16:43:22.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:22 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:22.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:22 smithi093 ceph-mon[97863]: from='osd.5 [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:43:22.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:22 smithi093 ceph-mon[97863]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:43:22.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:22 smithi125 ceph-mon[131076]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-16T16:43:22.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:22 smithi125 ceph-mon[131076]: osdmap e36: 6 total, 5 up, 6 in 2023-11-16T16:43:22.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:22 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:22.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:22 smithi125 ceph-mon[131076]: from='osd.5 [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:43:22.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:22 smithi125 ceph-mon[131076]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:43:22.916 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:43:22 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:43:22.558+0000 7f918fabf700 -1 osd.5 0 waiting for initial osdmap 2023-11-16T16:43:22.916 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:43:22 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:43:22.565+0000 7f91898b0700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:43:22.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:22 smithi159 ceph-mon[131528]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-16T16:43:22.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:22 smithi159 ceph-mon[131528]: osdmap e36: 6 total, 5 up, 6 in 2023-11-16T16:43:22.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:22 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:22.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:22 smithi159 ceph-mon[131528]: from='osd.5 [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:43:22.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:22 smithi159 ceph-mon[131528]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:43:23.438 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-16T16:43:23.439 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-16T16:43:23.439 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-11-16T16:43:23.439 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-11-16T16:43:23.439 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0307292 s, 341 MB/s 2023-11-16T16:43:23.439 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-11-16T16:43:23.820 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:23.820 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-11-16T16:43:23.820 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: osdmap e37: 6 total, 5 up, 6 in 2023-11-16T16:43:23.820 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:23.820 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:23.820 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: Detected new or changed devices on smithi159 2023-11-16T16:43:23.820 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:23.820 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:23.821 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:23.821 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:23.821 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi159 to 8584M 2023-11-16T16:43:23.821 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:23.821 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:23.821 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:23.822 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:43:23.822 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:23.823 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:43:23.823 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:23 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:43:23.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:23.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-11-16T16:43:23.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: osdmap e37: 6 total, 5 up, 6 in 2023-11-16T16:43:23.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:23.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:23.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: Detected new or changed devices on smithi159 2023-11-16T16:43:23.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:23.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:23.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:23.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:23.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi159 to 8584M 2023-11-16T16:43:23.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:23.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:23.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:23.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:43:23.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:23.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:43:23.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:23 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:43:23.849 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi093:vg_nvme/lv_2 2023-11-16T16:43:23.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:23.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-11-16T16:43:23.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: osdmap e37: 6 total, 5 up, 6 in 2023-11-16T16:43:23.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:23.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:23.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: Detected new or changed devices on smithi159 2023-11-16T16:43:23.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:23.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:23.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:23.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:23.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi159 to 8584M 2023-11-16T16:43:23.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:23.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:23.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:23.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:43:23.988 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:23.988 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:43:23.988 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:23 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:43:24.697 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:43:24.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:24 smithi125 ceph-mon[131076]: purged_snaps scrub starts 2023-11-16T16:43:24.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:24 smithi125 ceph-mon[131076]: purged_snaps scrub ok 2023-11-16T16:43:24.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:24.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:24 smithi125 ceph-mon[131076]: osd.5 [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909] boot 2023-11-16T16:43:24.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:24 smithi125 ceph-mon[131076]: osdmap e38: 6 total, 6 up, 6 in 2023-11-16T16:43:24.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:24.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:24 smithi093 ceph-mon[97863]: purged_snaps scrub starts 2023-11-16T16:43:24.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:24 smithi093 ceph-mon[97863]: purged_snaps scrub ok 2023-11-16T16:43:24.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:24.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:24 smithi093 ceph-mon[97863]: osd.5 [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909] boot 2023-11-16T16:43:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:24 smithi093 ceph-mon[97863]: osdmap e38: 6 total, 6 up, 6 in 2023-11-16T16:43:24.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:24.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:24 smithi159 ceph-mon[131528]: purged_snaps scrub starts 2023-11-16T16:43:24.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:24 smithi159 ceph-mon[131528]: purged_snaps scrub ok 2023-11-16T16:43:24.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:24.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:24 smithi159 ceph-mon[131528]: osd.5 [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909] boot 2023-11-16T16:43:24.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:24 smithi159 ceph-mon[131528]: osdmap e38: 6 total, 6 up, 6 in 2023-11-16T16:43:24.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:43:25.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:25 smithi093 ceph-mon[97863]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:25.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:25 smithi093 ceph-mon[97863]: osdmap e39: 6 total, 6 up, 6 in 2023-11-16T16:43:25.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:25 smithi159 ceph-mon[131528]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:25.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:25 smithi159 ceph-mon[131528]: osdmap e39: 6 total, 6 up, 6 in 2023-11-16T16:43:26.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:25 smithi125 ceph-mon[131076]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:43:26.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:25 smithi125 ceph-mon[131076]: osdmap e39: 6 total, 6 up, 6 in 2023-11-16T16:43:26.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.192+0000 7fc3256b2700 1 -- 172.21.15.93:0/1301369701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3201002f0 msgr2=0x7fc320100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:26.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.192+0000 7fc3256b2700 1 --2- 172.21.15.93:0/1301369701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3201002f0 0x7fc320100700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc310009990 tx=0x7fc310009ca0 comp rx=0 tx=0).stop 2023-11-16T16:43:26.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.192+0000 7fc3256b2700 1 -- 172.21.15.93:0/1301369701 shutdown_connections 2023-11-16T16:43:26.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.192+0000 7fc3256b2700 1 --2- 172.21.15.93:0/1301369701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3201002f0 0x7fc320100700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:26.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.192+0000 7fc3256b2700 1 --2- 172.21.15.93:0/1301369701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc320101e80 0x7fc320108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:26.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.192+0000 7fc3256b2700 1 --2- 172.21.15.93:0/1301369701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3201014f0 0x7fc320101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:26.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.192+0000 7fc3256b2700 1 -- 172.21.15.93:0/1301369701 >> 172.21.15.93:0/1301369701 conn(0x7fc3200fb900 msgr2=0x7fc3200fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:26.195 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.192+0000 7fc3256b2700 1 -- 172.21.15.93:0/1301369701 shutdown_connections 2023-11-16T16:43:26.195 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.192+0000 7fc3256b2700 1 -- 172.21.15.93:0/1301369701 wait complete. 2023-11-16T16:43:26.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.193+0000 7fc3256b2700 1 Processor -- start 2023-11-16T16:43:26.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.193+0000 7fc3256b2700 1 -- start start 2023-11-16T16:43:26.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.193+0000 7fc3256b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3201002f0 0x7fc320198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:26.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc3256b2700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3201014f0 0x7fc3201987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:26.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc3256b2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc320101e80 0x7fc32019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:26.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc3256b2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32019fe20 con 0x7fc3201002f0 2023-11-16T16:43:26.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc3256b2700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32019ff60 con 0x7fc3201014f0 2023-11-16T16:43:26.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc3256b2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3201a00f0 con 0x7fc320101e80 2023-11-16T16:43:26.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc31e7fc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3201014f0 0x7fc3201987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:26.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc31e7fc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3201014f0 0x7fc3201987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:58268/0 (socket says 172.21.15.93:58268) 2023-11-16T16:43:26.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc31e7fc700 1 -- 172.21.15.93:0/2044711714 learned_addr learned my addr 172.21.15.93:0/2044711714 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:43:26.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc31effd700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3201002f0 0x7fc320198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:26.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc31f7fe700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc320101e80 0x7fc32019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:26.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc31e7fc700 1 -- 172.21.15.93:0/2044711714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc320101e80 msgr2=0x7fc32019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:26.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc31e7fc700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc320101e80 0x7fc32019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:26.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc31e7fc700 1 -- 172.21.15.93:0/2044711714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3201002f0 msgr2=0x7fc320198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:26.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc31e7fc700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3201002f0 0x7fc320198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:26.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.194+0000 7fc31e7fc700 1 -- 172.21.15.93:0/2044711714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3100095f0 con 0x7fc3201014f0 2023-11-16T16:43:26.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.195+0000 7fc31e7fc700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3201014f0 0x7fc3201987d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc31400e9a0 tx=0x7fc31400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:26.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.195+0000 7fc307fff700 1 -- 172.21.15.93:0/2044711714 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314009790 con 0x7fc3201014f0 2023-11-16T16:43:26.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.195+0000 7fc307fff700 1 -- 172.21.15.93:0/2044711714 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc314010430 con 0x7fc3201014f0 2023-11-16T16:43:26.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.195+0000 7fc3256b2700 1 -- 172.21.15.93:0/2044711714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32010bc90 con 0x7fc3201014f0 2023-11-16T16:43:26.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.195+0000 7fc307fff700 1 -- 172.21.15.93:0/2044711714 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314018680 con 0x7fc3201014f0 2023-11-16T16:43:26.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.195+0000 7fc3256b2700 1 -- 172.21.15.93:0/2044711714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32010bb00 con 0x7fc3201014f0 2023-11-16T16:43:26.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.196+0000 7fc3256b2700 1 -- 172.21.15.93:0/2044711714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3201929f0 con 0x7fc3201014f0 2023-11-16T16:43:26.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.198+0000 7fc307fff700 1 -- 172.21.15.93:0/2044711714 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc3140105a0 con 0x7fc3201014f0 2023-11-16T16:43:26.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.199+0000 7fc307fff700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc3080a1150 0x7fc3080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:26.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.199+0000 7fc307fff700 1 -- 172.21.15.93:0/2044711714 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fc314014070 con 0x7fc3201014f0 2023-11-16T16:43:26.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.200+0000 7fc31effd700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc3080a1150 0x7fc3080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:26.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.200+0000 7fc31effd700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc3080a1150 0x7fc3080a3600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc31000b2e0 tx=0x7fc310005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:26.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.201+0000 7fc307fff700 1 -- 172.21.15.93:0/2044711714 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc314096670 con 0x7fc3201014f0 2023-11-16T16:43:26.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:26.405+0000 7fc3256b2700 1 -- 172.21.15.93:0/2044711714 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc320061ae0 con 0x7fc3080a1150 2023-11-16T16:43:26.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:26 smithi093 ceph-mon[97863]: osdmap e40: 6 total, 6 up, 6 in 2023-11-16T16:43:26.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:26 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:43:26.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:26 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:43:26.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:26 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:26.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:26 smithi159 ceph-mon[131528]: osdmap e40: 6 total, 6 up, 6 in 2023-11-16T16:43:26.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:26 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:43:26.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:26 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:43:26.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:26 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:27.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:26 smithi125 ceph-mon[131076]: osdmap e40: 6 total, 6 up, 6 in 2023-11-16T16:43:27.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:26 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:43:27.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:26 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:43:27.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:26 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:27.889 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:27 smithi093 ceph-mon[97863]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:43:27.889 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:27 smithi093 ceph-mon[97863]: pgmap v138: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-16T16:43:27.889 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:27 smithi093 ceph-mon[97863]: osdmap e41: 6 total, 6 up, 6 in 2023-11-16T16:43:27.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:27 smithi159 ceph-mon[131528]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:43:27.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:27 smithi159 ceph-mon[131528]: pgmap v138: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-16T16:43:27.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:27 smithi159 ceph-mon[131528]: osdmap e41: 6 total, 6 up, 6 in 2023-11-16T16:43:28.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:27 smithi125 ceph-mon[131076]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:43:28.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:27 smithi125 ceph-mon[131076]: pgmap v138: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-16T16:43:28.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:27 smithi125 ceph-mon[131076]: osdmap e41: 6 total, 6 up, 6 in 2023-11-16T16:43:28.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:28 smithi093 ceph-mon[97863]: osdmap e42: 6 total, 6 up, 6 in 2023-11-16T16:43:28.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:28 smithi159 ceph-mon[131528]: osdmap e42: 6 total, 6 up, 6 in 2023-11-16T16:43:29.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:28 smithi125 ceph-mon[131076]: osdmap e42: 6 total, 6 up, 6 in 2023-11-16T16:43:29.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:29 smithi093 ceph-mon[97863]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-16T16:43:29.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:29 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/793549420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ee6be21-c42d-441f-ac51-099370cf27bb"}]: dispatch 2023-11-16T16:43:29.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:29 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/793549420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ee6be21-c42d-441f-ac51-099370cf27bb"}]': finished 2023-11-16T16:43:29.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:29 smithi093 ceph-mon[97863]: osdmap e43: 7 total, 6 up, 7 in 2023-11-16T16:43:29.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:29 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:29.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:29 smithi159 ceph-mon[131528]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-16T16:43:29.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:29 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/793549420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ee6be21-c42d-441f-ac51-099370cf27bb"}]: dispatch 2023-11-16T16:43:29.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:29 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/793549420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ee6be21-c42d-441f-ac51-099370cf27bb"}]': finished 2023-11-16T16:43:29.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:29 smithi159 ceph-mon[131528]: osdmap e43: 7 total, 6 up, 7 in 2023-11-16T16:43:29.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:29 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:30.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:29 smithi125 ceph-mon[131076]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-16T16:43:30.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:29 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/793549420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ee6be21-c42d-441f-ac51-099370cf27bb"}]: dispatch 2023-11-16T16:43:30.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:29 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/793549420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ee6be21-c42d-441f-ac51-099370cf27bb"}]': finished 2023-11-16T16:43:30.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:29 smithi125 ceph-mon[131076]: osdmap e43: 7 total, 6 up, 7 in 2023-11-16T16:43:30.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:29 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:31.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:31 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2043176900' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:43:31.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:31 smithi159 ceph-mon[131528]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2023-11-16T16:43:31.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:31 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2043176900' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:43:31.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:31 smithi125 ceph-mon[131076]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2023-11-16T16:43:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:31 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2043176900' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:43:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:31 smithi093 ceph-mon[97863]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2023-11-16T16:43:33.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:33 smithi093 ceph-mon[97863]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:33 smithi125 ceph-mon[131076]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:33 smithi159 ceph-mon[131528]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:35.577 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:35 smithi093 ceph-mon[97863]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:35 smithi125 ceph-mon[131076]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:35.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:35 smithi159 ceph-mon[131528]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:37.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:37 smithi125 ceph-mon[131076]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:37.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:37 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-16T16:43:37.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:37 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:37.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:37 smithi125 ceph-mon[131076]: Deploying daemon osd.6 on smithi093 2023-11-16T16:43:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:37 smithi093 ceph-mon[97863]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:37 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-16T16:43:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:37 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:37.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:37 smithi093 ceph-mon[97863]: Deploying daemon osd.6 on smithi093 2023-11-16T16:43:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:37 smithi159 ceph-mon[131528]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:37 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-16T16:43:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:37 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:37 smithi159 ceph-mon[131528]: Deploying daemon osd.6 on smithi093 2023-11-16T16:43:39.827 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:39 smithi093 ceph-mon[97863]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:39.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:39 smithi125 ceph-mon[131076]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:39.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:39 smithi159 ceph-mon[131528]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:41.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:41 smithi159 ceph-mon[131528]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:41.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:41 smithi125 ceph-mon[131076]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:41.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:41 smithi093 ceph-mon[97863]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:43.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:43 smithi125 ceph-mon[131076]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:43.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:43.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:43.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:43.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:43 smithi093 ceph-mon[97863]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:43.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:43 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:43.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:43 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:43.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:43 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:43.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:43 smithi159 ceph-mon[131528]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:43.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:43 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:43.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:43 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:43.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:43 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:45.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:45 smithi125 ceph-mon[131076]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:45.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:45 smithi093 ceph-mon[97863]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:45.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:45 smithi159 ceph-mon[131528]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:46.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.864+0000 7fc307fff700 1 -- 172.21.15.93:0/2044711714 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc320061ae0 con 0x7fc3080a1150 2023-11-16T16:43:46.868 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 6 on host 'smithi093' 2023-11-16T16:43:46.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.867+0000 7fc3256b2700 1 -- 172.21.15.93:0/2044711714 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc3080a1150 msgr2=0x7fc3080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:46.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.867+0000 7fc3256b2700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc3080a1150 0x7fc3080a3600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc31000b2e0 tx=0x7fc310005fb0 comp rx=0 tx=0).stop 2023-11-16T16:43:46.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.867+0000 7fc3256b2700 1 -- 172.21.15.93:0/2044711714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3201014f0 msgr2=0x7fc3201987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:46.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.867+0000 7fc3256b2700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3201014f0 0x7fc3201987d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc31400e9a0 tx=0x7fc31400ecb0 comp rx=0 tx=0).stop 2023-11-16T16:43:46.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.867+0000 7fc3256b2700 1 -- 172.21.15.93:0/2044711714 shutdown_connections 2023-11-16T16:43:46.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.867+0000 7fc3256b2700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc3080a1150 0x7fc3080a3600 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:46.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.867+0000 7fc3256b2700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3201002f0 0x7fc320198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:46.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.867+0000 7fc3256b2700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc320101e80 0x7fc32019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:46.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.867+0000 7fc3256b2700 1 --2- 172.21.15.93:0/2044711714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3201014f0 0x7fc3201987d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:46.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.867+0000 7fc3256b2700 1 -- 172.21.15.93:0/2044711714 >> 172.21.15.93:0/2044711714 conn(0x7fc3200fb900 msgr2=0x7fc320106e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:46.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.867+0000 7fc3256b2700 1 -- 172.21.15.93:0/2044711714 shutdown_connections 2023-11-16T16:43:46.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:43:46.867+0000 7fc3256b2700 1 -- 172.21.15.93:0/2044711714 wait complete. 2023-11-16T16:43:47.341 DEBUG:teuthology.orchestra.run.smithi093:osd.6> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.6.service 2023-11-16T16:43:47.346 INFO:tasks.cephadm:Deploying osd.7 on smithi125 with /dev/vg_nvme/lv_2... 2023-11-16T16:43:47.346 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-16T16:43:47.418 INFO:journalctl@ceph.osd.6.smithi093.stdout:-- Logs begin at Thu 2023-11-16 16:19:38 UTC. -- 2023-11-16T16:43:47.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:47.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:47.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:43:47.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:43:47.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:43:47.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:47.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:47.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: from='osd.6 [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-16T16:43:47.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:47 smithi093 ceph-mon[97863]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-16T16:43:47.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:47.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:47.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:43:47.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:43:47.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:43:47.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:47.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:47.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: from='osd.6 [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-16T16:43:47.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:47 smithi159 ceph-mon[131528]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-16T16:43:47.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:47.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:47.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:43:47.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:43:47.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:43:47.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:47.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:47.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:47.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: from='osd.6 [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-16T16:43:47.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:47 smithi125 ceph-mon[131076]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-16T16:43:47.998 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:43:49.123 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:48 smithi125 ceph-mon[131076]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-16T16:43:49.124 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:48 smithi125 ceph-mon[131076]: osdmap e44: 7 total, 6 up, 7 in 2023-11-16T16:43:49.124 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:49.124 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:48 smithi125 ceph-mon[131076]: from='osd.6 [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:43:49.124 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:48 smithi125 ceph-mon[131076]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:43:49.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:48 smithi093 ceph-mon[97863]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-16T16:43:49.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:48 smithi093 ceph-mon[97863]: osdmap e44: 7 total, 6 up, 7 in 2023-11-16T16:43:49.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:49.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:48 smithi093 ceph-mon[97863]: from='osd.6 [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:43:49.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:48 smithi093 ceph-mon[97863]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:43:49.151 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:43:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:43:48.843+0000 7f12aab64700 -1 osd.6 0 waiting for initial osdmap 2023-11-16T16:43:49.151 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:43:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:43:48.854+0000 7f12a4955700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:43:49.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:48 smithi159 ceph-mon[131528]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-16T16:43:49.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:48 smithi159 ceph-mon[131528]: osdmap e44: 7 total, 6 up, 7 in 2023-11-16T16:43:49.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:49.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:48 smithi159 ceph-mon[131528]: from='osd.6 [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:43:49.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:48 smithi159 ceph-mon[131528]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:43:49.605 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-16T16:43:49.605 INFO:teuthology.orchestra.run.smithi125.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-16T16:43:49.606 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10+0 records in 2023-11-16T16:43:49.606 INFO:teuthology.orchestra.run.smithi125.stderr:10+0 records out 2023-11-16T16:43:49.606 INFO:teuthology.orchestra.run.smithi125.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116834 s, 897 MB/s 2023-11-16T16:43:49.606 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 2023-11-16T16:43:49.606 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping successful for: 2023-11-16T16:43:50.039 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:49 smithi125 ceph-mon[131076]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:50.039 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:49 smithi125 ceph-mon[131076]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-11-16T16:43:50.039 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:49 smithi125 ceph-mon[131076]: osdmap e45: 7 total, 6 up, 7 in 2023-11-16T16:43:50.039 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:49 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:50.039 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:49 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:50.042 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi125:vg_nvme/lv_2 2023-11-16T16:43:50.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:49 smithi093 ceph-mon[97863]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:50.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:49 smithi093 ceph-mon[97863]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-11-16T16:43:50.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:49 smithi093 ceph-mon[97863]: osdmap e45: 7 total, 6 up, 7 in 2023-11-16T16:43:50.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:49 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:50.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:49 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:50.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:49 smithi159 ceph-mon[131528]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:50.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:49 smithi159 ceph-mon[131528]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-11-16T16:43:50.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:49 smithi159 ceph-mon[131528]: osdmap e45: 7 total, 6 up, 7 in 2023-11-16T16:43:50.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:49 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:50.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:49 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:50.683 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:43:51.109 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: purged_snaps scrub starts 2023-11-16T16:43:51.109 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: purged_snaps scrub ok 2023-11-16T16:43:51.109 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: osd.6 [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195] boot 2023-11-16T16:43:51.110 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: osdmap e46: 7 total, 7 up, 7 in 2023-11-16T16:43:51.110 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:51.110 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:51.110 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:51.110 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:51.110 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:51.110 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:51.110 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:51.111 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:51.111 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:51.111 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:50 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:51.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: purged_snaps scrub starts 2023-11-16T16:43:51.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: purged_snaps scrub ok 2023-11-16T16:43:51.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: osd.6 [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195] boot 2023-11-16T16:43:51.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: osdmap e46: 7 total, 7 up, 7 in 2023-11-16T16:43:51.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:51.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:51.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:51.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:51.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:51.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:51.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:51.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:51.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:51.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:50 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:51.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: purged_snaps scrub starts 2023-11-16T16:43:51.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: purged_snaps scrub ok 2023-11-16T16:43:51.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: osd.6 [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195] boot 2023-11-16T16:43:51.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: osdmap e46: 7 total, 7 up, 7 in 2023-11-16T16:43:51.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:43:51.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:51.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:51.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:51.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:51.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:43:51.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:51.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:51.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:51.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:50 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:43:52.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.021+0000 7f6295456700 1 -- 172.21.15.125:0/2123131257 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6290100380 msgr2=0x7f6290100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:52.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.021+0000 7f6295456700 1 --2- 172.21.15.125:0/2123131257 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6290100380 0x7f6290100790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6284009990 tx=0x7f6284009ca0 comp rx=0 tx=0).stop 2023-11-16T16:43:52.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.021+0000 7f6295456700 1 -- 172.21.15.125:0/2123131257 shutdown_connections 2023-11-16T16:43:52.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.021+0000 7f6295456700 1 --2- 172.21.15.125:0/2123131257 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6290101f10 0x7f62901087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:52.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.021+0000 7f6295456700 1 --2- 172.21.15.125:0/2123131257 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6290100380 0x7f6290100790 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:52.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.021+0000 7f6295456700 1 --2- 172.21.15.125:0/2123131257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6290101580 0x7f62901019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:52.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.021+0000 7f6295456700 1 -- 172.21.15.125:0/2123131257 >> 172.21.15.125:0/2123131257 conn(0x7f62900fb9d0 msgr2=0x7f62900fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:52.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.021+0000 7f6295456700 1 -- 172.21.15.125:0/2123131257 shutdown_connections 2023-11-16T16:43:52.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.021+0000 7f6295456700 1 -- 172.21.15.125:0/2123131257 wait complete. 2023-11-16T16:43:52.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.021+0000 7f6295456700 1 Processor -- start 2023-11-16T16:43:52.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f6295456700 1 -- start start 2023-11-16T16:43:52.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f6295456700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6290100380 0x7f62901960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:52.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f6295456700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6290101580 0x7f62901965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:52.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f6295456700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6290101f10 0x7f629019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:52.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f6295456700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629019dc40 con 0x7f6290101580 2023-11-16T16:43:52.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f6295456700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629019dd80 con 0x7f6290100380 2023-11-16T16:43:52.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f6295456700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629019df10 con 0x7f6290101f10 2023-11-16T16:43:52.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f628effd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6290100380 0x7f62901960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:52.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f628effd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6290100380 0x7f62901960b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:34740/0 (socket says 172.21.15.125:34740) 2023-11-16T16:43:52.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f628effd700 1 -- 172.21.15.125:0/2043927002 learned_addr learned my addr 172.21.15.125:0/2043927002 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-11-16T16:43:52.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f628e7fc700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6290101580 0x7f62901965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:52.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f628effd700 1 -- 172.21.15.125:0/2043927002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6290101f10 msgr2=0x7f629019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:52.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f628f7fe700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6290101f10 0x7f629019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:52.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f628effd700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6290101f10 0x7f629019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:52.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f628effd700 1 -- 172.21.15.125:0/2043927002 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6290101580 msgr2=0x7f62901965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:52.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f628effd700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6290101580 0x7f62901965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:52.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f628effd700 1 -- 172.21.15.125:0/2043927002 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62840095f0 con 0x7f6290100380 2023-11-16T16:43:52.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f628e7fc700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6290101580 0x7f62901965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:43:52.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f628effd700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6290100380 0x7f62901960b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f62840095c0 tx=0x7f628400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:52.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f6277fff700 1 -- 172.21.15.125:0/2043927002 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628401d070 con 0x7f6290100380 2023-11-16T16:43:52.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f6277fff700 1 -- 172.21.15.125:0/2043927002 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6284004cf0 con 0x7f6290100380 2023-11-16T16:43:52.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.022+0000 7f6295456700 1 -- 172.21.15.125:0/2043927002 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629010bda0 con 0x7f6290100380 2023-11-16T16:43:52.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.023+0000 7f6277fff700 1 -- 172.21.15.125:0/2043927002 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284005560 con 0x7f6290100380 2023-11-16T16:43:52.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.023+0000 7f6295456700 1 -- 172.21.15.125:0/2043927002 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629010bc10 con 0x7f6290100380 2023-11-16T16:43:52.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.023+0000 7f6295456700 1 -- 172.21.15.125:0/2043927002 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62901079a0 con 0x7f6290100380 2023-11-16T16:43:52.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.024+0000 7f6277fff700 1 -- 172.21.15.125:0/2043927002 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f628400fb70 con 0x7f6290100380 2023-11-16T16:43:52.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.025+0000 7f6277fff700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f62780a1150 0x7f62780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:52.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.025+0000 7f6277fff700 1 -- 172.21.15.125:0/2043927002 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f62840cd3d0 con 0x7f6290100380 2023-11-16T16:43:52.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.025+0000 7f628e7fc700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f62780a1150 0x7f62780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:52.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.026+0000 7f628e7fc700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f62780a1150 0x7f62780a3600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6280005950 tx=0x7f62800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:52.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.028+0000 7f6277fff700 1 -- 172.21.15.125:0/2043927002 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f62840978c0 con 0x7f6290100380 2023-11-16T16:43:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:51 smithi125 ceph-mon[131076]: Detected new or changed devices on smithi093 2023-11-16T16:43:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:51 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi093 to 5757M 2023-11-16T16:43:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:51 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:43:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:51 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:43:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:51 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:43:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:51 smithi125 ceph-mon[131076]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:51 smithi125 ceph-mon[131076]: osdmap e47: 7 total, 7 up, 7 in 2023-11-16T16:43:52.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:51 smithi093 ceph-mon[97863]: Detected new or changed devices on smithi093 2023-11-16T16:43:52.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:51 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi093 to 5757M 2023-11-16T16:43:52.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:51 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:43:52.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:51 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:43:52.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:51 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:43:52.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:51 smithi093 ceph-mon[97863]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:52.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:51 smithi093 ceph-mon[97863]: osdmap e47: 7 total, 7 up, 7 in 2023-11-16T16:43:52.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:43:52.223+0000 7f6295456700 1 -- 172.21.15.125:0/2043927002 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6290061b20 con 0x7f62780a1150 2023-11-16T16:43:52.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:51 smithi159 ceph-mon[131528]: Detected new or changed devices on smithi093 2023-11-16T16:43:52.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:51 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi093 to 5757M 2023-11-16T16:43:52.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:51 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:43:52.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:51 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:43:52.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:51 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:43:52.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:51 smithi159 ceph-mon[131528]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:43:52.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:51 smithi159 ceph-mon[131528]: osdmap e47: 7 total, 7 up, 7 in 2023-11-16T16:43:53.129 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:43:53.130 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:43:53.130 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:53.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:43:53.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:43:53.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:53.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:43:53.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:43:53.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:54.013 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:53 smithi125 ceph-mon[131076]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:43:54.014 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:53 smithi125 ceph-mon[131076]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:43:54.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:53 smithi093 ceph-mon[97863]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:43:54.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:53 smithi093 ceph-mon[97863]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:43:54.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:53 smithi159 ceph-mon[131528]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:43:54.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:53 smithi159 ceph-mon[131528]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:43:55.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:54 smithi093 ceph-mon[97863]: from='client.? 172.21.15.125:0/3741648276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca36594b-cd4d-4d71-a448-b677bdc862aa"}]: dispatch 2023-11-16T16:43:55.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:54 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca36594b-cd4d-4d71-a448-b677bdc862aa"}]: dispatch 2023-11-16T16:43:55.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:54 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca36594b-cd4d-4d71-a448-b677bdc862aa"}]': finished 2023-11-16T16:43:55.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:54 smithi093 ceph-mon[97863]: osdmap e48: 8 total, 7 up, 8 in 2023-11-16T16:43:55.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:54 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:43:55.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:54 smithi159 ceph-mon[131528]: from='client.? 172.21.15.125:0/3741648276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca36594b-cd4d-4d71-a448-b677bdc862aa"}]: dispatch 2023-11-16T16:43:55.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:54 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca36594b-cd4d-4d71-a448-b677bdc862aa"}]: dispatch 2023-11-16T16:43:55.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:54 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca36594b-cd4d-4d71-a448-b677bdc862aa"}]': finished 2023-11-16T16:43:55.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:54 smithi159 ceph-mon[131528]: osdmap e48: 8 total, 7 up, 8 in 2023-11-16T16:43:55.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:54 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:43:55.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:54 smithi125 ceph-mon[131076]: from='client.? 172.21.15.125:0/3741648276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca36594b-cd4d-4d71-a448-b677bdc862aa"}]: dispatch 2023-11-16T16:43:55.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:54 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca36594b-cd4d-4d71-a448-b677bdc862aa"}]: dispatch 2023-11-16T16:43:55.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:54 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca36594b-cd4d-4d71-a448-b677bdc862aa"}]': finished 2023-11-16T16:43:55.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:54 smithi125 ceph-mon[131076]: osdmap e48: 8 total, 7 up, 8 in 2023-11-16T16:43:55.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:54 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:43:56.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:55 smithi093 ceph-mon[97863]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:43:56.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:55 smithi093 ceph-mon[97863]: from='client.? 172.21.15.125:0/2492639815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:43:56.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:55 smithi159 ceph-mon[131528]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:43:56.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:55 smithi159 ceph-mon[131528]: from='client.? 172.21.15.125:0/2492639815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:43:56.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:55 smithi125 ceph-mon[131076]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:43:56.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:55 smithi125 ceph-mon[131076]: from='client.? 172.21.15.125:0/2492639815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:43:58.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:57 smithi093 ceph-mon[97863]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:43:58.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:57 smithi159 ceph-mon[131528]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:43:58.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:57 smithi125 ceph-mon[131076]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:00.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:43:59 smithi125 ceph-mon[131076]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:00.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:43:59 smithi093 ceph-mon[97863]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:00.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:43:59 smithi159 ceph-mon[131528]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:01.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:01 smithi125 ceph-mon[131076]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:01.484 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:01 smithi159 ceph-mon[131528]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:01.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:01 smithi093 ceph-mon[97863]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:02.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:02 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-16T16:44:02.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:02 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:02.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:02 smithi159 ceph-mon[131528]: Deploying daemon osd.7 on smithi125 2023-11-16T16:44:02.555 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:02 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-16T16:44:02.556 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:02 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:02.556 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:02 smithi125 ceph-mon[131076]: Deploying daemon osd.7 on smithi125 2023-11-16T16:44:02.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:02 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-16T16:44:02.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:02 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:02.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:02 smithi093 ceph-mon[97863]: Deploying daemon osd.7 on smithi125 2023-11-16T16:44:03.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:03 smithi159 ceph-mon[131528]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:03.494 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:03 smithi125 ceph-mon[131076]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:03.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:03 smithi093 ceph-mon[97863]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:05.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:05 smithi125 ceph-mon[131076]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:05.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:05 smithi093 ceph-mon[97863]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:05.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:05 smithi159 ceph-mon[131528]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:07.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:07 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:07.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:07 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:07.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:07 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:07.446 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:07 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:07.446 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:07 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:07.447 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:07 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:07 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:07 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:07 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:08.255 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:08 smithi125 ceph-mon[131076]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:08.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:08 smithi093 ceph-mon[97863]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:08.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:08 smithi159 ceph-mon[131528]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:09.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.189+0000 7f6277fff700 1 -- 172.21.15.125:0/2043927002 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6290061b20 con 0x7f62780a1150 2023-11-16T16:44:09.191 INFO:teuthology.orchestra.run.smithi125.stdout:Created osd(s) 7 on host 'smithi125' 2023-11-16T16:44:09.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.191+0000 7f6295456700 1 -- 172.21.15.125:0/2043927002 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f62780a1150 msgr2=0x7f62780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:09.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.191+0000 7f6295456700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f62780a1150 0x7f62780a3600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6280005950 tx=0x7f62800058e0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.191+0000 7f6295456700 1 -- 172.21.15.125:0/2043927002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6290100380 msgr2=0x7f62901960b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:09.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.191+0000 7f6295456700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6290100380 0x7f62901960b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f62840095c0 tx=0x7f628400f690 comp rx=0 tx=0).stop 2023-11-16T16:44:09.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.191+0000 7f6295456700 1 -- 172.21.15.125:0/2043927002 shutdown_connections 2023-11-16T16:44:09.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.191+0000 7f6295456700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f62780a1150 0x7f62780a3600 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.191+0000 7f6295456700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6290101f10 0x7f629019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.191+0000 7f6295456700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6290100380 0x7f62901960b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.191+0000 7f6295456700 1 --2- 172.21.15.125:0/2043927002 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6290101580 0x7f62901965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.191+0000 7f6295456700 1 -- 172.21.15.125:0/2043927002 >> 172.21.15.125:0/2043927002 conn(0x7f62900fb9d0 msgr2=0x7f6290104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:09.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.191+0000 7f6295456700 1 -- 172.21.15.125:0/2043927002 shutdown_connections 2023-11-16T16:44:09.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:44:09.191+0000 7f6295456700 1 -- 172.21.15.125:0/2043927002 wait complete. 2023-11-16T16:44:09.565 DEBUG:teuthology.orchestra.run.smithi125:osd.7> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.7.service 2023-11-16T16:44:09.569 INFO:tasks.cephadm:Deploying osd.8 on smithi159 with /dev/vg_nvme/lv_2... 2023-11-16T16:44:09.569 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-16T16:44:09.600 INFO:journalctl@ceph.osd.7.smithi125.stdout:-- Logs begin at Thu 2023-11-16 16:14:41 UTC. -- 2023-11-16T16:44:09.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:09 smithi125 ceph-mon[131076]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:09.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:09.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:09.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:09 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:44:09.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:09 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:44:09.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:09 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:44:09.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:09.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:09 smithi093 ceph-mon[97863]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:09.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:09.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:09.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:09 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:44:09.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:09 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:44:09.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:09 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:44:09.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:09.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:09 smithi159 ceph-mon[131528]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:09.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:09 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:44:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:09 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:44:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:09 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:44:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:10.221 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:44:10.463 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:44:10 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:44:10.188+0000 7f2cd6fe1640 -1 osd.7 0 log_to_monitors true 2023-11-16T16:44:10.728 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:10 smithi125 ceph-mon[131076]: from='osd.7 [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-16T16:44:10.728 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:10 smithi125 ceph-mon[131076]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-16T16:44:10.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:10 smithi159 ceph-mon[131528]: from='osd.7 [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-16T16:44:10.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:10 smithi159 ceph-mon[131528]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-16T16:44:10.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:10 smithi093 ceph-mon[97863]: from='osd.7 [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-16T16:44:10.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:10 smithi093 ceph-mon[97863]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-16T16:44:11.715 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-16T16:44:11.715 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-16T16:44:11.715 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2023-11-16T16:44:11.715 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2023-11-16T16:44:11.715 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110253 s, 951 MB/s 2023-11-16T16:44:11.715 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2023-11-16T16:44:11.827 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:44:11 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:44:11.601+0000 7f2ccc658700 -1 osd.7 0 waiting for initial osdmap 2023-11-16T16:44:11.828 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:44:11 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:44:11.614+0000 7f2cc4c46700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:44:11.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:11 smithi125 ceph-mon[131076]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:11.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:11 smithi125 ceph-mon[131076]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-16T16:44:11.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:11 smithi125 ceph-mon[131076]: osdmap e49: 8 total, 7 up, 8 in 2023-11-16T16:44:11.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:11.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:11 smithi125 ceph-mon[131076]: from='osd.7 [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:44:11.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:11 smithi125 ceph-mon[131076]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:44:11.882 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:11 smithi159 ceph-mon[131528]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:11.882 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:11 smithi159 ceph-mon[131528]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-16T16:44:11.882 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:11 smithi159 ceph-mon[131528]: osdmap e49: 8 total, 7 up, 8 in 2023-11-16T16:44:11.882 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:11.882 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:11 smithi159 ceph-mon[131528]: from='osd.7 [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:44:11.882 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:11 smithi159 ceph-mon[131528]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:44:11.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:11 smithi093 ceph-mon[97863]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:11.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:11 smithi093 ceph-mon[97863]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-16T16:44:11.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:11 smithi093 ceph-mon[97863]: osdmap e49: 8 total, 7 up, 8 in 2023-11-16T16:44:11.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:11.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:11 smithi093 ceph-mon[97863]: from='osd.7 [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:44:11.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:11 smithi093 ceph-mon[97863]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:44:12.175 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi159:vg_nvme/lv_2 2023-11-16T16:44:12.821 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:44:12.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2023-11-16T16:44:12.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: osdmap e50: 8 total, 7 up, 8 in 2023-11-16T16:44:12.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:12.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: Detected new or changed devices on smithi125 2023-11-16T16:44:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi125 to 5722M 2023-11-16T16:44:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:44:12.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:12.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:44:12.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:12 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:44:12.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2023-11-16T16:44:12.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: osdmap e50: 8 total, 7 up, 8 in 2023-11-16T16:44:12.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:12.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:12.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: Detected new or changed devices on smithi125 2023-11-16T16:44:12.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:12.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:12.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:12.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:12.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:12.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi125 to 5722M 2023-11-16T16:44:12.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:12.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:12.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:12.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:44:12.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:12.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:44:12.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:12 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:44:13.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2023-11-16T16:44:13.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: osdmap e50: 8 total, 7 up, 8 in 2023-11-16T16:44:13.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:13.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:13.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: Detected new or changed devices on smithi125 2023-11-16T16:44:13.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:13.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:13.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:13.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:13.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:13.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi125 to 5722M 2023-11-16T16:44:13.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:13.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:13.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:13.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:44:13.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:13.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:44:13.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:12 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:44:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:13 smithi093 ceph-mon[97863]: purged_snaps scrub starts 2023-11-16T16:44:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:13 smithi093 ceph-mon[97863]: purged_snaps scrub ok 2023-11-16T16:44:13.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:13 smithi093 ceph-mon[97863]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:13.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:13 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:13.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:13 smithi093 ceph-mon[97863]: osd.7 [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922] boot 2023-11-16T16:44:13.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:13 smithi093 ceph-mon[97863]: osdmap e51: 8 total, 8 up, 8 in 2023-11-16T16:44:13.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:13 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:13.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:13 smithi159 ceph-mon[131528]: purged_snaps scrub starts 2023-11-16T16:44:13.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:13 smithi159 ceph-mon[131528]: purged_snaps scrub ok 2023-11-16T16:44:13.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:13 smithi159 ceph-mon[131528]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:13.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:13 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:13.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:13 smithi159 ceph-mon[131528]: osd.7 [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922] boot 2023-11-16T16:44:13.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:13 smithi159 ceph-mon[131528]: osdmap e51: 8 total, 8 up, 8 in 2023-11-16T16:44:13.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:13 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:14.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:13 smithi125 ceph-mon[131076]: purged_snaps scrub starts 2023-11-16T16:44:14.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:13 smithi125 ceph-mon[131076]: purged_snaps scrub ok 2023-11-16T16:44:14.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:13 smithi125 ceph-mon[131076]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:44:14.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:13 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:14.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:13 smithi125 ceph-mon[131076]: osd.7 [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922] boot 2023-11-16T16:44:14.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:13 smithi125 ceph-mon[131076]: osdmap e51: 8 total, 8 up, 8 in 2023-11-16T16:44:14.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:13 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:44:14.161 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.159+0000 7fcc1209d700 1 -- 172.21.15.159:0/473970074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc0c106020 msgr2=0x7fcc0c108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:14.162 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.159+0000 7fcc1209d700 1 --2- 172.21.15.159:0/473970074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc0c106020 0x7fcc0c108420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400b0a0 tx=0x7fcc0400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:44:14.162 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.159+0000 7fcc1209d700 1 -- 172.21.15.159:0/473970074 shutdown_connections 2023-11-16T16:44:14.162 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.159+0000 7fcc1209d700 1 --2- 172.21.15.159:0/473970074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc0c106020 0x7fcc0c108420 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:14.162 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.159+0000 7fcc1209d700 1 --2- 172.21.15.159:0/473970074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc0c100de0 0x7fcc0c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:14.162 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.159+0000 7fcc1209d700 1 --2- 172.21.15.159:0/473970074 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc0c103700 0x7fcc0c105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:14.162 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.159+0000 7fcc1209d700 1 -- 172.21.15.159:0/473970074 >> 172.21.15.159:0/473970074 conn(0x7fcc0c0fa820 msgr2=0x7fcc0c0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:14.162 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.159+0000 7fcc1209d700 1 -- 172.21.15.159:0/473970074 shutdown_connections 2023-11-16T16:44:14.163 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.159+0000 7fcc1209d700 1 -- 172.21.15.159:0/473970074 wait complete. 2023-11-16T16:44:14.165 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc1209d700 1 Processor -- start 2023-11-16T16:44:14.166 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc1209d700 1 -- start start 2023-11-16T16:44:14.166 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc1209d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc0c100de0 0x7fcc0c196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:14.166 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc1209d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc0c103700 0x7fcc0c196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:14.166 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc1209d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc0c106020 0x7fcc0c19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:14.166 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc1209d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c19dce0 con 0x7fcc0c103700 2023-11-16T16:44:14.166 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc1209d700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c19de20 con 0x7fcc0c106020 2023-11-16T16:44:14.166 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc1209d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c19dfb0 con 0x7fcc0c100de0 2023-11-16T16:44:14.166 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc0b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc0c100de0 0x7fcc0c196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:14.167 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc0b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc0c100de0 0x7fcc0c196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54274/0 (socket says 172.21.15.159:54274) 2023-11-16T16:44:14.167 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc0b7fe700 1 -- 172.21.15.159:0/1497971068 learned_addr learned my addr 172.21.15.159:0/1497971068 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-11-16T16:44:14.167 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc0bfff700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc0c106020 0x7fcc0c19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:14.167 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc0b7fe700 1 -- 172.21.15.159:0/1497971068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc0c106020 msgr2=0x7fcc0c19d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:14.167 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc0b7fe700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc0c106020 0x7fcc0c19d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:14.167 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc0b7fe700 1 -- 172.21.15.159:0/1497971068 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc0c103700 msgr2=0x7fcc0c196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:14.167 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc0affd700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc0c103700 0x7fcc0c196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:14.167 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc0b7fe700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc0c103700 0x7fcc0c196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:14.168 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc0b7fe700 1 -- 172.21.15.159:0/1497971068 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc04009d00 con 0x7fcc0c100de0 2023-11-16T16:44:14.168 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.160+0000 7fcc0affd700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc0c103700 0x7fcc0c196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:44:14.168 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.161+0000 7fcc0b7fe700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc0c100de0 0x7fcc0c196150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000c950 tx=0x7fcc0000cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:14.168 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.161+0000 7fcc08ff9700 1 -- 172.21.15.159:0/1497971068 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00007ac0 con 0x7fcc0c100de0 2023-11-16T16:44:14.168 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.161+0000 7fcc08ff9700 1 -- 172.21.15.159:0/1497971068 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc00004cf0 con 0x7fcc0c100de0 2023-11-16T16:44:14.168 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.161+0000 7fcc1209d700 1 -- 172.21.15.159:0/1497971068 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c10bcc0 con 0x7fcc0c100de0 2023-11-16T16:44:14.168 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.161+0000 7fcc08ff9700 1 -- 172.21.15.159:0/1497971068 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00005550 con 0x7fcc0c100de0 2023-11-16T16:44:14.168 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.161+0000 7fcc1209d700 1 -- 172.21.15.159:0/1497971068 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c10bb30 con 0x7fcc0c100de0 2023-11-16T16:44:14.168 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.162+0000 7fcc1209d700 1 -- 172.21.15.159:0/1497971068 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c0fc390 con 0x7fcc0c100de0 2023-11-16T16:44:14.169 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.163+0000 7fcc08ff9700 1 -- 172.21.15.159:0/1497971068 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcc00004750 con 0x7fcc0c100de0 2023-11-16T16:44:14.169 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.163+0000 7fcc08ff9700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcbf40a1150 0x7fcbf40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:14.169 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.163+0000 7fcc08ff9700 1 -- 172.21.15.159:0/1497971068 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fcc000cb9d0 con 0x7fcc0c100de0 2023-11-16T16:44:14.169 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.164+0000 7fcc0affd700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcbf40a1150 0x7fcbf40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:14.169 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.164+0000 7fcc0affd700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcbf40a1150 0x7fcbf40a3600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc007900 tx=0x7fcbfc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:14.170 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.168+0000 7fcc08ff9700 1 -- 172.21.15.159:0/1497971068 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcc00095c60 con 0x7fcc0c100de0 2023-11-16T16:44:14.362 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:14.361+0000 7fcc1209d700 1 -- 172.21.15.159:0/1497971068 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc0c061c80 con 0x7fcbf40a1150 2023-11-16T16:44:14.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:14 smithi093 ceph-mon[97863]: osdmap e52: 8 total, 8 up, 8 in 2023-11-16T16:44:14.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:14 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:44:14.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:14 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:44:14.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:14 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:14.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:14 smithi159 ceph-mon[131528]: osdmap e52: 8 total, 8 up, 8 in 2023-11-16T16:44:14.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:14 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:44:14.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:14 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:44:14.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:14 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:15.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:14 smithi125 ceph-mon[131076]: osdmap e52: 8 total, 8 up, 8 in 2023-11-16T16:44:15.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:14 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:44:15.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:14 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:44:15.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:14 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:15.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:15 smithi093 ceph-mon[97863]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:44:15.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:15 smithi093 ceph-mon[97863]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:15.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:15 smithi093 ceph-mon[97863]: osdmap e53: 8 total, 8 up, 8 in 2023-11-16T16:44:15.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:15 smithi093 ceph-mon[97863]: osdmap e54: 8 total, 8 up, 8 in 2023-11-16T16:44:15.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:15 smithi159 ceph-mon[131528]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:44:15.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:15 smithi159 ceph-mon[131528]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:15.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:15 smithi159 ceph-mon[131528]: osdmap e53: 8 total, 8 up, 8 in 2023-11-16T16:44:15.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:15 smithi159 ceph-mon[131528]: osdmap e54: 8 total, 8 up, 8 in 2023-11-16T16:44:16.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:15 smithi125 ceph-mon[131076]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:44:16.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:15 smithi125 ceph-mon[131076]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:16.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:15 smithi125 ceph-mon[131076]: osdmap e53: 8 total, 8 up, 8 in 2023-11-16T16:44:16.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:15 smithi125 ceph-mon[131076]: osdmap e54: 8 total, 8 up, 8 in 2023-11-16T16:44:17.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:17 smithi159 ceph-mon[131528]: osdmap e55: 8 total, 8 up, 8 in 2023-11-16T16:44:17.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:17 smithi159 ceph-mon[131528]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:17.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:17 smithi159 ceph-mon[131528]: from='client.? 172.21.15.159:0/1364498740' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4ec0a32-5a4c-441b-97cb-548c992ec12d"}]: dispatch 2023-11-16T16:44:17.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:17 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4ec0a32-5a4c-441b-97cb-548c992ec12d"}]: dispatch 2023-11-16T16:44:17.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:17 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4ec0a32-5a4c-441b-97cb-548c992ec12d"}]': finished 2023-11-16T16:44:17.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:17 smithi159 ceph-mon[131528]: osdmap e56: 9 total, 8 up, 9 in 2023-11-16T16:44:17.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:17.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:17 smithi125 ceph-mon[131076]: osdmap e55: 8 total, 8 up, 8 in 2023-11-16T16:44:17.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:17 smithi125 ceph-mon[131076]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:17.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:17 smithi125 ceph-mon[131076]: from='client.? 172.21.15.159:0/1364498740' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4ec0a32-5a4c-441b-97cb-548c992ec12d"}]: dispatch 2023-11-16T16:44:17.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:17 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4ec0a32-5a4c-441b-97cb-548c992ec12d"}]: dispatch 2023-11-16T16:44:17.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:17 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4ec0a32-5a4c-441b-97cb-548c992ec12d"}]': finished 2023-11-16T16:44:17.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:17 smithi125 ceph-mon[131076]: osdmap e56: 9 total, 8 up, 9 in 2023-11-16T16:44:17.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:17.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:17 smithi093 ceph-mon[97863]: osdmap e55: 8 total, 8 up, 8 in 2023-11-16T16:44:17.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:17 smithi093 ceph-mon[97863]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:17 smithi093 ceph-mon[97863]: from='client.? 172.21.15.159:0/1364498740' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4ec0a32-5a4c-441b-97cb-548c992ec12d"}]: dispatch 2023-11-16T16:44:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:17 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4ec0a32-5a4c-441b-97cb-548c992ec12d"}]: dispatch 2023-11-16T16:44:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:17 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4ec0a32-5a4c-441b-97cb-548c992ec12d"}]': finished 2023-11-16T16:44:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:17 smithi093 ceph-mon[97863]: osdmap e56: 9 total, 8 up, 9 in 2023-11-16T16:44:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:18.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:18 smithi159 ceph-mon[131528]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-11-16T16:44:18.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:18 smithi159 ceph-mon[131528]: from='client.? 172.21.15.159:0/1724634764' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:44:18.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:18 smithi125 ceph-mon[131076]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-11-16T16:44:18.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:18 smithi125 ceph-mon[131076]: from='client.? 172.21.15.159:0/1724634764' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:44:18.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:18 smithi093 ceph-mon[97863]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-11-16T16:44:18.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:18 smithi093 ceph-mon[97863]: from='client.? 172.21.15.159:0/1724634764' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:44:19.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:19 smithi125 ceph-mon[131076]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:19.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:19 smithi093 ceph-mon[97863]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:19.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:19 smithi159 ceph-mon[131528]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:21.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:21 smithi125 ceph-mon[131076]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 49 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:21.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:21 smithi093 ceph-mon[97863]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 49 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:21.661 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:21 smithi159 ceph-mon[131528]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 49 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:22.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:22 smithi159 ceph-mon[131528]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-11-16T16:44:22.503 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:22 smithi159 ceph-mon[131528]: Cluster is now healthy 2023-11-16T16:44:22.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:22 smithi125 ceph-mon[131076]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-11-16T16:44:22.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:22 smithi125 ceph-mon[131076]: Cluster is now healthy 2023-11-16T16:44:22.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:22 smithi093 ceph-mon[97863]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-11-16T16:44:22.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:22 smithi093 ceph-mon[97863]: Cluster is now healthy 2023-11-16T16:44:23.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:23 smithi125 ceph-mon[131076]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:23.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:23 smithi093 ceph-mon[97863]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:23.734 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:23 smithi159 ceph-mon[131528]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:24.321 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-16T16:44:24.322 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:24.322 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:24 smithi159 ceph-mon[131528]: Deploying daemon osd.8 on smithi159 2023-11-16T16:44:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-16T16:44:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:24 smithi125 ceph-mon[131076]: Deploying daemon osd.8 on smithi159 2023-11-16T16:44:24.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-16T16:44:24.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:24.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:24 smithi093 ceph-mon[97863]: Deploying daemon osd.8 on smithi159 2023-11-16T16:44:25.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:25 smithi159 ceph-mon[131528]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:25.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:25 smithi125 ceph-mon[131076]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:25.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:25 smithi093 ceph-mon[97863]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:27.801 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:27 smithi159 ceph-mon[131528]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:27.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:27 smithi125 ceph-mon[131076]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:27.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:27 smithi093 ceph-mon[97863]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:29.038 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:29.038 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:29.038 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:29.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:29.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:29.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:29.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:29.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:29.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:30.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:29 smithi093 ceph-mon[97863]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:30.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:29 smithi159 ceph-mon[131528]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:30.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:29 smithi125 ceph-mon[131076]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:30.726 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.724+0000 7fcc08ff9700 1 -- 172.21.15.159:0/1497971068 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcc0c061c80 con 0x7fcbf40a1150 2023-11-16T16:44:30.727 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 8 on host 'smithi159' 2023-11-16T16:44:30.729 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.727+0000 7fcc1209d700 1 -- 172.21.15.159:0/1497971068 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcbf40a1150 msgr2=0x7fcbf40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:30.729 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.727+0000 7fcc1209d700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcbf40a1150 0x7fcbf40a3600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc007900 tx=0x7fcbfc008040 comp rx=0 tx=0).stop 2023-11-16T16:44:30.729 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.727+0000 7fcc1209d700 1 -- 172.21.15.159:0/1497971068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc0c100de0 msgr2=0x7fcc0c196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:30.729 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.727+0000 7fcc1209d700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc0c100de0 0x7fcc0c196150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000c950 tx=0x7fcc0000cd10 comp rx=0 tx=0).stop 2023-11-16T16:44:30.729 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.727+0000 7fcc1209d700 1 -- 172.21.15.159:0/1497971068 shutdown_connections 2023-11-16T16:44:30.729 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.727+0000 7fcc1209d700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcbf40a1150 0x7fcbf40a3600 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:30.729 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.727+0000 7fcc1209d700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc0c106020 0x7fcc0c19d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:30.730 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.727+0000 7fcc1209d700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc0c100de0 0x7fcc0c196150 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:30.730 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.727+0000 7fcc1209d700 1 --2- 172.21.15.159:0/1497971068 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc0c103700 0x7fcc0c196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:30.730 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.727+0000 7fcc1209d700 1 -- 172.21.15.159:0/1497971068 >> 172.21.15.159:0/1497971068 conn(0x7fcc0c0fa820 msgr2=0x7fcc0c101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:30.730 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.727+0000 7fcc1209d700 1 -- 172.21.15.159:0/1497971068 shutdown_connections 2023-11-16T16:44:30.730 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:44:30.727+0000 7fcc1209d700 1 -- 172.21.15.159:0/1497971068 wait complete. 2023-11-16T16:44:31.113 DEBUG:teuthology.orchestra.run.smithi159:osd.8> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.8.service 2023-11-16T16:44:31.116 INFO:tasks.cephadm:Deploying osd.9 on smithi093 with /dev/vg_nvme/lv_1... 2023-11-16T16:44:31.116 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-16T16:44:31.144 INFO:journalctl@ceph.osd.8.smithi159.stdout:-- Logs begin at Thu 2023-11-16 16:15:38 UTC. -- 2023-11-16T16:44:31.408 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.408 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.408 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:31.408 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:31.408 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:44:31.408 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:31 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:31 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:31 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:31 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:31 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:44:31.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:31 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:31 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:44:31.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:31 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:44:31.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:31 smithi093 ceph-mon[97863]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:31.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:31 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:31 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:31 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:31.663 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:44:31.664 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:44:31.664 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-mon[131528]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:31.664 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.664 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.664 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:31 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:31 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:31 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:31 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:31 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:44:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:31 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:31 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:44:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:31 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:44:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:31 smithi125 ceph-mon[131076]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:31 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:31 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:31 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:31.962 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:44:31 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:44:31.798+0000 7fc09e51b640 -1 osd.8 0 log_to_monitors true 2023-11-16T16:44:31.965 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:44:32.571 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:32 smithi093 ceph-mon[97863]: from='osd.8 [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-16T16:44:32.571 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:32 smithi093 ceph-mon[97863]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-16T16:44:32.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:32 smithi159 ceph-mon[131528]: from='osd.8 [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-16T16:44:32.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:32 smithi159 ceph-mon[131528]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-16T16:44:32.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:32 smithi125 ceph-mon[131076]: from='osd.8 [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-16T16:44:32.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:32 smithi125 ceph-mon[131076]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-16T16:44:33.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-16T16:44:33.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: osdmap e57: 9 total, 8 up, 9 in 2023-11-16T16:44:33.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:33.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='osd.8 [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:44:33.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:44:33.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:33.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:33.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:33.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:33.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:33.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:33.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:33.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:33.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:33.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:33.735 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:44:33.438+0000 7fc093b92700 -1 osd.8 0 waiting for initial osdmap 2023-11-16T16:44:33.735 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:44:33.451+0000 7fc08c180700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:44:33.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-16T16:44:33.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: osdmap e57: 9 total, 8 up, 9 in 2023-11-16T16:44:33.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:33.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='osd.8 [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:44:33.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:44:33.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:33.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:33.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:33.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:33.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:33.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:33.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:33.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:33.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:33.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:33.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-16T16:44:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: osdmap e57: 9 total, 8 up, 9 in 2023-11-16T16:44:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='osd.8 [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:44:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:44:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:44:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:33.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:33.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:33.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:33.979 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-16T16:44:33.979 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-16T16:44:33.979 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-11-16T16:44:33.979 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-11-16T16:44:33.979 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0400831 s, 262 MB/s 2023-11-16T16:44:33.979 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-11-16T16:44:34.445 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi093:vg_nvme/lv_1 2023-11-16T16:44:34.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:34 smithi159 ceph-mon[131528]: Detected new or changed devices on smithi159 2023-11-16T16:44:34.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:34 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi159 to 5722M 2023-11-16T16:44:34.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:34 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:44:34.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:34 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:44:34.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:34 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:44:34.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:34 smithi159 ceph-mon[131528]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-11-16T16:44:34.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:34 smithi159 ceph-mon[131528]: osdmap e58: 9 total, 8 up, 9 in 2023-11-16T16:44:34.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:34 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:34.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:34 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:34 smithi125 ceph-mon[131076]: Detected new or changed devices on smithi159 2023-11-16T16:44:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:34 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi159 to 5722M 2023-11-16T16:44:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:34 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:44:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:34 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:44:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:34 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:44:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:34 smithi125 ceph-mon[131076]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-11-16T16:44:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:34 smithi125 ceph-mon[131076]: osdmap e58: 9 total, 8 up, 9 in 2023-11-16T16:44:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:34 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:34 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:34.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:34 smithi093 ceph-mon[97863]: Detected new or changed devices on smithi159 2023-11-16T16:44:34.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:34 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi159 to 5722M 2023-11-16T16:44:34.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:34 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:44:34.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:34 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:44:34.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:34 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:44:34.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:34 smithi093 ceph-mon[97863]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-11-16T16:44:34.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:34 smithi093 ceph-mon[97863]: osdmap e58: 9 total, 8 up, 9 in 2023-11-16T16:44:34.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:34 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:34.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:34 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:35.290 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:44:35.719 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:35 smithi093 ceph-mon[97863]: purged_snaps scrub starts 2023-11-16T16:44:35.719 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:35 smithi093 ceph-mon[97863]: purged_snaps scrub ok 2023-11-16T16:44:35.719 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:35 smithi093 ceph-mon[97863]: osd.8 [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083] boot 2023-11-16T16:44:35.720 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:35 smithi093 ceph-mon[97863]: osdmap e59: 9 total, 9 up, 9 in 2023-11-16T16:44:35.720 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:35 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:35.720 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:35 smithi093 ceph-mon[97863]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:35.720 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:35 smithi093 ceph-mon[97863]: osdmap e60: 9 total, 9 up, 9 in 2023-11-16T16:44:35.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:35 smithi159 ceph-mon[131528]: purged_snaps scrub starts 2023-11-16T16:44:35.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:35 smithi159 ceph-mon[131528]: purged_snaps scrub ok 2023-11-16T16:44:35.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:35 smithi159 ceph-mon[131528]: osd.8 [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083] boot 2023-11-16T16:44:35.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:35 smithi159 ceph-mon[131528]: osdmap e59: 9 total, 9 up, 9 in 2023-11-16T16:44:35.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:35.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:35 smithi159 ceph-mon[131528]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:35.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:35 smithi159 ceph-mon[131528]: osdmap e60: 9 total, 9 up, 9 in 2023-11-16T16:44:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:35 smithi125 ceph-mon[131076]: purged_snaps scrub starts 2023-11-16T16:44:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:35 smithi125 ceph-mon[131076]: purged_snaps scrub ok 2023-11-16T16:44:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:35 smithi125 ceph-mon[131076]: osd.8 [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083] boot 2023-11-16T16:44:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:35 smithi125 ceph-mon[131076]: osdmap e59: 9 total, 9 up, 9 in 2023-11-16T16:44:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:35 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:44:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:35 smithi125 ceph-mon[131076]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:44:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:35 smithi125 ceph-mon[131076]: osdmap e60: 9 total, 9 up, 9 in 2023-11-16T16:44:36.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.790+0000 7f321fe1c700 1 -- 172.21.15.93:0/791333371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32181014f0 msgr2=0x7f3218101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:36.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.790+0000 7f321fe1c700 1 --2- 172.21.15.93:0/791333371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32181014f0 0x7f3218101940 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3208009990 tx=0x7f3208009ca0 comp rx=0 tx=0).stop 2023-11-16T16:44:36.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.790+0000 7f321fe1c700 1 -- 172.21.15.93:0/791333371 shutdown_connections 2023-11-16T16:44:36.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.790+0000 7f321fe1c700 1 --2- 172.21.15.93:0/791333371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3218101e80 0x7f3218108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:36.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.790+0000 7f321fe1c700 1 --2- 172.21.15.93:0/791333371 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32181002f0 0x7f3218100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:36.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.790+0000 7f321fe1c700 1 --2- 172.21.15.93:0/791333371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32181014f0 0x7f3218101940 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:36.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.790+0000 7f321fe1c700 1 -- 172.21.15.93:0/791333371 >> 172.21.15.93:0/791333371 conn(0x7f32180fb900 msgr2=0x7f32180fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:36.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.790+0000 7f321fe1c700 1 -- 172.21.15.93:0/791333371 shutdown_connections 2023-11-16T16:44:36.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.790+0000 7f321fe1c700 1 -- 172.21.15.93:0/791333371 wait complete. 2023-11-16T16:44:36.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.791+0000 7f321fe1c700 1 Processor -- start 2023-11-16T16:44:36.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.791+0000 7f321fe1c700 1 -- start start 2023-11-16T16:44:36.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321fe1c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32181002f0 0x7f3218196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:36.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321fe1c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32181014f0 0x7f3218196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:36.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321fe1c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3218101e80 0x7f321819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:36.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321fe1c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321819dbd0 con 0x7f32181014f0 2023-11-16T16:44:36.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321fe1c700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321819dd10 con 0x7f32181002f0 2023-11-16T16:44:36.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321fe1c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321819dea0 con 0x7f3218101e80 2023-11-16T16:44:36.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321d3b7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32181014f0 0x7f3218196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:36.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321d3b7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32181014f0 0x7f3218196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41890/0 (socket says 172.21.15.93:41890) 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321d3b7700 1 -- 172.21.15.93:0/2581454754 learned_addr learned my addr 172.21.15.93:0/2581454754 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321dbb8700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32181002f0 0x7f3218196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321e3b9700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3218101e80 0x7f321819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321d3b7700 1 -- 172.21.15.93:0/2581454754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3218101e80 msgr2=0x7f321819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321d3b7700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3218101e80 0x7f321819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321d3b7700 1 -- 172.21.15.93:0/2581454754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32181002f0 msgr2=0x7f3218196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321d3b7700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32181002f0 0x7f3218196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321d3b7700 1 -- 172.21.15.93:0/2581454754 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32080095f0 con 0x7f32181014f0 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321e3b9700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3218101e80 0x7f321819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.792+0000 7f321dbb8700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32181002f0 0x7f3218196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.793+0000 7f321d3b7700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32181014f0 0x7f3218196580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3208004770 tx=0x7f3208009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.793+0000 7f320effd700 1 -- 172.21.15.93:0/2581454754 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320801d070 con 0x7f32181014f0 2023-11-16T16:44:36.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.793+0000 7f320effd700 1 -- 172.21.15.93:0/2581454754 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32080054f0 con 0x7f32181014f0 2023-11-16T16:44:36.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.793+0000 7f321fe1c700 1 -- 172.21.15.93:0/2581454754 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321810bc90 con 0x7f32181014f0 2023-11-16T16:44:36.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.793+0000 7f320effd700 1 -- 172.21.15.93:0/2581454754 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208004c70 con 0x7f32181014f0 2023-11-16T16:44:36.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.793+0000 7f321fe1c700 1 -- 172.21.15.93:0/2581454754 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321810bb00 con 0x7f32181014f0 2023-11-16T16:44:36.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.794+0000 7f321fe1c700 1 -- 172.21.15.93:0/2581454754 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3218107230 con 0x7f32181014f0 2023-11-16T16:44:36.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.796+0000 7f320effd700 1 -- 172.21.15.93:0/2581454754 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f320800fbc0 con 0x7f32181014f0 2023-11-16T16:44:36.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.797+0000 7f320effd700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f32040a1150 0x7f32040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:36.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.797+0000 7f320effd700 1 -- 172.21.15.93:0/2581454754 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f32080cd250 con 0x7f32181014f0 2023-11-16T16:44:36.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.797+0000 7f321dbb8700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f32040a1150 0x7f32040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:36.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.798+0000 7f321dbb8700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f32040a1150 0x7f32040a3600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3218101350 tx=0x7f3214009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:36.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:36.801+0000 7f320effd700 1 -- 172.21.15.93:0/2581454754 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f320805a730 con 0x7f32181014f0 2023-11-16T16:44:37.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:37.005+0000 7f321fe1c700 1 -- 172.21.15.93:0/2581454754 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3218061ac0 con 0x7f32040a1150 2023-11-16T16:44:37.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:37 smithi125 ceph-mon[131076]: osdmap e61: 9 total, 9 up, 9 in 2023-11-16T16:44:37.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:37 smithi125 ceph-mon[131076]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:37.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:37 smithi125 ceph-mon[131076]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:44:37.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:37 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:44:37.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:37 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:44:37.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:37 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:37.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:37 smithi093 ceph-mon[97863]: osdmap e61: 9 total, 9 up, 9 in 2023-11-16T16:44:37.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:37 smithi093 ceph-mon[97863]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:37.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:37 smithi093 ceph-mon[97863]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:44:37.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:37 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:44:37.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:37 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:44:37.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:37 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:37.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:37 smithi159 ceph-mon[131528]: osdmap e61: 9 total, 9 up, 9 in 2023-11-16T16:44:37.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:37 smithi159 ceph-mon[131528]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:37.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:37 smithi159 ceph-mon[131528]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:44:37.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:37 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:44:37.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:37 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:44:37.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:37 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:38.478 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:38 smithi093 ceph-mon[97863]: osdmap e62: 9 total, 9 up, 9 in 2023-11-16T16:44:38.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:38 smithi125 ceph-mon[131076]: osdmap e62: 9 total, 9 up, 9 in 2023-11-16T16:44:38.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:38 smithi159 ceph-mon[131528]: osdmap e62: 9 total, 9 up, 9 in 2023-11-16T16:44:39.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:39 smithi125 ceph-mon[131076]: osdmap e63: 9 total, 9 up, 9 in 2023-11-16T16:44:39.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:39 smithi125 ceph-mon[131076]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:39.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:39 smithi093 ceph-mon[97863]: osdmap e63: 9 total, 9 up, 9 in 2023-11-16T16:44:39.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:39 smithi093 ceph-mon[97863]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:39.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:39 smithi159 ceph-mon[131528]: osdmap e63: 9 total, 9 up, 9 in 2023-11-16T16:44:39.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:39 smithi159 ceph-mon[131528]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:40.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:40 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2807803454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed"}]: dispatch 2023-11-16T16:44:40.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:40 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed"}]: dispatch 2023-11-16T16:44:40.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:40 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed"}]': finished 2023-11-16T16:44:40.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:40 smithi125 ceph-mon[131076]: osdmap e64: 10 total, 9 up, 10 in 2023-11-16T16:44:40.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:40 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:44:40.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:40 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2807803454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed"}]: dispatch 2023-11-16T16:44:40.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:40 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed"}]: dispatch 2023-11-16T16:44:40.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:40 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed"}]': finished 2023-11-16T16:44:40.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:40 smithi093 ceph-mon[97863]: osdmap e64: 10 total, 9 up, 10 in 2023-11-16T16:44:40.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:40 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:44:40.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:40 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2807803454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed"}]: dispatch 2023-11-16T16:44:40.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:40 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed"}]: dispatch 2023-11-16T16:44:40.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:40 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed"}]': finished 2023-11-16T16:44:40.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:40 smithi159 ceph-mon[131528]: osdmap e64: 10 total, 9 up, 10 in 2023-11-16T16:44:40.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:40 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:44:41.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:41 smithi125 ceph-mon[131076]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 136 KiB/s, 0 objects/s recovering 2023-11-16T16:44:41.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:41 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/435877034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:44:41.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:41 smithi093 ceph-mon[97863]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 136 KiB/s, 0 objects/s recovering 2023-11-16T16:44:41.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:41 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/435877034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:44:41.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:41 smithi159 ceph-mon[131528]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 136 KiB/s, 0 objects/s recovering 2023-11-16T16:44:41.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:41 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/435877034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:44:43.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:43 smithi125 ceph-mon[131076]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-16T16:44:43.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:43 smithi093 ceph-mon[97863]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-16T16:44:43.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:43 smithi159 ceph-mon[131528]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-16T16:44:45.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:45 smithi125 ceph-mon[131076]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-11-16T16:44:45.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:45 smithi093 ceph-mon[97863]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-11-16T16:44:45.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:45 smithi159 ceph-mon[131528]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-11-16T16:44:47.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:47 smithi093 ceph-mon[97863]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-11-16T16:44:47.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:47 smithi125 ceph-mon[131076]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-11-16T16:44:47.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:47 smithi159 ceph-mon[131528]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-11-16T16:44:48.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-16T16:44:48.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:48 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:48.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:48 smithi125 ceph-mon[131076]: Deploying daemon osd.9 on smithi093 2023-11-16T16:44:48.833 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-16T16:44:48.833 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:48 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:48.833 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:48 smithi093 ceph-mon[97863]: Deploying daemon osd.9 on smithi093 2023-11-16T16:44:48.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-16T16:44:48.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:48 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:48.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:48 smithi159 ceph-mon[131528]: Deploying daemon osd.9 on smithi093 2023-11-16T16:44:49.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:49 smithi125 ceph-mon[131076]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-16T16:44:49.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:49 smithi093 ceph-mon[97863]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-16T16:44:49.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:49 smithi159 ceph-mon[131528]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-16T16:44:51.547 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:51 smithi093 ceph-mon[97863]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:51 smithi125 ceph-mon[131076]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:51.734 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:51 smithi159 ceph-mon[131528]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:53.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:53 smithi125 ceph-mon[131076]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:53.899 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:53 smithi093 ceph-mon[97863]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:53.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:53 smithi159 ceph-mon[131528]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:55.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:55.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:55.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:55.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:55 smithi159 ceph-mon[131528]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:55.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:55.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:55.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:55.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:55 smithi125 ceph-mon[131076]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:55.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:55.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:55.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:55.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:55 smithi093 ceph-mon[97863]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:57.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:57 smithi125 ceph-mon[131076]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:57.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:57 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:57.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:57 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:57.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:57 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:57.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:57 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:57.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:57 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:57.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:57 smithi093 ceph-mon[97863]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:57.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:57 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:57.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:57 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:57.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:57 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:57.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:57 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:57.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:57 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:57 smithi159 ceph-mon[131528]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:57 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:57 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:57 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:57 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:57 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:58.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.323+0000 7f320effd700 1 -- 172.21.15.93:0/2581454754 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3218061ac0 con 0x7f32040a1150 2023-11-16T16:44:58.325 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 9 on host 'smithi093' 2023-11-16T16:44:58.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.325+0000 7f321fe1c700 1 -- 172.21.15.93:0/2581454754 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f32040a1150 msgr2=0x7f32040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:58.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.325+0000 7f321fe1c700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f32040a1150 0x7f32040a3600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3218101350 tx=0x7f3214009260 comp rx=0 tx=0).stop 2023-11-16T16:44:58.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.325+0000 7f321fe1c700 1 -- 172.21.15.93:0/2581454754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32181014f0 msgr2=0x7f3218196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:58.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.325+0000 7f321fe1c700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32181014f0 0x7f3218196580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3208004770 tx=0x7f3208009580 comp rx=0 tx=0).stop 2023-11-16T16:44:58.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.325+0000 7f321fe1c700 1 -- 172.21.15.93:0/2581454754 shutdown_connections 2023-11-16T16:44:58.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.325+0000 7f321fe1c700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f32040a1150 0x7f32040a3600 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:58.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.325+0000 7f321fe1c700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3218101e80 0x7f321819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:58.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.325+0000 7f321fe1c700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32181002f0 0x7f3218196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:58.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.325+0000 7f321fe1c700 1 --2- 172.21.15.93:0/2581454754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32181014f0 0x7f3218196580 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:58.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.325+0000 7f321fe1c700 1 -- 172.21.15.93:0/2581454754 >> 172.21.15.93:0/2581454754 conn(0x7f32180fb900 msgr2=0x7f32180fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:58.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.326+0000 7f321fe1c700 1 -- 172.21.15.93:0/2581454754 shutdown_connections 2023-11-16T16:44:58.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:44:58.326+0000 7f321fe1c700 1 -- 172.21.15.93:0/2581454754 wait complete. 2023-11-16T16:44:58.821 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:58 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:44:58.822 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:58 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:44:58.822 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:58 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:44:58.822 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:58 smithi093 ceph-mon[97863]: from='osd.9 [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-16T16:44:58.822 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:58 smithi093 ceph-mon[97863]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-16T16:44:58.822 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:58.822 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:58.822 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:58.825 DEBUG:teuthology.orchestra.run.smithi093:osd.9> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.9.service 2023-11-16T16:44:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:58 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:44:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:58 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:44:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:58 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:44:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:58 smithi125 ceph-mon[131076]: from='osd.9 [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-16T16:44:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:58 smithi125 ceph-mon[131076]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-16T16:44:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:58.829 INFO:tasks.cephadm:Deploying osd.10 on smithi125 with /dev/vg_nvme/lv_1... 2023-11-16T16:44:58.829 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-16T16:44:58.901 INFO:journalctl@ceph.osd.9.smithi093.stdout:-- Logs begin at Thu 2023-11-16 16:19:38 UTC. -- 2023-11-16T16:44:58.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:58 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:44:58.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:58 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:44:58.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:58 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:44:58.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:58 smithi159 ceph-mon[131528]: from='osd.9 [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-16T16:44:58.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:58 smithi159 ceph-mon[131528]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-16T16:44:58.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:58.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:44:58.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:59.484 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:44:59.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:59 smithi125 ceph-mon[131076]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:59.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:59 smithi125 ceph-mon[131076]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-16T16:44:59.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:59 smithi125 ceph-mon[131076]: osdmap e65: 10 total, 9 up, 10 in 2023-11-16T16:44:59.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:59 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:44:59.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:59 smithi125 ceph-mon[131076]: from='osd.9 [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:44:59.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:44:59 smithi125 ceph-mon[131076]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:44:59.858 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:59 smithi093 ceph-mon[97863]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:59.858 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:59 smithi093 ceph-mon[97863]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-16T16:44:59.858 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:59 smithi093 ceph-mon[97863]: osdmap e65: 10 total, 9 up, 10 in 2023-11-16T16:44:59.859 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:59 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:44:59.859 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:59 smithi093 ceph-mon[97863]: from='osd.9 [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:44:59.859 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:44:59 smithi093 ceph-mon[97863]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:44:59.859 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:44:59 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:44:59.561+0000 7f9830cb6700 -1 osd.9 0 waiting for initial osdmap 2023-11-16T16:44:59.859 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:44:59 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:44:59.579+0000 7f982b2a8700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:44:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:59 smithi159 ceph-mon[131528]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:44:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:59 smithi159 ceph-mon[131528]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-16T16:44:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:59 smithi159 ceph-mon[131528]: osdmap e65: 10 total, 9 up, 10 in 2023-11-16T16:44:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:59 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:44:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:59 smithi159 ceph-mon[131528]: from='osd.9 [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:44:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:44:59 smithi159 ceph-mon[131528]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-11-16T16:45:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:00 smithi125 ceph-mon[131076]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-11-16T16:45:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:00 smithi125 ceph-mon[131076]: osdmap e66: 10 total, 9 up, 10 in 2023-11-16T16:45:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:45:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:45:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:00 smithi093 ceph-mon[97863]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-11-16T16:45:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:00 smithi093 ceph-mon[97863]: osdmap e66: 10 total, 9 up, 10 in 2023-11-16T16:45:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:45:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:45:00.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:00 smithi159 ceph-mon[131528]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-11-16T16:45:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:00 smithi159 ceph-mon[131528]: osdmap e66: 10 total, 9 up, 10 in 2023-11-16T16:45:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:45:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:45:01.006 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-16T16:45:01.006 INFO:teuthology.orchestra.run.smithi125.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-16T16:45:01.006 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10+0 records in 2023-11-16T16:45:01.006 INFO:teuthology.orchestra.run.smithi125.stderr:10+0 records out 2023-11-16T16:45:01.006 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117698 s, 891 MB/s 2023-11-16T16:45:01.006 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping successful for: 2023-11-16T16:45:01.443 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi125:vg_nvme/lv_1 2023-11-16T16:45:01.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:01 smithi125 ceph-mon[131076]: purged_snaps scrub starts 2023-11-16T16:45:01.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:01 smithi125 ceph-mon[131076]: purged_snaps scrub ok 2023-11-16T16:45:01.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:01 smithi125 ceph-mon[131076]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:45:01.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:01 smithi125 ceph-mon[131076]: osd.9 [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260] boot 2023-11-16T16:45:01.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:01 smithi125 ceph-mon[131076]: osdmap e67: 10 total, 10 up, 10 in 2023-11-16T16:45:01.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:01 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:45:01.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:01 smithi093 ceph-mon[97863]: purged_snaps scrub starts 2023-11-16T16:45:01.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:01 smithi093 ceph-mon[97863]: purged_snaps scrub ok 2023-11-16T16:45:01.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:01 smithi093 ceph-mon[97863]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:45:01.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:01 smithi093 ceph-mon[97863]: osd.9 [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260] boot 2023-11-16T16:45:01.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:01 smithi093 ceph-mon[97863]: osdmap e67: 10 total, 10 up, 10 in 2023-11-16T16:45:01.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:01 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:45:01.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:01 smithi159 ceph-mon[131528]: purged_snaps scrub starts 2023-11-16T16:45:01.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:01 smithi159 ceph-mon[131528]: purged_snaps scrub ok 2023-11-16T16:45:01.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:01 smithi159 ceph-mon[131528]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:45:01.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:01 smithi159 ceph-mon[131528]: osd.9 [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260] boot 2023-11-16T16:45:01.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:01 smithi159 ceph-mon[131528]: osdmap e67: 10 total, 10 up, 10 in 2023-11-16T16:45:01.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:01 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:45:02.085 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:45:03.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: Detected new or changed devices on smithi093 2023-11-16T16:45:03.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:03.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:03.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:03.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:03.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:03.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:03.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi093 to 4317M 2023-11-16T16:45:03.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:03.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:03.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:03.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:45:03.330 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:03.330 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:45:03.330 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:45:03.330 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:03 smithi125 ceph-mon[131076]: osdmap e68: 10 total, 10 up, 10 in 2023-11-16T16:45:03.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.393+0000 7fb1dec8d700 1 -- 172.21.15.125:0/1442572956 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1d8100d70 msgr2=0x7fb1d8103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:03.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.393+0000 7fb1dec8d700 1 --2- 172.21.15.125:0/1442572956 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1d8100d70 0x7fb1d8103150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009990 tx=0x7fb1cc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:45:03.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.393+0000 7fb1dec8d700 1 -- 172.21.15.125:0/1442572956 shutdown_connections 2023-11-16T16:45:03.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.393+0000 7fb1dec8d700 1 --2- 172.21.15.125:0/1442572956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d8105fb0 0x7fb1d8108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:03.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.393+0000 7fb1dec8d700 1 --2- 172.21.15.125:0/1442572956 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1d8100d70 0x7fb1d8103150 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:03.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.393+0000 7fb1dec8d700 1 --2- 172.21.15.125:0/1442572956 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1d8103690 0x7fb1d8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:03.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.393+0000 7fb1dec8d700 1 -- 172.21.15.125:0/1442572956 >> 172.21.15.125:0/1442572956 conn(0x7fb1d80fa7f0 msgr2=0x7fb1d80fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:03.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.393+0000 7fb1dec8d700 1 -- 172.21.15.125:0/1442572956 shutdown_connections 2023-11-16T16:45:03.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.393+0000 7fb1dec8d700 1 -- 172.21.15.125:0/1442572956 wait complete. 2023-11-16T16:45:03.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1dec8d700 1 Processor -- start 2023-11-16T16:45:03.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1dec8d700 1 -- start start 2023-11-16T16:45:03.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1dec8d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1d8100d70 0x7fb1d8196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:03.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1dec8d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1d8103690 0x7fb1d8196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:03.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1dec8d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d8105fb0 0x7fb1d819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:03.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1dec8d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d819dbc0 con 0x7fb1d8100d70 2023-11-16T16:45:03.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1dec8d700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d819dd00 con 0x7fb1d8103690 2023-11-16T16:45:03.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1dec8d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d819de90 con 0x7fb1d8105fb0 2023-11-16T16:45:03.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1d7fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1d8103690 0x7fb1d8196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:03.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1d7fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1d8103690 0x7fb1d8196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:34896/0 (socket says 172.21.15.125:34896) 2023-11-16T16:45:03.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1d7fff700 1 -- 172.21.15.125:0/442283645 learned_addr learned my addr 172.21.15.125:0/442283645 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-11-16T16:45:03.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1dd22a700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d8105fb0 0x7fb1d819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:03.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.394+0000 7fb1dca29700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1d8100d70 0x7fb1d8196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:03.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.395+0000 7fb1d7fff700 1 -- 172.21.15.125:0/442283645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d8105fb0 msgr2=0x7fb1d819d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:03.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.395+0000 7fb1d7fff700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d8105fb0 0x7fb1d819d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:03.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.395+0000 7fb1d7fff700 1 -- 172.21.15.125:0/442283645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1d8100d70 msgr2=0x7fb1d8196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:03.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.395+0000 7fb1d7fff700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1d8100d70 0x7fb1d8196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:03.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.395+0000 7fb1d7fff700 1 -- 172.21.15.125:0/442283645 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc0095f0 con 0x7fb1d8103690 2023-11-16T16:45:03.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.395+0000 7fb1dca29700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1d8100d70 0x7fb1d8196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:45:03.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.395+0000 7fb1d7fff700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1d8103690 0x7fb1d8196570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400c920 tx=0x7fb1c400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:03.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.395+0000 7fb1d5ffb700 1 -- 172.21.15.125:0/442283645 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4007b00 con 0x7fb1d8103690 2023-11-16T16:45:03.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.395+0000 7fb1d5ffb700 1 -- 172.21.15.125:0/442283645 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1c400ce70 con 0x7fb1d8103690 2023-11-16T16:45:03.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.395+0000 7fb1dec8d700 1 -- 172.21.15.125:0/442283645 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d810bcd0 con 0x7fb1d8103690 2023-11-16T16:45:03.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.395+0000 7fb1d5ffb700 1 -- 172.21.15.125:0/442283645 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40054e0 con 0x7fb1d8103690 2023-11-16T16:45:03.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.395+0000 7fb1dec8d700 1 -- 172.21.15.125:0/442283645 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d810bb40 con 0x7fb1d8103690 2023-11-16T16:45:03.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: Detected new or changed devices on smithi093 2023-11-16T16:45:03.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:03.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:03.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:03.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:03.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:03.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:03.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi093 to 4317M 2023-11-16T16:45:03.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:03.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:03.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:03.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:45:03.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:03.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:45:03.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:45:03.402 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:03 smithi093 ceph-mon[97863]: osdmap e68: 10 total, 10 up, 10 in 2023-11-16T16:45:03.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.396+0000 7fb1dec8d700 1 -- 172.21.15.125:0/442283645 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d819e620 con 0x7fb1d8103690 2023-11-16T16:45:03.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.397+0000 7fb1d5ffb700 1 -- 172.21.15.125:0/442283645 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb1c4004750 con 0x7fb1d8103690 2023-11-16T16:45:03.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.398+0000 7fb1d5ffb700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb1c80a11d0 0x7fb1c80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:03.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.398+0000 7fb1d5ffb700 1 -- 172.21.15.125:0/442283645 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fb1c40cbbe0 con 0x7fb1d8103690 2023-11-16T16:45:03.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.398+0000 7fb1dca29700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb1c80a11d0 0x7fb1c80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:03.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.398+0000 7fb1dca29700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb1c80a11d0 0x7fb1c80a3680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00b5c0 tx=0x7fb1cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:03.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.402+0000 7fb1d5ffb700 1 -- 172.21.15.125:0/442283645 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb1c4095b30 con 0x7fb1d8103690 2023-11-16T16:45:03.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: Detected new or changed devices on smithi093 2023-11-16T16:45:03.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:03.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:03.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:03.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:03.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:03.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:03.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi093 to 4317M 2023-11-16T16:45:03.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:03.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:03.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:03.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:45:03.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:03.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:45:03.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:45:03.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:03 smithi159 ceph-mon[131528]: osdmap e68: 10 total, 10 up, 10 in 2023-11-16T16:45:03.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:03.594+0000 7fb1dec8d700 1 -- 172.21.15.125:0/442283645 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1d819ebd0 con 0x7fb1c80a11d0 2023-11-16T16:45:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:04 smithi125 ceph-mon[131076]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:45:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:45:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:04 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:04.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:04 smithi093 ceph-mon[97863]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:04.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:45:04.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:45:04.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:04 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:04.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:04 smithi159 ceph-mon[131528]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:04.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:45:04.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:45:04.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:04 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:05.309 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:05 smithi125 ceph-mon[131076]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:45:05.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:05 smithi093 ceph-mon[97863]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:45:05.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:05 smithi159 ceph-mon[131528]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:45:06.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:06 smithi125 ceph-mon[131076]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:06.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:06 smithi093 ceph-mon[97863]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:06.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:06 smithi159 ceph-mon[131528]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:07 smithi125 ceph-mon[131076]: from='client.? 172.21.15.125:0/1993423346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7c0597-8595-4c28-8b1b-12e74fc554eb"}]: dispatch 2023-11-16T16:45:07.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:07 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7c0597-8595-4c28-8b1b-12e74fc554eb"}]: dispatch 2023-11-16T16:45:07.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:07 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d7c0597-8595-4c28-8b1b-12e74fc554eb"}]': finished 2023-11-16T16:45:07.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:07 smithi125 ceph-mon[131076]: osdmap e69: 11 total, 10 up, 11 in 2023-11-16T16:45:07.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:07 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:07.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:07 smithi125 ceph-mon[131076]: from='client.? 172.21.15.125:0/3420921735' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:45:07.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:07 smithi093 ceph-mon[97863]: from='client.? 172.21.15.125:0/1993423346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7c0597-8595-4c28-8b1b-12e74fc554eb"}]: dispatch 2023-11-16T16:45:07.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:07 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7c0597-8595-4c28-8b1b-12e74fc554eb"}]: dispatch 2023-11-16T16:45:07.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:07 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d7c0597-8595-4c28-8b1b-12e74fc554eb"}]': finished 2023-11-16T16:45:07.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:07 smithi093 ceph-mon[97863]: osdmap e69: 11 total, 10 up, 11 in 2023-11-16T16:45:07.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:07 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:07.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:07 smithi093 ceph-mon[97863]: from='client.? 172.21.15.125:0/3420921735' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:45:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:07 smithi159 ceph-mon[131528]: from='client.? 172.21.15.125:0/1993423346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7c0597-8595-4c28-8b1b-12e74fc554eb"}]: dispatch 2023-11-16T16:45:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:07 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7c0597-8595-4c28-8b1b-12e74fc554eb"}]: dispatch 2023-11-16T16:45:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:07 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d7c0597-8595-4c28-8b1b-12e74fc554eb"}]': finished 2023-11-16T16:45:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:07 smithi159 ceph-mon[131528]: osdmap e69: 11 total, 10 up, 11 in 2023-11-16T16:45:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:07 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:07.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:07 smithi159 ceph-mon[131528]: from='client.? 172.21.15.125:0/3420921735' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:45:08.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:08 smithi125 ceph-mon[131076]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:08.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:08 smithi093 ceph-mon[97863]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:08.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:08 smithi159 ceph-mon[131528]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:10.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:10 smithi125 ceph-mon[131076]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:10.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:10 smithi093 ceph-mon[97863]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:10.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:10 smithi159 ceph-mon[131528]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:11.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:11 smithi125 ceph-mon[131076]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:11.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:11 smithi093 ceph-mon[97863]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:11.734 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:11 smithi159 ceph-mon[131528]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:13.785 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:13 smithi125 ceph-mon[131076]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:13.785 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:13 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-16T16:45:13.786 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:13 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:13.786 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:13 smithi125 ceph-mon[131076]: Deploying daemon osd.10 on smithi125 2023-11-16T16:45:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:13 smithi093 ceph-mon[97863]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:13 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-16T16:45:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:13 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:13.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:13 smithi093 ceph-mon[97863]: Deploying daemon osd.10 on smithi125 2023-11-16T16:45:13.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:13 smithi159 ceph-mon[131528]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:13.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:13 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-16T16:45:13.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:13 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:13.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:13 smithi159 ceph-mon[131528]: Deploying daemon osd.10 on smithi125 2023-11-16T16:45:15.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:15 smithi125 ceph-mon[131076]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:15.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:15 smithi093 ceph-mon[97863]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:15.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:15 smithi159 ceph-mon[131528]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:17.801 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:17 smithi125 ceph-mon[131076]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:17.801 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:17.801 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:17.801 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:17 smithi093 ceph-mon[97863]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:17 smithi159 ceph-mon[131528]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:19.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:19 smithi125 ceph-mon[131076]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:19 smithi093 ceph-mon[97863]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:19.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:19 smithi159 ceph-mon[131528]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:20.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.408+0000 7fb1d5ffb700 1 -- 172.21.15.125:0/442283645 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fb1d819ebd0 con 0x7fb1c80a11d0 2023-11-16T16:45:20.411 INFO:teuthology.orchestra.run.smithi125.stdout:Created osd(s) 10 on host 'smithi125' 2023-11-16T16:45:20.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.410+0000 7fb1dec8d700 1 -- 172.21.15.125:0/442283645 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb1c80a11d0 msgr2=0x7fb1c80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:20.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.410+0000 7fb1dec8d700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb1c80a11d0 0x7fb1c80a3680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00b5c0 tx=0x7fb1cc005fb0 comp rx=0 tx=0).stop 2023-11-16T16:45:20.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.410+0000 7fb1dec8d700 1 -- 172.21.15.125:0/442283645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1d8103690 msgr2=0x7fb1d8196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:20.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.410+0000 7fb1dec8d700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1d8103690 0x7fb1d8196570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400c920 tx=0x7fb1c400cc30 comp rx=0 tx=0).stop 2023-11-16T16:45:20.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.410+0000 7fb1dec8d700 1 -- 172.21.15.125:0/442283645 shutdown_connections 2023-11-16T16:45:20.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.410+0000 7fb1dec8d700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb1c80a11d0 0x7fb1c80a3680 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:20.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.410+0000 7fb1dec8d700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1d8100d70 0x7fb1d8196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:20.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.410+0000 7fb1dec8d700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d8105fb0 0x7fb1d819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:20.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.410+0000 7fb1dec8d700 1 --2- 172.21.15.125:0/442283645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1d8103690 0x7fb1d8196570 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:20.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.410+0000 7fb1dec8d700 1 -- 172.21.15.125:0/442283645 >> 172.21.15.125:0/442283645 conn(0x7fb1d80fa7f0 msgr2=0x7fb1d8102230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:20.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.410+0000 7fb1dec8d700 1 -- 172.21.15.125:0/442283645 shutdown_connections 2023-11-16T16:45:20.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-11-16T16:45:20.411+0000 7fb1dec8d700 1 -- 172.21.15.125:0/442283645 wait complete. 2023-11-16T16:45:20.765 DEBUG:teuthology.orchestra.run.smithi125:osd.10> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.10.service 2023-11-16T16:45:20.767 INFO:tasks.cephadm:Deploying osd.11 on smithi159 with /dev/vg_nvme/lv_1... 2023-11-16T16:45:20.767 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-16T16:45:20.796 INFO:journalctl@ceph.osd.10.smithi125.stdout:-- Logs begin at Thu 2023-11-16 16:14:41 UTC. -- 2023-11-16T16:45:21.387 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:45:21 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:45:21.340+0000 7fd2e536a640 -1 osd.10 0 log_to_monitors true 2023-11-16T16:45:21.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:21.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:21.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:21 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:45:21.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:21 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:45:21.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:21 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:45:21.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.389 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:21.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:21.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:21.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:21 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:45:21.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:21 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:45:21.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:21 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:45:21.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:21.419 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:45:21.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:21.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:21.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:21 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:45:21.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:21 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:45:21.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:21 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:45:21.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:21.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:22.190 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:22 smithi159 ceph-mon[131528]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:22.190 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:22 smithi159 ceph-mon[131528]: from='osd.10 [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-16T16:45:22.190 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:22 smithi159 ceph-mon[131528]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-16T16:45:22.223 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:22 smithi125 ceph-mon[131076]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:22.223 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:22 smithi125 ceph-mon[131076]: from='osd.10 [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-16T16:45:22.224 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:22 smithi125 ceph-mon[131076]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-16T16:45:22.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:22 smithi093 ceph-mon[97863]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:22.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:22 smithi093 ceph-mon[97863]: from='osd.10 [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-16T16:45:22.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:22 smithi093 ceph-mon[97863]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-16T16:45:22.898 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-16T16:45:22.898 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-16T16:45:22.898 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2023-11-16T16:45:22.898 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2023-11-16T16:45:22.898 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112954 s, 928 MB/s 2023-11-16T16:45:22.898 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2023-11-16T16:45:23.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:23 smithi159 ceph-mon[131528]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-16T16:45:23.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:23 smithi159 ceph-mon[131528]: osdmap e70: 11 total, 10 up, 11 in 2023-11-16T16:45:23.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:23 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:23.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:23 smithi159 ceph-mon[131528]: from='osd.10 [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:45:23.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:23 smithi159 ceph-mon[131528]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:45:23.372 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi159:vg_nvme/lv_1 2023-11-16T16:45:23.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:23 smithi093 ceph-mon[97863]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-16T16:45:23.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:23 smithi093 ceph-mon[97863]: osdmap e70: 11 total, 10 up, 11 in 2023-11-16T16:45:23.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:23 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:23.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:23 smithi093 ceph-mon[97863]: from='osd.10 [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:45:23.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:23 smithi093 ceph-mon[97863]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:45:23.577 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:45:23 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:45:23.141+0000 7fd2da9e1700 -1 osd.10 0 waiting for initial osdmap 2023-11-16T16:45:23.578 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:45:23 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:45:23.160+0000 7fd2d4fd3700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:45:23.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:23 smithi125 ceph-mon[131076]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-16T16:45:23.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:23 smithi125 ceph-mon[131076]: osdmap e70: 11 total, 10 up, 11 in 2023-11-16T16:45:23.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:23 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:23.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:23 smithi125 ceph-mon[131076]: from='osd.10 [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:45:23.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:23 smithi125 ceph-mon[131076]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2023-11-16T16:45:24.016 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:45:24.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:24.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2023-11-16T16:45:24.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: osdmap e71: 11 total, 10 up, 11 in 2023-11-16T16:45:24.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:24.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:24.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:24.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:24.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:24.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:24.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:24.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:24.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:24.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:24.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:24.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:24 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:24.458 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:24.458 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2023-11-16T16:45:24.458 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: osdmap e71: 11 total, 10 up, 11 in 2023-11-16T16:45:24.458 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:24.458 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:24.458 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:24.458 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:24.459 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:24.459 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:24.459 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:24.459 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:24.459 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:24.459 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:24.459 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:24.459 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:24 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:45:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2023-11-16T16:45:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: osdmap e71: 11 total, 10 up, 11 in 2023-11-16T16:45:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:24.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:24.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:24.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:24.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:24.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:24.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:24 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:25.307 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.303+0000 7f6d54192700 1 -- 172.21.15.159:0/319840554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d4c100d70 msgr2=0x7f6d4c107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:25.308 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.303+0000 7f6d54192700 1 --2- 172.21.15.159:0/319840554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d4c100d70 0x7f6d4c107630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400b0a0 tx=0x7f6d4400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:45:25.308 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 -- 172.21.15.159:0/319840554 shutdown_connections 2023-11-16T16:45:25.308 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 --2- 172.21.15.159:0/319840554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d4c0ff1e0 0x7f6d4c0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:25.308 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 --2- 172.21.15.159:0/319840554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d4c100d70 0x7f6d4c107630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:25.308 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 --2- 172.21.15.159:0/319840554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d4c1003e0 0x7f6d4c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:25.308 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 -- 172.21.15.159:0/319840554 >> 172.21.15.159:0/319840554 conn(0x7f6d4c0fa7f0 msgr2=0x7f6d4c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:25.308 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 -- 172.21.15.159:0/319840554 shutdown_connections 2023-11-16T16:45:25.308 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 -- 172.21.15.159:0/319840554 wait complete. 2023-11-16T16:45:25.308 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 Processor -- start 2023-11-16T16:45:25.308 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 -- start start 2023-11-16T16:45:25.308 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d4c0ff1e0 0x7f6d4c196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:25.309 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d4c1003e0 0x7f6d4c196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:25.309 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d4c100d70 0x7f6d4c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:25.309 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c19dca0 con 0x7f6d4c0ff1e0 2023-11-16T16:45:25.309 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c19dde0 con 0x7f6d4c100d70 2023-11-16T16:45:25.309 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d54192700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c19df70 con 0x7f6d4c1003e0 2023-11-16T16:45:25.309 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d5172d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d4c1003e0 0x7f6d4c196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:25.309 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.304+0000 7f6d5172d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d4c1003e0 0x7f6d4c196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54432/0 (socket says 172.21.15.159:54432) 2023-11-16T16:45:25.309 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d5172d700 1 -- 172.21.15.159:0/644577326 learned_addr learned my addr 172.21.15.159:0/644577326 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-11-16T16:45:25.309 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d5272f700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d4c100d70 0x7f6d4c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:25.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d51f2e700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d4c0ff1e0 0x7f6d4c196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:25.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d5172d700 1 -- 172.21.15.159:0/644577326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d4c100d70 msgr2=0x7f6d4c19d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:25.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d5172d700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d4c100d70 0x7f6d4c19d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:25.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d5172d700 1 -- 172.21.15.159:0/644577326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d4c0ff1e0 msgr2=0x7f6d4c196110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:25.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d5172d700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d4c0ff1e0 0x7f6d4c196110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:25.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d5172d700 1 -- 172.21.15.159:0/644577326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d44009d00 con 0x7f6d4c1003e0 2023-11-16T16:45:25.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d5172d700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d4c1003e0 0x7f6d4c196650 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00d9b0 tx=0x7f6d3c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:25.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d42ffd700 1 -- 172.21.15.159:0/644577326 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c004220 con 0x7f6d4c1003e0 2023-11-16T16:45:25.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d42ffd700 1 -- 172.21.15.159:0/644577326 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d3c007640 con 0x7f6d4c1003e0 2023-11-16T16:45:25.310 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d54192700 1 -- 172.21.15.159:0/644577326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4c10ab30 con 0x7f6d4c1003e0 2023-11-16T16:45:25.311 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d42ffd700 1 -- 172.21.15.159:0/644577326 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c005650 con 0x7f6d4c1003e0 2023-11-16T16:45:25.311 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.305+0000 7f6d54192700 1 -- 172.21.15.159:0/644577326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c10a9a0 con 0x7f6d4c1003e0 2023-11-16T16:45:25.311 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.306+0000 7f6d54192700 1 -- 172.21.15.159:0/644577326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4c19e650 con 0x7f6d4c1003e0 2023-11-16T16:45:25.312 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.307+0000 7f6d42ffd700 1 -- 172.21.15.159:0/644577326 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6d3c004750 con 0x7f6d4c1003e0 2023-11-16T16:45:25.312 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.308+0000 7f6d42ffd700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6d380a1150 0x7f6d380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:25.312 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.308+0000 7f6d42ffd700 1 -- 172.21.15.159:0/644577326 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f6d3c0cc680 con 0x7f6d4c1003e0 2023-11-16T16:45:25.312 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.308+0000 7f6d51f2e700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6d380a1150 0x7f6d380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:25.312 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.308+0000 7f6d51f2e700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6d380a1150 0x7f6d380a3600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6d48005950 tx=0x7f6d480058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:25.312 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.311+0000 7f6d42ffd700 1 -- 172.21.15.159:0/644577326 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6d3c096400 con 0x7f6d4c1003e0 2023-11-16T16:45:25.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:25 smithi159 ceph-mon[131528]: purged_snaps scrub starts 2023-11-16T16:45:25.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:25 smithi159 ceph-mon[131528]: purged_snaps scrub ok 2023-11-16T16:45:25.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:25 smithi159 ceph-mon[131528]: Detected new or changed devices on smithi125 2023-11-16T16:45:25.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:25 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi125 to 4292M 2023-11-16T16:45:25.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:25 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:45:25.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:25 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:45:25.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:25 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:45:25.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:25 smithi159 ceph-mon[131528]: osd.10 [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100] boot 2023-11-16T16:45:25.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:25 smithi159 ceph-mon[131528]: osdmap e72: 11 total, 11 up, 11 in 2023-11-16T16:45:25.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:25 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:25.506 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:25.505+0000 7f6d54192700 1 -- 172.21.15.159:0/644577326 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d4c068ce0 con 0x7f6d380a1150 2023-11-16T16:45:25.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:25 smithi125 ceph-mon[131076]: purged_snaps scrub starts 2023-11-16T16:45:25.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:25 smithi125 ceph-mon[131076]: purged_snaps scrub ok 2023-11-16T16:45:25.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:25 smithi125 ceph-mon[131076]: Detected new or changed devices on smithi125 2023-11-16T16:45:25.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:25 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi125 to 4292M 2023-11-16T16:45:25.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:25 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:45:25.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:25 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:45:25.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:25 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:45:25.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:25 smithi125 ceph-mon[131076]: osd.10 [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100] boot 2023-11-16T16:45:25.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:25 smithi125 ceph-mon[131076]: osdmap e72: 11 total, 11 up, 11 in 2023-11-16T16:45:25.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:25 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:25.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:25 smithi093 ceph-mon[97863]: purged_snaps scrub starts 2023-11-16T16:45:25.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:25 smithi093 ceph-mon[97863]: purged_snaps scrub ok 2023-11-16T16:45:25.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:25 smithi093 ceph-mon[97863]: Detected new or changed devices on smithi125 2023-11-16T16:45:25.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:25 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi125 to 4292M 2023-11-16T16:45:25.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:25 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:45:25.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:25 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:45:25.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:25 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:45:25.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:25 smithi093 ceph-mon[97863]: osd.10 [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100] boot 2023-11-16T16:45:25.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:25 smithi093 ceph-mon[97863]: osdmap e72: 11 total, 11 up, 11 in 2023-11-16T16:45:25.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:25 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:45:26.423 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:26 smithi159 ceph-mon[131528]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:26.423 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:26 smithi159 ceph-mon[131528]: osdmap e73: 11 total, 11 up, 11 in 2023-11-16T16:45:26.423 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:26 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:45:26.423 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:26 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:45:26.423 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:26 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:26.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:26 smithi125 ceph-mon[131076]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:26.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:26 smithi125 ceph-mon[131076]: osdmap e73: 11 total, 11 up, 11 in 2023-11-16T16:45:26.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:26 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:45:26.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:26 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:45:26.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:26 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:26.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:26 smithi093 ceph-mon[97863]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:26.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:26 smithi093 ceph-mon[97863]: osdmap e73: 11 total, 11 up, 11 in 2023-11-16T16:45:26.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:26 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:45:26.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:26 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:45:26.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:26 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:27.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:27 smithi159 ceph-mon[131528]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:45:27.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:27 smithi159 ceph-mon[131528]: osdmap e74: 11 total, 11 up, 11 in 2023-11-16T16:45:27.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:27 smithi125 ceph-mon[131076]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:45:27.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:27 smithi125 ceph-mon[131076]: osdmap e74: 11 total, 11 up, 11 in 2023-11-16T16:45:27.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:27 smithi093 ceph-mon[97863]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:45:27.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:27 smithi093 ceph-mon[97863]: osdmap e74: 11 total, 11 up, 11 in 2023-11-16T16:45:28.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:28 smithi159 ceph-mon[131528]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:28.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:28 smithi159 ceph-mon[131528]: osdmap e75: 11 total, 11 up, 11 in 2023-11-16T16:45:28.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:28 smithi159 ceph-mon[131528]: from='client.? 172.21.15.159:0/1121880587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77e5e471-2cae-44cc-82e3-13888533f31d"}]: dispatch 2023-11-16T16:45:28.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:28 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77e5e471-2cae-44cc-82e3-13888533f31d"}]: dispatch 2023-11-16T16:45:28.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:28 smithi159 ceph-mon[131528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77e5e471-2cae-44cc-82e3-13888533f31d"}]': finished 2023-11-16T16:45:28.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:28 smithi159 ceph-mon[131528]: osdmap e76: 12 total, 11 up, 12 in 2023-11-16T16:45:28.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:28.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:28 smithi125 ceph-mon[131076]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:28.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:28 smithi125 ceph-mon[131076]: osdmap e75: 11 total, 11 up, 11 in 2023-11-16T16:45:28.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:28 smithi125 ceph-mon[131076]: from='client.? 172.21.15.159:0/1121880587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77e5e471-2cae-44cc-82e3-13888533f31d"}]: dispatch 2023-11-16T16:45:28.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:28 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77e5e471-2cae-44cc-82e3-13888533f31d"}]: dispatch 2023-11-16T16:45:28.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:28 smithi125 ceph-mon[131076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77e5e471-2cae-44cc-82e3-13888533f31d"}]': finished 2023-11-16T16:45:28.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:28 smithi125 ceph-mon[131076]: osdmap e76: 12 total, 11 up, 12 in 2023-11-16T16:45:28.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:28.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:28 smithi093 ceph-mon[97863]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:28.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:28 smithi093 ceph-mon[97863]: osdmap e75: 11 total, 11 up, 11 in 2023-11-16T16:45:28.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:28 smithi093 ceph-mon[97863]: from='client.? 172.21.15.159:0/1121880587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77e5e471-2cae-44cc-82e3-13888533f31d"}]: dispatch 2023-11-16T16:45:28.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:28 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77e5e471-2cae-44cc-82e3-13888533f31d"}]: dispatch 2023-11-16T16:45:28.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:28 smithi093 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77e5e471-2cae-44cc-82e3-13888533f31d"}]': finished 2023-11-16T16:45:28.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:28 smithi093 ceph-mon[97863]: osdmap e76: 12 total, 11 up, 12 in 2023-11-16T16:45:28.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:29.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:29 smithi159 ceph-mon[131528]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:29.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:29 smithi159 ceph-mon[131528]: from='client.? 172.21.15.159:0/2927724267' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:45:29.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:29 smithi125 ceph-mon[131076]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:29.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:29 smithi125 ceph-mon[131076]: from='client.? 172.21.15.159:0/2927724267' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:45:29.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:29 smithi093 ceph-mon[97863]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:29.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:29 smithi093 ceph-mon[97863]: from='client.? 172.21.15.159:0/2927724267' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:45:31.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:31 smithi125 ceph-mon[131076]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2023-11-16T16:45:31.637 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:31 smithi159 ceph-mon[131528]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2023-11-16T16:45:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:31 smithi093 ceph-mon[97863]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2023-11-16T16:45:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:33 smithi125 ceph-mon[131076]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-11-16T16:45:33.834 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:33 smithi159 ceph-mon[131528]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-11-16T16:45:33.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:33 smithi093 ceph-mon[97863]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-11-16T16:45:35.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:35 smithi159 ceph-mon[131528]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-16T16:45:35.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-16T16:45:35.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:35 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:35.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:35 smithi159 ceph-mon[131528]: Deploying daemon osd.11 on smithi159 2023-11-16T16:45:35.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:35 smithi125 ceph-mon[131076]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-16T16:45:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:35 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-16T16:45:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:35 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:35 smithi125 ceph-mon[131076]: Deploying daemon osd.11 on smithi159 2023-11-16T16:45:35.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:35 smithi093 ceph-mon[97863]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-16T16:45:35.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:35 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-16T16:45:35.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:35 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:35.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:35 smithi093 ceph-mon[97863]: Deploying daemon osd.11 on smithi159 2023-11-16T16:45:37.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:37 smithi159 ceph-mon[131528]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-11-16T16:45:37.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:37 smithi125 ceph-mon[131076]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-11-16T16:45:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:37 smithi093 ceph-mon[97863]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-11-16T16:45:39.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:39 smithi159 ceph-mon[131528]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-11-16T16:45:39.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:39.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:39.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:39 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:39.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:39 smithi125 ceph-mon[131076]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-11-16T16:45:39.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:39.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:39.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:39 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:39 smithi093 ceph-mon[97863]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-11-16T16:45:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:39 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:41.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:41 smithi159 ceph-mon[131528]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-16T16:45:41.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:41 smithi125 ceph-mon[131076]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-16T16:45:41.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:41 smithi093 ceph-mon[97863]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-16T16:45:42.271 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 11 on host 'smithi159' 2023-11-16T16:45:42.272 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.269+0000 7f6d42ffd700 1 -- 172.21.15.159:0/644577326 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f6d4c068ce0 con 0x7f6d380a1150 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.271+0000 7f6d54192700 1 -- 172.21.15.159:0/644577326 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6d380a1150 msgr2=0x7f6d380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.271+0000 7f6d54192700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6d380a1150 0x7f6d380a3600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6d48005950 tx=0x7f6d480058e0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.271+0000 7f6d54192700 1 -- 172.21.15.159:0/644577326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d4c1003e0 msgr2=0x7f6d4c196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.271+0000 7f6d54192700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d4c1003e0 0x7f6d4c196650 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00d9b0 tx=0x7f6d3c00dcc0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.271+0000 7f6d54192700 1 -- 172.21.15.159:0/644577326 shutdown_connections 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.271+0000 7f6d54192700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6d380a1150 0x7f6d380a3600 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.271+0000 7f6d54192700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d4c0ff1e0 0x7f6d4c196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.271+0000 7f6d54192700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d4c100d70 0x7f6d4c19d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.271+0000 7f6d54192700 1 --2- 172.21.15.159:0/644577326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d4c1003e0 0x7f6d4c196650 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.271+0000 7f6d54192700 1 -- 172.21.15.159:0/644577326 >> 172.21.15.159:0/644577326 conn(0x7f6d4c0fa7f0 msgr2=0x7f6d4c0685d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.271+0000 7f6d54192700 1 -- 172.21.15.159:0/644577326 shutdown_connections 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:42.271+0000 7f6d54192700 1 -- 172.21.15.159:0/644577326 wait complete. 2023-11-16T16:45:42.694 DEBUG:teuthology.orchestra.run.smithi159:osd.11> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.11.service 2023-11-16T16:45:42.696 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-11-16T16:45:42.697 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd stat -f json 2023-11-16T16:45:42.726 INFO:journalctl@ceph.osd.11.smithi159.stdout:-- Logs begin at Thu 2023-11-16 16:15:38 UTC. -- 2023-11-16T16:45:43.511 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:45:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:45:43.317+0000 7f79b28b6640 -1 osd.11 0 log_to_monitors true 2023-11-16T16:45:43.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:43 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:43.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:43 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:43.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:43 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:43.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:43 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:43.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:43 smithi159 ceph-mon[131528]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:43.564 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:45:43.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:43.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:43.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:43.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:43 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:43.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:43 smithi125 ceph-mon[131076]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:43.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:43 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:43.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:43 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:43.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:43 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:43.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:43 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:43.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:43 smithi093 ceph-mon[97863]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:44.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:44 smithi093 ceph-mon[97863]: from='osd.11 [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-16T16:45:44.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:44 smithi093 ceph-mon[97863]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-16T16:45:44.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:44 smithi125 ceph-mon[131076]: from='osd.11 [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-16T16:45:44.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:44 smithi125 ceph-mon[131076]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-16T16:45:44.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:44 smithi159 ceph-mon[131528]: from='osd.11 [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-16T16:45:44.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:44 smithi159 ceph-mon[131528]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-16T16:45:44.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.942+0000 7f41f4570700 1 -- 172.21.15.93:0/2251016195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41ec1014f0 msgr2=0x7f41ec101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:44.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.942+0000 7f41f4570700 1 --2- 172.21.15.93:0/2251016195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41ec1014f0 0x7f41ec101940 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009990 tx=0x7f41dc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.942+0000 7f41f4570700 1 -- 172.21.15.93:0/2251016195 shutdown_connections 2023-11-16T16:45:44.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.942+0000 7f41f4570700 1 --2- 172.21.15.93:0/2251016195 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41ec101e80 0x7f41ec108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.942+0000 7f41f4570700 1 --2- 172.21.15.93:0/2251016195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41ec1002f0 0x7f41ec100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.942+0000 7f41f4570700 1 --2- 172.21.15.93:0/2251016195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41ec1014f0 0x7f41ec101940 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.942+0000 7f41f4570700 1 -- 172.21.15.93:0/2251016195 >> 172.21.15.93:0/2251016195 conn(0x7f41ec0fb900 msgr2=0x7f41ec0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:44.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.942+0000 7f41f4570700 1 -- 172.21.15.93:0/2251016195 shutdown_connections 2023-11-16T16:45:44.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.942+0000 7f41f4570700 1 -- 172.21.15.93:0/2251016195 wait complete. 2023-11-16T16:45:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.943+0000 7f41f4570700 1 Processor -- start 2023-11-16T16:45:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.943+0000 7f41f4570700 1 -- start start 2023-11-16T16:45:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.943+0000 7f41f4570700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41ec1002f0 0x7f41ec196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.943+0000 7f41f4570700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41ec1014f0 0x7f41ec1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.943+0000 7f41f4570700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41ec101e80 0x7f41ec19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.943+0000 7f41f4570700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec19dc20 con 0x7f41ec101e80 2023-11-16T16:45:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.943+0000 7f41f4570700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec19dd60 con 0x7f41ec1014f0 2023-11-16T16:45:44.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.943+0000 7f41f4570700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec19def0 con 0x7f41ec1002f0 2023-11-16T16:45:44.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.943+0000 7f41f230c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41ec1002f0 0x7f41ec196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:44.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41f2b0d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41ec101e80 0x7f41ec19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:44.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41f1b0b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41ec1014f0 0x7f41ec1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:44.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41f230c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41ec1002f0 0x7f41ec196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:52204/0 (socket says 172.21.15.93:52204) 2023-11-16T16:45:44.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41f230c700 1 -- 172.21.15.93:0/2051132312 learned_addr learned my addr 172.21.15.93:0/2051132312 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:45:44.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41f230c700 1 -- 172.21.15.93:0/2051132312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41ec1014f0 msgr2=0x7f41ec1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:44.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41f230c700 1 --2- 172.21.15.93:0/2051132312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41ec1014f0 0x7f41ec1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41f230c700 1 -- 172.21.15.93:0/2051132312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41ec101e80 msgr2=0x7f41ec19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:44.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41f230c700 1 --2- 172.21.15.93:0/2051132312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41ec101e80 0x7f41ec19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41f230c700 1 -- 172.21.15.93:0/2051132312 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e4009c40 con 0x7f41ec1002f0 2023-11-16T16:45:44.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41f230c700 1 --2- 172.21.15.93:0/2051132312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41ec1002f0 0x7f41ec196090 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f41e800e9a0 tx=0x7f41e800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:44.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41e37fe700 1 -- 172.21.15.93:0/2051132312 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8009790 con 0x7f41ec1002f0 2023-11-16T16:45:44.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41e37fe700 1 -- 172.21.15.93:0/2051132312 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41e800c7c0 con 0x7f41ec1002f0 2023-11-16T16:45:44.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41f4570700 1 -- 172.21.15.93:0/2051132312 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41dc0095f0 con 0x7f41ec1002f0 2023-11-16T16:45:44.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.944+0000 7f41e37fe700 1 -- 172.21.15.93:0/2051132312 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8003c20 con 0x7f41ec1002f0 2023-11-16T16:45:44.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.945+0000 7f41f4570700 1 -- 172.21.15.93:0/2051132312 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec10bc90 con 0x7f41ec1002f0 2023-11-16T16:45:44.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.945+0000 7f41f4570700 1 -- 172.21.15.93:0/2051132312 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ec1078e0 con 0x7f41ec1002f0 2023-11-16T16:45:44.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.947+0000 7f41e37fe700 1 -- 172.21.15.93:0/2051132312 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f41e800cc30 con 0x7f41ec1002f0 2023-11-16T16:45:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.949+0000 7f41e37fe700 1 --2- 172.21.15.93:0/2051132312 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f41d80a1150 0x7f41d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.949+0000 7f41e37fe700 1 -- 172.21.15.93:0/2051132312 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f41e8014070 con 0x7f41ec1002f0 2023-11-16T16:45:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.949+0000 7f41f1b0b700 1 --2- 172.21.15.93:0/2051132312 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f41d80a1150 0x7f41d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:44.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.949+0000 7f41f1b0b700 1 --2- 172.21.15.93:0/2051132312 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f41d80a1150 0x7f41d80a3600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00b2e0 tx=0x7f41dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:44.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:44.953+0000 7f41e37fe700 1 -- 172.21.15.93:0/2051132312 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f41e80961d0 con 0x7f41ec1002f0 2023-11-16T16:45:45.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.150+0000 7f41f4570700 1 -- 172.21.15.93:0/2051132312 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f41ec02cfe0 con 0x7f41ec1002f0 2023-11-16T16:45:45.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.153+0000 7f41e37fe700 1 -- 172.21.15.93:0/2051132312 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f41e80997f0 con 0x7f41ec1002f0 2023-11-16T16:45:45.155 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:45:45.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.156+0000 7f41f4570700 1 -- 172.21.15.93:0/2051132312 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f41d80a1150 msgr2=0x7f41d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:45.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.156+0000 7f41f4570700 1 --2- 172.21.15.93:0/2051132312 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f41d80a1150 0x7f41d80a3600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00b2e0 tx=0x7f41dc005fb0 comp rx=0 tx=0).stop 2023-11-16T16:45:45.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.156+0000 7f41f4570700 1 -- 172.21.15.93:0/2051132312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41ec1002f0 msgr2=0x7f41ec196090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:45.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.156+0000 7f41f4570700 1 --2- 172.21.15.93:0/2051132312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41ec1002f0 0x7f41ec196090 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f41e800e9a0 tx=0x7f41e800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:45:45.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.156+0000 7f41f4570700 1 -- 172.21.15.93:0/2051132312 shutdown_connections 2023-11-16T16:45:45.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.156+0000 7f41f4570700 1 --2- 172.21.15.93:0/2051132312 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f41d80a1150 0x7f41d80a3600 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:45.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.156+0000 7f41f4570700 1 --2- 172.21.15.93:0/2051132312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41ec101e80 0x7f41ec19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:45.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.156+0000 7f41f4570700 1 --2- 172.21.15.93:0/2051132312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41ec1002f0 0x7f41ec196090 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:45.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.156+0000 7f41f4570700 1 --2- 172.21.15.93:0/2051132312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41ec1014f0 0x7f41ec1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:45.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.156+0000 7f41f4570700 1 -- 172.21.15.93:0/2051132312 >> 172.21.15.93:0/2051132312 conn(0x7f41ec0fb900 msgr2=0x7f41ec104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:45.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.156+0000 7f41f4570700 1 -- 172.21.15.93:0/2051132312 shutdown_connections 2023-11-16T16:45:45.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:45.156+0000 7f41f4570700 1 -- 172.21.15.93:0/2051132312 wait complete. 2023-11-16T16:45:45.528 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-16T16:45:45.528 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: osdmap e77: 12 total, 11 up, 12 in 2023-11-16T16:45:45.528 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:45.528 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='osd.11 [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:45:45.529 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:45:45.529 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:45.529 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: Detected new or changed devices on smithi159 2023-11-16T16:45:45.529 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:45.529 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2051132312' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-16T16:45:45.529 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:45.529 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:45.529 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:45.529 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:45.529 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:45.529 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi159 to 4292M 2023-11-16T16:45:45.530 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:45.530 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:45.530 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:45.530 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:45.539 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1700153124,"num_in_osds":12,"osd_in_since":1700153127,"num_remapped_pgs":0} 2023-11-16T16:45:45.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-16T16:45:45.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: osdmap e77: 12 total, 11 up, 12 in 2023-11-16T16:45:45.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:45.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='osd.11 [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:45:45.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:45:45.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:45.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: Detected new or changed devices on smithi159 2023-11-16T16:45:45.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:45.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2051132312' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-16T16:45:45.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:45.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:45.580 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:45.580 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:45.580 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:45.580 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi159 to 4292M 2023-11-16T16:45:45.580 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:45.580 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:45.581 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:45.581 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:45.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-16T16:45:45.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: osdmap e77: 12 total, 11 up, 12 in 2023-11-16T16:45:45.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:45.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='osd.11 [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:45:45.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-11-16T16:45:45.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:45.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: Detected new or changed devices on smithi159 2023-11-16T16:45:45.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:45.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2051132312' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-16T16:45:45.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:45.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:45.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:45.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:45.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:45:45.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi159 to 4292M 2023-11-16T16:45:45.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:45.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:45.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:45.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:45.740 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:45:45.260+0000 7f79a7f2d700 -1 osd.11 0 waiting for initial osdmap 2023-11-16T16:45:45.740 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:45:45 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:45:45.274+0000 7f799fd1a700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:45:46.540 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd stat -f json 2023-11-16T16:45:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:45:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:45:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:45:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-11-16T16:45:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: osdmap e78: 12 total, 11 up, 12 in 2023-11-16T16:45:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:45:46.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:46.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:45:46.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:46 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:45:46.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:45:46.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:45:46.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:45:46.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-11-16T16:45:46.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: osdmap e78: 12 total, 11 up, 12 in 2023-11-16T16:45:46.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:46.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:46.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:46.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:46.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:46.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:45:46.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:46.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:45:46.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:46 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:45:46.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:45:46.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:45:46.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:45:46.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-11-16T16:45:46.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: osdmap e78: 12 total, 11 up, 12 in 2023-11-16T16:45:46.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:46.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:46.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:46.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:46.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:46.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:45:46.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:46.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:45:46.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:46 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:45:47.393 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:45:47.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:47 smithi125 ceph-mon[131076]: purged_snaps scrub starts 2023-11-16T16:45:47.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:47 smithi125 ceph-mon[131076]: purged_snaps scrub ok 2023-11-16T16:45:47.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:47 smithi125 ceph-mon[131076]: osd.11 [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422] boot 2023-11-16T16:45:47.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:47 smithi125 ceph-mon[131076]: osdmap e79: 12 total, 12 up, 12 in 2023-11-16T16:45:47.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:47 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:47.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:47 smithi125 ceph-mon[131076]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:47.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:47 smithi093 ceph-mon[97863]: purged_snaps scrub starts 2023-11-16T16:45:47.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:47 smithi093 ceph-mon[97863]: purged_snaps scrub ok 2023-11-16T16:45:47.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:47 smithi093 ceph-mon[97863]: osd.11 [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422] boot 2023-11-16T16:45:47.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:47 smithi093 ceph-mon[97863]: osdmap e79: 12 total, 12 up, 12 in 2023-11-16T16:45:47.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:47 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:47.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:47 smithi093 ceph-mon[97863]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:47.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:47 smithi159 ceph-mon[131528]: purged_snaps scrub starts 2023-11-16T16:45:47.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:47 smithi159 ceph-mon[131528]: purged_snaps scrub ok 2023-11-16T16:45:47.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:47 smithi159 ceph-mon[131528]: osd.11 [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422] boot 2023-11-16T16:45:47.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:47 smithi159 ceph-mon[131528]: osdmap e79: 12 total, 12 up, 12 in 2023-11-16T16:45:47.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:47 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:45:47.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:47 smithi159 ceph-mon[131528]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:45:48.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:48 smithi125 ceph-mon[131076]: osdmap e80: 12 total, 12 up, 12 in 2023-11-16T16:45:48.644 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:48 smithi093 ceph-mon[97863]: osdmap e80: 12 total, 12 up, 12 in 2023-11-16T16:45:48.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:48 smithi159 ceph-mon[131528]: osdmap e80: 12 total, 12 up, 12 in 2023-11-16T16:45:48.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.884+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/3184087391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac1002f0 msgr2=0x7fbcac100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:48.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.884+0000 7fbcb1b7f700 1 --2- 172.21.15.93:0/3184087391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac1002f0 0x7fbcac100700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009990 tx=0x7fbc9c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:45:48.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.884+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/3184087391 shutdown_connections 2023-11-16T16:45:48.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.884+0000 7fbcb1b7f700 1 --2- 172.21.15.93:0/3184087391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac1002f0 0x7fbcac100700 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:48.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.884+0000 7fbcb1b7f700 1 --2- 172.21.15.93:0/3184087391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac101e80 0x7fbcac108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:48.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.884+0000 7fbcb1b7f700 1 --2- 172.21.15.93:0/3184087391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbcac1014f0 0x7fbcac101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:48.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.884+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/3184087391 >> 172.21.15.93:0/3184087391 conn(0x7fbcac0fb900 msgr2=0x7fbcac0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:48.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.884+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/3184087391 shutdown_connections 2023-11-16T16:45:48.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.884+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/3184087391 wait complete. 2023-11-16T16:45:48.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.885+0000 7fbcb1b7f700 1 Processor -- start 2023-11-16T16:45:48.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.885+0000 7fbcb1b7f700 1 -- start start 2023-11-16T16:45:48.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcb1b7f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac1002f0 0x7fbcac196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:48.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcb1b7f700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbcac1014f0 0x7fbcac1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:48.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcb1b7f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac101e80 0x7fbcac19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:48.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcb1b7f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac19dc20 con 0x7fbcac101e80 2023-11-16T16:45:48.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcb1b7f700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac19dd60 con 0x7fbcac1014f0 2023-11-16T16:45:48.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcb1b7f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac19def0 con 0x7fbcac1002f0 2023-11-16T16:45:48.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac101e80 0x7fbcac19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:48.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcab7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac1002f0 0x7fbcac196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:48.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac101e80 0x7fbcac19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49844/0 (socket says 172.21.15.93:49844) 2023-11-16T16:45:48.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcab7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac1002f0 0x7fbcac196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:45844/0 (socket says 172.21.15.93:45844) 2023-11-16T16:45:48.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcabfff700 1 -- 172.21.15.93:0/1518017067 learned_addr learned my addr 172.21.15.93:0/1518017067 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:45:48.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcaaffd700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbcac1014f0 0x7fbcac1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:48.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcabfff700 1 -- 172.21.15.93:0/1518017067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac1002f0 msgr2=0x7fbcac196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:48.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.886+0000 7fbcabfff700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac1002f0 0x7fbcac196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:48.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.887+0000 7fbcabfff700 1 -- 172.21.15.93:0/1518017067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbcac1014f0 msgr2=0x7fbcac1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:48.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.887+0000 7fbcabfff700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbcac1014f0 0x7fbcac1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:48.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.887+0000 7fbcabfff700 1 -- 172.21.15.93:0/1518017067 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c0095f0 con 0x7fbcac101e80 2023-11-16T16:45:48.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.887+0000 7fbcabfff700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac101e80 0x7fbcac19d650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbca4007f10 tx=0x7fbca400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:48.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.887+0000 7fbca8ff9700 1 -- 172.21.15.93:0/1518017067 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4017070 con 0x7fbcac101e80 2023-11-16T16:45:48.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.887+0000 7fbca8ff9700 1 -- 172.21.15.93:0/1518017067 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbca400f040 con 0x7fbcac101e80 2023-11-16T16:45:48.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.887+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/1518017067 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcac10bcb0 con 0x7fbcac101e80 2023-11-16T16:45:48.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.887+0000 7fbca8ff9700 1 -- 172.21.15.93:0/1518017067 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4005610 con 0x7fbcac101e80 2023-11-16T16:45:48.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.887+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/1518017067 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac10bb20 con 0x7fbcac101e80 2023-11-16T16:45:48.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.888+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/1518017067 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcac19e580 con 0x7fbcac101e80 2023-11-16T16:45:48.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.890+0000 7fbca8ff9700 1 -- 172.21.15.93:0/1518017067 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbca4004ad0 con 0x7fbcac101e80 2023-11-16T16:45:48.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.891+0000 7fbca8ff9700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbc940a1150 0x7fbc940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:48.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.892+0000 7fbca8ff9700 1 -- 172.21.15.93:0/1518017067 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbca40cbbf0 con 0x7fbcac101e80 2023-11-16T16:45:48.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.892+0000 7fbcab7fe700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbc940a1150 0x7fbc940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:48.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.892+0000 7fbcab7fe700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbc940a1150 0x7fbc940a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00b5c0 tx=0x7fbc9c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:48.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:48.895+0000 7fbca8ff9700 1 -- 172.21.15.93:0/1518017067 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbca4095740 con 0x7fbcac101e80 2023-11-16T16:45:49.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.094+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/1518017067 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fbcac107250 con 0x7fbcac101e80 2023-11-16T16:45:49.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.095+0000 7fbca8ff9700 1 -- 172.21.15.93:0/1518017067 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7fbca4098d60 con 0x7fbcac101e80 2023-11-16T16:45:49.097 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:45:49.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.097+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/1518017067 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbc940a1150 msgr2=0x7fbc940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:49.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.097+0000 7fbcb1b7f700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbc940a1150 0x7fbc940a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00b5c0 tx=0x7fbc9c005fd0 comp rx=0 tx=0).stop 2023-11-16T16:45:49.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.097+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/1518017067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac101e80 msgr2=0x7fbcac19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:49.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.097+0000 7fbcb1b7f700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac101e80 0x7fbcac19d650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbca4007f10 tx=0x7fbca400d3b0 comp rx=0 tx=0).stop 2023-11-16T16:45:49.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.097+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/1518017067 shutdown_connections 2023-11-16T16:45:49.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.097+0000 7fbcb1b7f700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbc940a1150 0x7fbc940a3600 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:49.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.097+0000 7fbcb1b7f700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac101e80 0x7fbcac19d650 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:49.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.097+0000 7fbcb1b7f700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac1002f0 0x7fbcac196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:49.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.097+0000 7fbcb1b7f700 1 --2- 172.21.15.93:0/1518017067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbcac1014f0 0x7fbcac1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:49.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.097+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/1518017067 >> 172.21.15.93:0/1518017067 conn(0x7fbcac0fb900 msgr2=0x7fbcac0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:49.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.097+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/1518017067 shutdown_connections 2023-11-16T16:45:49.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:49.097+0000 7fbcb1b7f700 1 -- 172.21.15.93:0/1518017067 wait complete. 2023-11-16T16:45:49.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:49 smithi093 ceph-mon[97863]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:49.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:49 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1518017067' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-16T16:45:49.570 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1700153146,"num_in_osds":12,"osd_in_since":1700153127,"num_remapped_pgs":0} 2023-11-16T16:45:49.571 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd dump --format=json 2023-11-16T16:45:49.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:49 smithi125 ceph-mon[131076]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:49.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:49 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1518017067' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-16T16:45:49.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:49 smithi159 ceph-mon[131528]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:49.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:49 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1518017067' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-16T16:45:50.416 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:45:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:51 smithi125 ceph-mon[131076]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:51.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:51 smithi093 ceph-mon[97863]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:51 smithi159 ceph-mon[131528]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:51.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.909+0000 7fec7dfae700 1 -- 172.21.15.93:0/2645083112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec781055e0 msgr2=0x7fec7810d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:51.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.909+0000 7fec7dfae700 1 --2- 172.21.15.93:0/2645083112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec781055e0 0x7fec7810d160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fec7000b0d0 tx=0x7fec7000b3e0 comp rx=0 tx=0).stop 2023-11-16T16:45:51.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.910+0000 7fec7dfae700 1 -- 172.21.15.93:0/2645083112 shutdown_connections 2023-11-16T16:45:51.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.910+0000 7fec7dfae700 1 --2- 172.21.15.93:0/2645083112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec781055e0 0x7fec7810d160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:51.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.910+0000 7fec7dfae700 1 --2- 172.21.15.93:0/2645083112 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec78100540 0x7fec78100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:51.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.910+0000 7fec7dfae700 1 --2- 172.21.15.93:0/2645083112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec78100ee0 0x7fec78104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:51.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.910+0000 7fec7dfae700 1 -- 172.21.15.93:0/2645083112 >> 172.21.15.93:0/2645083112 conn(0x7fec780fbdc0 msgr2=0x7fec780fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:51.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.910+0000 7fec7dfae700 1 -- 172.21.15.93:0/2645083112 shutdown_connections 2023-11-16T16:45:51.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.910+0000 7fec7dfae700 1 -- 172.21.15.93:0/2645083112 wait complete. 2023-11-16T16:45:51.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.911+0000 7fec7dfae700 1 Processor -- start 2023-11-16T16:45:51.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.911+0000 7fec7dfae700 1 -- start start 2023-11-16T16:45:51.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.911+0000 7fec7dfae700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec78100540 0x7fec78198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:51.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.911+0000 7fec7dfae700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec78100ee0 0x7fec78198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:51.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.911+0000 7fec7dfae700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec781055e0 0x7fec7819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:51.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.911+0000 7fec7dfae700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7819c5c0 con 0x7fec781055e0 2023-11-16T16:45:51.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.911+0000 7fec7dfae700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7819c730 con 0x7fec78100ee0 2023-11-16T16:45:51.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.911+0000 7fec7dfae700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7819c8f0 con 0x7fec78100540 2023-11-16T16:45:51.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec77fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec781055e0 0x7fec7819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:51.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec777fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec78100540 0x7fec78198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:51.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec77fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec781055e0 0x7fec7819bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49848/0 (socket says 172.21.15.93:49848) 2023-11-16T16:45:51.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec77fff700 1 -- 172.21.15.93:0/1115291204 learned_addr learned my addr 172.21.15.93:0/1115291204 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:45:51.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec76ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec78100ee0 0x7fec78198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:51.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec77fff700 1 -- 172.21.15.93:0/1115291204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec78100540 msgr2=0x7fec78198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:51.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec77fff700 1 --2- 172.21.15.93:0/1115291204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec78100540 0x7fec78198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:51.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec77fff700 1 -- 172.21.15.93:0/1115291204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec78100ee0 msgr2=0x7fec78198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:51.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec77fff700 1 --2- 172.21.15.93:0/1115291204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec78100ee0 0x7fec78198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:51.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec77fff700 1 -- 172.21.15.93:0/1115291204 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec70009d00 con 0x7fec781055e0 2023-11-16T16:45:51.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec77fff700 1 --2- 172.21.15.93:0/1115291204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec781055e0 0x7fec7819bfc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fec70007890 tx=0x7fec700078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:51.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec74ff9700 1 -- 172.21.15.93:0/1115291204 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7000e040 con 0x7fec781055e0 2023-11-16T16:45:51.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec74ff9700 1 -- 172.21.15.93:0/1115291204 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec7000bd30 con 0x7fec781055e0 2023-11-16T16:45:51.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec74ff9700 1 -- 172.21.15.93:0/1115291204 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7001d470 con 0x7fec781055e0 2023-11-16T16:45:51.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.912+0000 7fec7dfae700 1 -- 172.21.15.93:0/1115291204 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec78110370 con 0x7fec781055e0 2023-11-16T16:45:51.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.913+0000 7fec7dfae700 1 -- 172.21.15.93:0/1115291204 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec781101e0 con 0x7fec781055e0 2023-11-16T16:45:51.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.914+0000 7fec7dfae700 1 -- 172.21.15.93:0/1115291204 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec78110500 con 0x7fec781055e0 2023-11-16T16:45:51.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.915+0000 7fec74ff9700 1 -- 172.21.15.93:0/1115291204 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fec70004a50 con 0x7fec781055e0 2023-11-16T16:45:51.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.917+0000 7fec74ff9700 1 --2- 172.21.15.93:0/1115291204 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fec600a11d0 0x7fec600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:51.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.917+0000 7fec74ff9700 1 -- 172.21.15.93:0/1115291204 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fec700cd290 con 0x7fec781055e0 2023-11-16T16:45:51.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.917+0000 7fec777fe700 1 --2- 172.21.15.93:0/1115291204 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fec600a11d0 0x7fec600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:51.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.918+0000 7fec777fe700 1 --2- 172.21.15.93:0/1115291204 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fec600a11d0 0x7fec600a3680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fec68009c60 tx=0x7fec68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:51.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:51.921+0000 7fec74ff9700 1 -- 172.21.15.93:0/1115291204 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fec70096e70 con 0x7fec781055e0 2023-11-16T16:45:52.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.117+0000 7fec7dfae700 1 -- 172.21.15.93:0/1115291204 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec781a5770 con 0x7fec781055e0 2023-11-16T16:45:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.118+0000 7fec74ff9700 1 -- 172.21.15.93:0/1115291204 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18432 (secure 0 0 0) 0x7fec7009a490 con 0x7fec781055e0 2023-11-16T16:45:52.120 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:45:52.120 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":80,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:45:47.266367+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:45:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.120+0000 7fec7dfae700 1 -- 172.21.15.93:0/1115291204 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fec600a11d0 msgr2=0x7fec600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.120+0000 7fec7dfae700 1 --2- 172.21.15.93:0/1115291204 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fec600a11d0 0x7fec600a3680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fec68009c60 tx=0x7fec68009260 comp rx=0 tx=0).stop 2023-11-16T16:45:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.120+0000 7fec7dfae700 1 -- 172.21.15.93:0/1115291204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec781055e0 msgr2=0x7fec7819bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.120+0000 7fec7dfae700 1 --2- 172.21.15.93:0/1115291204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec781055e0 0x7fec7819bfc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fec70007890 tx=0x7fec700078c0 comp rx=0 tx=0).stop 2023-11-16T16:45:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.120+0000 7fec7dfae700 1 -- 172.21.15.93:0/1115291204 shutdown_connections 2023-11-16T16:45:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.120+0000 7fec7dfae700 1 --2- 172.21.15.93:0/1115291204 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fec600a11d0 0x7fec600a3680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.120+0000 7fec7dfae700 1 --2- 172.21.15.93:0/1115291204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec781055e0 0x7fec7819bfc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.120+0000 7fec7dfae700 1 --2- 172.21.15.93:0/1115291204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec78100540 0x7fec78198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.120+0000 7fec7dfae700 1 --2- 172.21.15.93:0/1115291204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec78100ee0 0x7fec78198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.121+0000 7fec7dfae700 1 -- 172.21.15.93:0/1115291204 >> 172.21.15.93:0/1115291204 conn(0x7fec780fbdc0 msgr2=0x7fec7810b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.121+0000 7fec7dfae700 1 -- 172.21.15.93:0/1115291204 shutdown_connections 2023-11-16T16:45:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:52.121+0000 7fec7dfae700 1 -- 172.21.15.93:0/1115291204 wait complete. 2023-11-16T16:45:52.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:52 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1115291204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:45:52.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:52 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1115291204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:45:52.594 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:42:13.029135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:45:52.594 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-11-16T16:45:52.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:52 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1115291204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:45:53.438 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:45:53.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:53 smithi125 ceph-mon[131076]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:53.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:53 smithi093 ceph-mon[97863]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:53.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:53 smithi159 ceph-mon[131528]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:54.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.821+0000 7f2abbd33700 1 -- 172.21.15.93:0/1039534920 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ab4100ee0 msgr2=0x7f2ab4104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:54.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.821+0000 7f2abbd33700 1 --2- 172.21.15.93:0/1039534920 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ab4100ee0 0x7f2ab4104eb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40099c0 tx=0x7f2aa4009cd0 comp rx=0 tx=0).stop 2023-11-16T16:45:54.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.821+0000 7f2abbd33700 1 -- 172.21.15.93:0/1039534920 shutdown_connections 2023-11-16T16:45:54.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.821+0000 7f2abbd33700 1 --2- 172.21.15.93:0/1039534920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ab41055e0 0x7f2ab410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:54.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.821+0000 7f2abbd33700 1 --2- 172.21.15.93:0/1039534920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4100540 0x7f2ab4100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:54.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.821+0000 7f2abbd33700 1 --2- 172.21.15.93:0/1039534920 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ab4100ee0 0x7f2ab4104eb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:54.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.821+0000 7f2abbd33700 1 -- 172.21.15.93:0/1039534920 >> 172.21.15.93:0/1039534920 conn(0x7f2ab40fbdc0 msgr2=0x7f2ab40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:54.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.821+0000 7f2abbd33700 1 -- 172.21.15.93:0/1039534920 shutdown_connections 2023-11-16T16:45:54.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.821+0000 7f2abbd33700 1 -- 172.21.15.93:0/1039534920 wait complete. 2023-11-16T16:45:54.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.822+0000 7f2abbd33700 1 Processor -- start 2023-11-16T16:45:54.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.823+0000 7f2abbd33700 1 -- start start 2023-11-16T16:45:54.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.823+0000 7f2abbd33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ab4100540 0x7f2ab4072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:54.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.823+0000 7f2abbd33700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4100ee0 0x7f2ab4073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:54.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.823+0000 7f2abbd33700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ab41055e0 0x7f2ab40738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:54.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.823+0000 7f2abbd33700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab41a4860 con 0x7f2ab4100540 2023-11-16T16:45:54.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.823+0000 7f2abbd33700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab41a49d0 con 0x7f2ab4100ee0 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.823+0000 7f2abbd33700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab41a4b90 con 0x7f2ab41055e0 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.823+0000 7f2ab9acf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ab4100540 0x7f2ab4072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.823+0000 7f2ab92ce700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4100ee0 0x7f2ab4073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.823+0000 7f2aba2d0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ab41055e0 0x7f2ab40738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.823+0000 7f2ab92ce700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4100ee0 0x7f2ab4073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:34720/0 (socket says 172.21.15.93:34720) 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.823+0000 7f2ab92ce700 1 -- 172.21.15.93:0/3015030039 learned_addr learned my addr 172.21.15.93:0/3015030039 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2ab92ce700 1 -- 172.21.15.93:0/3015030039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ab41055e0 msgr2=0x7f2ab40738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2ab92ce700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ab41055e0 0x7f2ab40738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2ab92ce700 1 -- 172.21.15.93:0/3015030039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ab4100540 msgr2=0x7f2ab4072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2ab92ce700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ab4100540 0x7f2ab4072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2ab92ce700 1 -- 172.21.15.93:0/3015030039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa40095f0 con 0x7f2ab4100ee0 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2aba2d0700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ab41055e0 0x7f2ab40738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2ab9acf700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ab4100540 0x7f2ab4072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:54.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2ab92ce700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4100ee0 0x7f2ab4073330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009990 tx=0x7f2aa400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:54.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2aaaffd700 1 -- 172.21.15.93:0/3015030039 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa401d070 con 0x7f2ab4100ee0 2023-11-16T16:45:54.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2abbd33700 1 -- 172.21.15.93:0/3015030039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab4110380 con 0x7f2ab4100ee0 2023-11-16T16:45:54.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2aaaffd700 1 -- 172.21.15.93:0/3015030039 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aa4005030 con 0x7f2ab4100ee0 2023-11-16T16:45:54.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2aaaffd700 1 -- 172.21.15.93:0/3015030039 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4004a50 con 0x7f2ab4100ee0 2023-11-16T16:45:54.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.824+0000 7f2abbd33700 1 -- 172.21.15.93:0/3015030039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab41101f0 con 0x7f2ab4100ee0 2023-11-16T16:45:54.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.825+0000 7f2abbd33700 1 -- 172.21.15.93:0/3015030039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab4110510 con 0x7f2ab4100ee0 2023-11-16T16:45:54.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.827+0000 7f2aaaffd700 1 -- 172.21.15.93:0/3015030039 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2aa4005430 con 0x7f2ab4100ee0 2023-11-16T16:45:54.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.829+0000 7f2aaaffd700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f2aa00a1150 0x7f2aa00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:54.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.829+0000 7f2aaaffd700 1 -- 172.21.15.93:0/3015030039 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2aa40cd5f0 con 0x7f2ab4100ee0 2023-11-16T16:45:54.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.829+0000 7f2ab9acf700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f2aa00a1150 0x7f2aa00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:54.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.829+0000 7f2ab9acf700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f2aa00a1150 0x7f2aa00a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0005950 tx=0x7f2ab000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:54.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:54.831+0000 7f2aaaffd700 1 -- 172.21.15.93:0/3015030039 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2aa40971d0 con 0x7f2ab4100ee0 2023-11-16T16:45:55.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.029+0000 7f2abbd33700 1 -- 172.21.15.93:0/3015030039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ab41a54b0 con 0x7f2ab4100ee0 2023-11-16T16:45:55.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.030+0000 7f2aaaffd700 1 -- 172.21.15.93:0/3015030039 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2aa409a7f0 con 0x7f2ab4100ee0 2023-11-16T16:45:55.031 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-11-16T16:45:55.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.032+0000 7f2abbd33700 1 -- 172.21.15.93:0/3015030039 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f2aa00a1150 msgr2=0x7f2aa00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:55.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.032+0000 7f2abbd33700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f2aa00a1150 0x7f2aa00a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0005950 tx=0x7f2ab000b410 comp rx=0 tx=0).stop 2023-11-16T16:45:55.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.032+0000 7f2abbd33700 1 -- 172.21.15.93:0/3015030039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4100ee0 msgr2=0x7f2ab4073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:55.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.032+0000 7f2abbd33700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4100ee0 0x7f2ab4073330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009990 tx=0x7f2aa400f690 comp rx=0 tx=0).stop 2023-11-16T16:45:55.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.032+0000 7f2abbd33700 1 -- 172.21.15.93:0/3015030039 shutdown_connections 2023-11-16T16:45:55.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.032+0000 7f2abbd33700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f2aa00a1150 0x7f2aa00a3600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:55.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.032+0000 7f2abbd33700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ab4100540 0x7f2ab4072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:55.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.032+0000 7f2abbd33700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ab41055e0 0x7f2ab40738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:55.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.032+0000 7f2abbd33700 1 --2- 172.21.15.93:0/3015030039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4100ee0 0x7f2ab4073330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:55.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.032+0000 7f2abbd33700 1 -- 172.21.15.93:0/3015030039 >> 172.21.15.93:0/3015030039 conn(0x7f2ab40fbdc0 msgr2=0x7f2ab40fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:55.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.033+0000 7f2abbd33700 1 -- 172.21.15.93:0/3015030039 shutdown_connections 2023-11-16T16:45:55.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:55.033+0000 7f2abbd33700 1 -- 172.21.15.93:0/3015030039 wait complete. 2023-11-16T16:45:55.513 INFO:tasks.cephadm:Adding mds.a on smithi093 2023-11-16T16:45:55.513 INFO:tasks.cephadm:Adding mds.d on smithi093 2023-11-16T16:45:55.513 INFO:tasks.cephadm:Adding mds.g on smithi093 2023-11-16T16:45:55.513 INFO:tasks.cephadm:Adding mds.j on smithi093 2023-11-16T16:45:55.513 INFO:tasks.cephadm:Adding mds.b on smithi125 2023-11-16T16:45:55.513 INFO:tasks.cephadm:Adding mds.e on smithi125 2023-11-16T16:45:55.513 INFO:tasks.cephadm:Adding mds.h on smithi125 2023-11-16T16:45:55.513 INFO:tasks.cephadm:Adding mds.k on smithi125 2023-11-16T16:45:55.513 INFO:tasks.cephadm:Adding mds.c on smithi159 2023-11-16T16:45:55.513 INFO:tasks.cephadm:Adding mds.f on smithi159 2023-11-16T16:45:55.514 INFO:tasks.cephadm:Adding mds.i on smithi159 2023-11-16T16:45:55.514 INFO:tasks.cephadm:Adding mds.l on smithi159 2023-11-16T16:45:55.514 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph orch apply mds all '12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l' 2023-11-16T16:45:55.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:55 smithi125 ceph-mon[131076]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:55.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:55 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3015030039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:45:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:55 smithi093 ceph-mon[97863]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:55 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3015030039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:45:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:55 smithi159 ceph-mon[131528]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:55 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3015030039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:45:56.170 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:45:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.347+0000 7fa91639e700 1 -- 172.21.15.159:0/3088113853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9100ff1e0 msgr2=0x7fa9100ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.347+0000 7fa91639e700 1 --2- 172.21.15.159:0/3088113853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9100ff1e0 0x7fa9100ff5f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa900009990 tx=0x7fa900009ca0 comp rx=0 tx=0).stop 2023-11-16T16:45:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.347+0000 7fa91639e700 1 -- 172.21.15.159:0/3088113853 shutdown_connections 2023-11-16T16:45:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.347+0000 7fa91639e700 1 --2- 172.21.15.159:0/3088113853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa910100d70 0x7fa910107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.347+0000 7fa91639e700 1 --2- 172.21.15.159:0/3088113853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9100ff1e0 0x7fa9100ff5f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:57.350 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.347+0000 7fa91639e700 1 --2- 172.21.15.159:0/3088113853 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa9101003e0 0x7fa910100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:57.350 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.347+0000 7fa91639e700 1 -- 172.21.15.159:0/3088113853 >> 172.21.15.159:0/3088113853 conn(0x7fa9100fa7f0 msgr2=0x7fa9100fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:57.350 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.347+0000 7fa91639e700 1 -- 172.21.15.159:0/3088113853 shutdown_connections 2023-11-16T16:45:57.350 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.347+0000 7fa91639e700 1 -- 172.21.15.159:0/3088113853 wait complete. 2023-11-16T16:45:57.350 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.347+0000 7fa91639e700 1 Processor -- start 2023-11-16T16:45:57.350 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.347+0000 7fa91639e700 1 -- start start 2023-11-16T16:45:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa91639e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa9100ff1e0 0x7fa910194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa91639e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9101003e0 0x7fa9101954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa91639e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa910100d70 0x7fa91019c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa91639e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91019cb40 con 0x7fa9100ff1e0 2023-11-16T16:45:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa91639e700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91019cc80 con 0x7fa910100d70 2023-11-16T16:45:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa91639e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91019ce10 con 0x7fa9101003e0 2023-11-16T16:45:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa90f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9101003e0 0x7fa9101954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa90f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9101003e0 0x7fa9101954f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54562/0 (socket says 172.21.15.159:54562) 2023-11-16T16:45:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa90f7fe700 1 -- 172.21.15.159:0/2823849204 learned_addr learned my addr 172.21.15.159:0/2823849204 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-11-16T16:45:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa91493b700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa910100d70 0x7fa91019c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa90ffff700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa9100ff1e0 0x7fa910194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa90f7fe700 1 -- 172.21.15.159:0/2823849204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa910100d70 msgr2=0x7fa91019c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa90f7fe700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa910100d70 0x7fa91019c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa90f7fe700 1 -- 172.21.15.159:0/2823849204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa9100ff1e0 msgr2=0x7fa910194fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa90f7fe700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa9100ff1e0 0x7fa910194fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa90f7fe700 1 -- 172.21.15.159:0/2823849204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9000095f0 con 0x7fa9101003e0 2023-11-16T16:45:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa90ffff700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa9100ff1e0 0x7fa910194fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:45:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa91493b700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa910100d70 0x7fa91019c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:45:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.348+0000 7fa90f7fe700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9101003e0 0x7fa9101954f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9040099a0 tx=0x7fa904009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.349+0000 7fa90d7fa700 1 -- 172.21.15.159:0/2823849204 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904012070 con 0x7fa9101003e0 2023-11-16T16:45:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.349+0000 7fa91639e700 1 -- 172.21.15.159:0/2823849204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91010ab30 con 0x7fa9101003e0 2023-11-16T16:45:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.349+0000 7fa90d7fa700 1 -- 172.21.15.159:0/2823849204 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa90400d260 con 0x7fa9101003e0 2023-11-16T16:45:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.349+0000 7fa90d7fa700 1 -- 172.21.15.159:0/2823849204 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90400caa0 con 0x7fa9101003e0 2023-11-16T16:45:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.349+0000 7fa91639e700 1 -- 172.21.15.159:0/2823849204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91010a9a0 con 0x7fa9101003e0 2023-11-16T16:45:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.350+0000 7fa91639e700 1 -- 172.21.15.159:0/2823849204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa910066ea0 con 0x7fa9101003e0 2023-11-16T16:45:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.351+0000 7fa90d7fa700 1 -- 172.21.15.159:0/2823849204 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa904007ce0 con 0x7fa9101003e0 2023-11-16T16:45:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.351+0000 7fa90d7fa700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa8f80a1150 0x7fa8f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.351+0000 7fa90d7fa700 1 -- 172.21.15.159:0/2823849204 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa9040cd530 con 0x7fa9101003e0 2023-11-16T16:45:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.352+0000 7fa90ffff700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa8f80a1150 0x7fa8f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.352+0000 7fa90ffff700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa8f80a1150 0x7fa8f80a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa9000095c0 tx=0x7fa900009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.356+0000 7fa90d7fa700 1 -- 172.21.15.159:0/2823849204 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa904050730 con 0x7fa9101003e0 2023-11-16T16:45:57.551 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.550+0000 7fa91639e700 1 -- 172.21.15.159:0/2823849204 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fa910061030 con 0x7fa8f80a1150 2023-11-16T16:45:57.560 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.559+0000 7fa90d7fa700 1 -- 172.21.15.159:0/2823849204 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fa910061030 con 0x7fa8f80a1150 2023-11-16T16:45:57.560 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mds.all update... 2023-11-16T16:45:57.562 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.561+0000 7fa91639e700 1 -- 172.21.15.159:0/2823849204 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa8f80a1150 msgr2=0x7fa8f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:57.562 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.561+0000 7fa91639e700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa8f80a1150 0x7fa8f80a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa9000095c0 tx=0x7fa900009510 comp rx=0 tx=0).stop 2023-11-16T16:45:57.563 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.561+0000 7fa91639e700 1 -- 172.21.15.159:0/2823849204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9101003e0 msgr2=0x7fa9101954f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:57.563 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.561+0000 7fa91639e700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9101003e0 0x7fa9101954f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9040099a0 tx=0x7fa904009d60 comp rx=0 tx=0).stop 2023-11-16T16:45:57.563 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.561+0000 7fa91639e700 1 -- 172.21.15.159:0/2823849204 shutdown_connections 2023-11-16T16:45:57.563 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.561+0000 7fa91639e700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa8f80a1150 0x7fa8f80a3600 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:57.563 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.561+0000 7fa91639e700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa9100ff1e0 0x7fa910194fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:57.563 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.561+0000 7fa91639e700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa910100d70 0x7fa91019c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:57.564 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.561+0000 7fa91639e700 1 --2- 172.21.15.159:0/2823849204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9101003e0 0x7fa9101954f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:57.564 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.561+0000 7fa91639e700 1 -- 172.21.15.159:0/2823849204 >> 172.21.15.159:0/2823849204 conn(0x7fa9100fa7f0 msgr2=0x7fa9100fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:57.564 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.561+0000 7fa91639e700 1 -- 172.21.15.159:0/2823849204 shutdown_connections 2023-11-16T16:45:57.564 INFO:teuthology.orchestra.run.smithi159.stderr:2023-11-16T16:45:57.561+0000 7fa91639e700 1 -- 172.21.15.159:0/2823849204 wait complete. 2023-11-16T16:45:57.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:57 smithi125 ceph-mon[131076]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:57.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:57 smithi093 ceph-mon[97863]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:57.947 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:57 smithi159 ceph-mon[131528]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:58.024 DEBUG:teuthology.orchestra.run.smithi093:mds.a> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.a.service 2023-11-16T16:45:58.029 DEBUG:teuthology.orchestra.run.smithi093:mds.d> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.d.service 2023-11-16T16:45:58.075 DEBUG:teuthology.orchestra.run.smithi093:mds.g> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.g.service 2023-11-16T16:45:58.115 INFO:journalctl@ceph.mds.a.smithi093.stdout:-- Logs begin at Thu 2023-11-16 16:19:38 UTC. -- 2023-11-16T16:45:58.119 DEBUG:teuthology.orchestra.run.smithi093:mds.j> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.j.service 2023-11-16T16:45:58.157 INFO:journalctl@ceph.mds.d.smithi093.stdout:-- Logs begin at Thu 2023-11-16 16:19:38 UTC. -- 2023-11-16T16:45:58.161 DEBUG:teuthology.orchestra.run.smithi125:mds.b> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.b.service 2023-11-16T16:45:58.163 DEBUG:teuthology.orchestra.run.smithi125:mds.e> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.e.service 2023-11-16T16:45:58.193 INFO:journalctl@ceph.mds.b.smithi125.stdout:-- Logs begin at Thu 2023-11-16 16:14:41 UTC. -- 2023-11-16T16:45:58.194 DEBUG:teuthology.orchestra.run.smithi125:mds.h> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.h.service 2023-11-16T16:45:58.196 INFO:journalctl@ceph.mds.g.smithi093.stdout:-- Logs begin at Thu 2023-11-16 16:19:38 UTC. -- 2023-11-16T16:45:58.220 INFO:journalctl@ceph.mds.e.smithi125.stdout:-- Logs begin at Thu 2023-11-16 16:14:41 UTC. -- 2023-11-16T16:45:58.221 DEBUG:teuthology.orchestra.run.smithi125:mds.k> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.k.service 2023-11-16T16:45:58.236 INFO:journalctl@ceph.mds.j.smithi093.stdout:-- Logs begin at Thu 2023-11-16 16:19:38 UTC. -- 2023-11-16T16:45:58.246 INFO:journalctl@ceph.mds.h.smithi125.stdout:-- Logs begin at Thu 2023-11-16 16:14:41 UTC. -- 2023-11-16T16:45:58.247 DEBUG:teuthology.orchestra.run.smithi159:mds.c> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.c.service 2023-11-16T16:45:58.250 DEBUG:teuthology.orchestra.run.smithi159:mds.f> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.f.service 2023-11-16T16:45:58.272 INFO:journalctl@ceph.mds.k.smithi125.stdout:-- Logs begin at Thu 2023-11-16 16:14:41 UTC. -- 2023-11-16T16:45:58.280 INFO:journalctl@ceph.mds.c.smithi159.stdout:-- Logs begin at Thu 2023-11-16 16:15:38 UTC. -- 2023-11-16T16:45:58.281 DEBUG:teuthology.orchestra.run.smithi159:mds.i> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.i.service 2023-11-16T16:45:58.307 INFO:journalctl@ceph.mds.f.smithi159.stdout:-- Logs begin at Thu 2023-11-16 16:15:38 UTC. -- 2023-11-16T16:45:58.307 DEBUG:teuthology.orchestra.run.smithi159:mds.l> sudo journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.l.service 2023-11-16T16:45:58.334 INFO:journalctl@ceph.mds.i.smithi159.stdout:-- Logs begin at Thu 2023-11-16 16:15:38 UTC. -- 2023-11-16T16:45:58.335 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-11-16T16:45:58.335 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:45:58.361 INFO:journalctl@ceph.mds.l.smithi159.stdout:-- Logs begin at Thu 2023-11-16 16:15:38 UTC. -- 2023-11-16T16:45:58.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.546+0000 7f44cb3ff700 1 -- 172.21.15.93:0/3744667719 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44c4101480 msgr2=0x7f44c41018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:58.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.546+0000 7f44cb3ff700 1 --2- 172.21.15.93:0/3744667719 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44c4101480 0x7f44c41018d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009990 tx=0x7f44b4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.546+0000 7f44cb3ff700 1 -- 172.21.15.93:0/3744667719 shutdown_connections 2023-11-16T16:45:58.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.546+0000 7f44cb3ff700 1 --2- 172.21.15.93:0/3744667719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44c4101e10 0x7f44c41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.546+0000 7f44cb3ff700 1 --2- 172.21.15.93:0/3744667719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c4100290 0x7f44c41006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.546+0000 7f44cb3ff700 1 --2- 172.21.15.93:0/3744667719 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44c4101480 0x7f44c41018d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.546+0000 7f44cb3ff700 1 -- 172.21.15.93:0/3744667719 >> 172.21.15.93:0/3744667719 conn(0x7f44c40fb870 msgr2=0x7f44c40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:58.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.547+0000 7f44cb3ff700 1 -- 172.21.15.93:0/3744667719 shutdown_connections 2023-11-16T16:45:58.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.547+0000 7f44cb3ff700 1 -- 172.21.15.93:0/3744667719 wait complete. 2023-11-16T16:45:58.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44cb3ff700 1 Processor -- start 2023-11-16T16:45:58.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44cb3ff700 1 -- start start 2023-11-16T16:45:58.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44cb3ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44c4100290 0x7f44c4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:58.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44cb3ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44c4101480 0x7f44c41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:58.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44cb3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c4101e10 0x7f44c419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:58.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44cb3ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c419fd10 con 0x7f44c4101480 2023-11-16T16:45:58.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44cb3ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c419fe50 con 0x7f44c4100290 2023-11-16T16:45:58.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44cb3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c419ffe0 con 0x7f44c4101e10 2023-11-16T16:45:58.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44c899a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44c4101480 0x7f44c41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:58.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44c899a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44c4101480 0x7f44c41986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42206/0 (socket says 172.21.15.93:42206) 2023-11-16T16:45:58.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44c919b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44c4100290 0x7f44c4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:58.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44c899a700 1 -- 172.21.15.93:0/2135147041 learned_addr learned my addr 172.21.15.93:0/2135147041 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:45:58.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.548+0000 7f44c999c700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c4101e10 0x7f44c419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:58.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.549+0000 7f44c919b700 1 -- 172.21.15.93:0/2135147041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c4101e10 msgr2=0x7f44c419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:58.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.549+0000 7f44c919b700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c4101e10 0x7f44c419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.549+0000 7f44c919b700 1 -- 172.21.15.93:0/2135147041 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44c4101480 msgr2=0x7f44c41986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:58.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.549+0000 7f44c919b700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44c4101480 0x7f44c41986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.549+0000 7f44c919b700 1 -- 172.21.15.93:0/2135147041 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b40095f0 con 0x7f44c4100290 2023-11-16T16:45:58.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.549+0000 7f44c899a700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44c4101480 0x7f44c41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:58.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.549+0000 7f44c919b700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44c4100290 0x7f44c4198190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f44c000d720 tx=0x7f44c000da30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:58.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.549+0000 7f44ba7fc700 1 -- 172.21.15.93:0/2135147041 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c000f690 con 0x7f44c4100290 2023-11-16T16:45:58.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.549+0000 7f44cb3ff700 1 -- 172.21.15.93:0/2135147041 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c410bc50 con 0x7f44c4100290 2023-11-16T16:45:58.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.549+0000 7f44ba7fc700 1 -- 172.21.15.93:0/2135147041 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44c000fcb0 con 0x7f44c4100290 2023-11-16T16:45:58.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.549+0000 7f44ba7fc700 1 -- 172.21.15.93:0/2135147041 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0010e60 con 0x7f44c4100290 2023-11-16T16:45:58.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.549+0000 7f44cb3ff700 1 -- 172.21.15.93:0/2135147041 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c410bac0 con 0x7f44c4100290 2023-11-16T16:45:58.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.550+0000 7f44cb3ff700 1 -- 172.21.15.93:0/2135147041 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c41928e0 con 0x7f44c4100290 2023-11-16T16:45:58.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.552+0000 7f44ba7fc700 1 -- 172.21.15.93:0/2135147041 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f44c000f7f0 con 0x7f44c4100290 2023-11-16T16:45:58.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.553+0000 7f44ba7fc700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f44b00a1150 0x7f44b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:58.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.554+0000 7f44ba7fc700 1 -- 172.21.15.93:0/2135147041 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f44c00cc110 con 0x7f44c4100290 2023-11-16T16:45:58.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.554+0000 7f44c899a700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f44b00a1150 0x7f44b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:58.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.554+0000 7f44c899a700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f44b00a1150 0x7f44b00a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44b4006010 tx=0x7f44b400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:58.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.557+0000 7f44ba7fc700 1 -- 172.21.15.93:0/2135147041 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f44c0095c60 con 0x7f44c4100290 2023-11-16T16:45:58.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:58 smithi093 ceph-mon[97863]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:45:58.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:58 smithi093 ceph-mon[97863]: Saving service mds.all spec with placement smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2023-11-16T16:45:58.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:58.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:58.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:58.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:58.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:58.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:45:58.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:45:58.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:58 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:58.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.762+0000 7f44cb3ff700 1 -- 172.21.15.93:0/2135147041 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44c4067a60 con 0x7f44c4100290 2023-11-16T16:45:58.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.763+0000 7f44ba7fc700 1 -- 172.21.15.93:0/2135147041 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18432 (secure 0 0 0) 0x7f44c0099280 con 0x7f44c4100290 2023-11-16T16:45:58.765 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:45:58.765 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":80,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:45:47.266367+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:45:58.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.766+0000 7f44cb3ff700 1 -- 172.21.15.93:0/2135147041 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f44b00a1150 msgr2=0x7f44b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:58.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.766+0000 7f44cb3ff700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f44b00a1150 0x7f44b00a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44b4006010 tx=0x7f44b400b260 comp rx=0 tx=0).stop 2023-11-16T16:45:58.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.766+0000 7f44cb3ff700 1 -- 172.21.15.93:0/2135147041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44c4100290 msgr2=0x7f44c4198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:58.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.766+0000 7f44cb3ff700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44c4100290 0x7f44c4198190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f44c000d720 tx=0x7f44c000da30 comp rx=0 tx=0).stop 2023-11-16T16:45:58.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.766+0000 7f44cb3ff700 1 -- 172.21.15.93:0/2135147041 shutdown_connections 2023-11-16T16:45:58.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.766+0000 7f44cb3ff700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f44b00a1150 0x7f44b00a3600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.766+0000 7f44cb3ff700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c4101e10 0x7f44c419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.766+0000 7f44cb3ff700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44c4100290 0x7f44c4198190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.766+0000 7f44cb3ff700 1 --2- 172.21.15.93:0/2135147041 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44c4101480 0x7f44c41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.766+0000 7f44cb3ff700 1 -- 172.21.15.93:0/2135147041 >> 172.21.15.93:0/2135147041 conn(0x7f44c40fb870 msgr2=0x7f44c4106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:58.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.766+0000 7f44cb3ff700 1 -- 172.21.15.93:0/2135147041 shutdown_connections 2023-11-16T16:45:58.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.766+0000 7f44cb3ff700 1 -- 172.21.15.93:0/2135147041 wait complete. 2023-11-16T16:45:58.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:42:13.029135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:45:58.788 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:45:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:58 smithi125 ceph-mon[131076]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:45:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:58 smithi125 ceph-mon[131076]: Saving service mds.all spec with placement smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2023-11-16T16:45:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:45:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:45:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:58 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:58.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:58 smithi159 ceph-mon[131528]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:45:58.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:58 smithi159 ceph-mon[131528]: Saving service mds.all spec with placement smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2023-11-16T16:45:58.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:58.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:58.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:58.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:58.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:45:58.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:45:58.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:45:58.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:58 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:58.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.987+0000 7f3af65ff700 1 -- 172.21.15.93:0/3181150924 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3af0101e40 msgr2=0x7f3af0108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:58.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.987+0000 7f3af65ff700 1 --2- 172.21.15.93:0/3181150924 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3af0101e40 0x7f3af0108700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800b0a0 tx=0x7f3ae800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.987+0000 7f3af65ff700 1 -- 172.21.15.93:0/3181150924 shutdown_connections 2023-11-16T16:45:58.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.987+0000 7f3af65ff700 1 --2- 172.21.15.93:0/3181150924 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3af0101e40 0x7f3af0108700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.987+0000 7f3af65ff700 1 --2- 172.21.15.93:0/3181150924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3af01002c0 0x7f3af01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.987+0000 7f3af65ff700 1 --2- 172.21.15.93:0/3181150924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3af01014b0 0x7f3af0101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.987+0000 7f3af65ff700 1 -- 172.21.15.93:0/3181150924 >> 172.21.15.93:0/3181150924 conn(0x7f3af00fb8a0 msgr2=0x7f3af00fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:58.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.987+0000 7f3af65ff700 1 -- 172.21.15.93:0/3181150924 shutdown_connections 2023-11-16T16:45:58.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.987+0000 7f3af65ff700 1 -- 172.21.15.93:0/3181150924 wait complete. 2023-11-16T16:45:58.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.988+0000 7f3af65ff700 1 Processor -- start 2023-11-16T16:45:58.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.988+0000 7f3af65ff700 1 -- start start 2023-11-16T16:45:58.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3af65ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3af01002c0 0x7f3af00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:58.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3af65ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3af01014b0 0x7f3af0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3af65ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3af0101e40 0x7f3af0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3af65ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0073bc0 con 0x7f3af01002c0 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3af65ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0073d30 con 0x7f3af01014b0 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3af65ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af01a01a0 con 0x7f3af0101e40 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3aeffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3af01002c0 0x7f3af00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3aeffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3af01002c0 0x7f3af00748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42230/0 (socket says 172.21.15.93:42230) 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3aeffff700 1 -- 172.21.15.93:0/3989222264 learned_addr learned my addr 172.21.15.93:0/3989222264 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3aef7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3af01014b0 0x7f3af0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3af4b9c700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3af0101e40 0x7f3af0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3aeffff700 1 -- 172.21.15.93:0/3989222264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3af0101e40 msgr2=0x7f3af0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3aeffff700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3af0101e40 0x7f3af0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3aeffff700 1 -- 172.21.15.93:0/3989222264 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3af01014b0 msgr2=0x7f3af0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:58.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3aeffff700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3af01014b0 0x7f3af0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:58.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3aeffff700 1 -- 172.21.15.93:0/3989222264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0009520 con 0x7f3af01002c0 2023-11-16T16:45:58.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.989+0000 7f3aef7fe700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3af01014b0 0x7f3af0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:58.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.990+0000 7f3aeffff700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3af01002c0 0x7f3af00748a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000ea50 tx=0x7f3ae000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:58.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.990+0000 7f3aed7fa700 1 -- 172.21.15.93:0/3989222264 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000cbd0 con 0x7f3af01002c0 2023-11-16T16:45:58.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.990+0000 7f3aed7fa700 1 -- 172.21.15.93:0/3989222264 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ae000cd30 con 0x7f3af01002c0 2023-11-16T16:45:58.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.990+0000 7f3af65ff700 1 -- 172.21.15.93:0/3989222264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae8009d00 con 0x7f3af01002c0 2023-11-16T16:45:58.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.990+0000 7f3aed7fa700 1 -- 172.21.15.93:0/3989222264 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0018680 con 0x7f3af01002c0 2023-11-16T16:45:58.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.990+0000 7f3af65ff700 1 -- 172.21.15.93:0/3989222264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af010bc90 con 0x7f3af01002c0 2023-11-16T16:45:58.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.991+0000 7f3af65ff700 1 -- 172.21.15.93:0/3989222264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af0067b00 con 0x7f3af01002c0 2023-11-16T16:45:58.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.993+0000 7f3aed7fa700 1 -- 172.21.15.93:0/3989222264 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3ae0010430 con 0x7f3af01002c0 2023-11-16T16:45:58.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.994+0000 7f3aed7fa700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3ad80a1150 0x7f3ad80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:58.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.994+0000 7f3aed7fa700 1 -- 172.21.15.93:0/3989222264 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3ae0014070 con 0x7f3af01002c0 2023-11-16T16:45:58.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.994+0000 7f3aef7fe700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3ad80a1150 0x7f3ad80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:58.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.995+0000 7f3aef7fe700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3ad80a1150 0x7f3ad80a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4011490 tx=0x7f3ae4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:58.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:58.998+0000 7f3aed7fa700 1 -- 172.21.15.93:0/3989222264 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3ae0096500 con 0x7f3af01002c0 2023-11-16T16:45:59.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.198+0000 7f3af65ff700 1 -- 172.21.15.93:0/3989222264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3af01070d0 con 0x7f3af01002c0 2023-11-16T16:45:59.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.198+0000 7f3aed7fa700 1 -- 172.21.15.93:0/3989222264 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ae0099b20 con 0x7f3af01002c0 2023-11-16T16:45:59.200 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-11-16T16:45:59.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.201+0000 7f3af65ff700 1 -- 172.21.15.93:0/3989222264 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3ad80a1150 msgr2=0x7f3ad80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:59.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.201+0000 7f3af65ff700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3ad80a1150 0x7f3ad80a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4011490 tx=0x7f3ae4009260 comp rx=0 tx=0).stop 2023-11-16T16:45:59.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.201+0000 7f3af65ff700 1 -- 172.21.15.93:0/3989222264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3af01002c0 msgr2=0x7f3af00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:59.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.201+0000 7f3af65ff700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3af01002c0 0x7f3af00748a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000ea50 tx=0x7f3ae000ed60 comp rx=0 tx=0).stop 2023-11-16T16:45:59.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.201+0000 7f3af65ff700 1 -- 172.21.15.93:0/3989222264 shutdown_connections 2023-11-16T16:45:59.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.201+0000 7f3af65ff700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3ad80a1150 0x7f3ad80a3600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.201+0000 7f3af65ff700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3af01002c0 0x7f3af00748a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.201+0000 7f3af65ff700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3af0101e40 0x7f3af0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.201+0000 7f3af65ff700 1 --2- 172.21.15.93:0/3989222264 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3af01014b0 0x7f3af0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.201+0000 7f3af65ff700 1 -- 172.21.15.93:0/3989222264 >> 172.21.15.93:0/3989222264 conn(0x7f3af00fb8a0 msgr2=0x7f3af00fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:59.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.201+0000 7f3af65ff700 1 -- 172.21.15.93:0/3989222264 shutdown_connections 2023-11-16T16:45:59.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.201+0000 7f3af65ff700 1 -- 172.21.15.93:0/3989222264 wait complete. 2023-11-16T16:45:59.222 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:45:59.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.417+0000 7f93a2fff700 1 -- 172.21.15.93:0/1283457522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f939c101480 msgr2=0x7f939c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:59.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.417+0000 7f93a2fff700 1 --2- 172.21.15.93:0/1283457522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f939c101480 0x7f939c1018d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9388009990 tx=0x7f9388009ca0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.417+0000 7f93a2fff700 1 -- 172.21.15.93:0/1283457522 shutdown_connections 2023-11-16T16:45:59.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.417+0000 7f93a2fff700 1 --2- 172.21.15.93:0/1283457522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939c101e10 0x7f939c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.417+0000 7f93a2fff700 1 --2- 172.21.15.93:0/1283457522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f939c100290 0x7f939c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.417+0000 7f93a2fff700 1 --2- 172.21.15.93:0/1283457522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f939c101480 0x7f939c1018d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.417+0000 7f93a2fff700 1 -- 172.21.15.93:0/1283457522 >> 172.21.15.93:0/1283457522 conn(0x7f939c0fb870 msgr2=0x7f939c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:59.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.417+0000 7f93a2fff700 1 -- 172.21.15.93:0/1283457522 shutdown_connections 2023-11-16T16:45:59.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.417+0000 7f93a2fff700 1 -- 172.21.15.93:0/1283457522 wait complete. 2023-11-16T16:45:59.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.418+0000 7f93a2fff700 1 Processor -- start 2023-11-16T16:45:59.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.418+0000 7f93a2fff700 1 -- start start 2023-11-16T16:45:59.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.418+0000 7f93a2fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f939c100290 0x7f939c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:59.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a2fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f939c101480 0x7f939c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:59.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a2fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939c101e10 0x7f939c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:59.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a2fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c19fd30 con 0x7f939c101480 2023-11-16T16:45:59.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a2fff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c19fe70 con 0x7f939c101e10 2023-11-16T16:45:59.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a2fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c1a0000 con 0x7f939c100290 2023-11-16T16:45:59.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a0d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f939c100290 0x7f939c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:59.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a0d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f939c100290 0x7f939c1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:53594/0 (socket says 172.21.15.93:53594) 2023-11-16T16:45:59.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a0d9b700 1 -- 172.21.15.93:0/311839745 learned_addr learned my addr 172.21.15.93:0/311839745 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:45:59.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a159c700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939c101e10 0x7f939c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:59.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f939bfff700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f939c101480 0x7f939c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:59.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a0d9b700 1 -- 172.21.15.93:0/311839745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939c101e10 msgr2=0x7f939c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:59.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a0d9b700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939c101e10 0x7f939c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a0d9b700 1 -- 172.21.15.93:0/311839745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f939c101480 msgr2=0x7f939c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:59.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a0d9b700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f939c101480 0x7f939c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a0d9b700 1 -- 172.21.15.93:0/311839745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93880095f0 con 0x7f939c100290 2023-11-16T16:45:59.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.419+0000 7f93a0d9b700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f939c100290 0x7f939c1981b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f939000d980 tx=0x7f939000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:59.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.420+0000 7f9399ffb700 1 -- 172.21.15.93:0/311839745 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390004020 con 0x7f939c100290 2023-11-16T16:45:59.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.420+0000 7f9399ffb700 1 -- 172.21.15.93:0/311839745 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f939000f690 con 0x7f939c100290 2023-11-16T16:45:59.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.420+0000 7f93a2fff700 1 -- 172.21.15.93:0/311839745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c10bc50 con 0x7f939c100290 2023-11-16T16:45:59.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.420+0000 7f9399ffb700 1 -- 172.21.15.93:0/311839745 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390003a60 con 0x7f939c100290 2023-11-16T16:45:59.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.420+0000 7f93a2fff700 1 -- 172.21.15.93:0/311839745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c10bac0 con 0x7f939c100290 2023-11-16T16:45:59.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.421+0000 7f93a2fff700 1 -- 172.21.15.93:0/311839745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939c1928e0 con 0x7f939c100290 2023-11-16T16:45:59.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.423+0000 7f9399ffb700 1 -- 172.21.15.93:0/311839745 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f939000fb00 con 0x7f939c100290 2023-11-16T16:45:59.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.424+0000 7f9399ffb700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f938c0a1150 0x7f938c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:59.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.424+0000 7f9399ffb700 1 -- 172.21.15.93:0/311839745 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f93900cc100 con 0x7f939c100290 2023-11-16T16:45:59.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.424+0000 7f939bfff700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f938c0a1150 0x7f938c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:59.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.425+0000 7f939bfff700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f938c0a1150 0x7f938c0a3600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f938800b2e0 tx=0x7f9388005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:59.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.428+0000 7f9399ffb700 1 -- 172.21.15.93:0/311839745 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9390095c50 con 0x7f939c100290 2023-11-16T16:45:59.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.675+0000 7f93a2fff700 1 -- 172.21.15.93:0/311839745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f939c02cfe0 con 0x7f939c100290 2023-11-16T16:45:59.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.675+0000 7f9399ffb700 1 -- 172.21.15.93:0/311839745 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f9390099270 con 0x7f939c100290 2023-11-16T16:45:59.678 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:45:59.678 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-11-16T16:45:59.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.678+0000 7f93a2fff700 1 -- 172.21.15.93:0/311839745 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f938c0a1150 msgr2=0x7f938c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:59.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.678+0000 7f93a2fff700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f938c0a1150 0x7f938c0a3600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f938800b2e0 tx=0x7f9388005fb0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.678+0000 7f93a2fff700 1 -- 172.21.15.93:0/311839745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f939c100290 msgr2=0x7f939c1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:59.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.678+0000 7f93a2fff700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f939c100290 0x7f939c1981b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f939000d980 tx=0x7f939000dc90 comp rx=0 tx=0).stop 2023-11-16T16:45:59.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.678+0000 7f93a2fff700 1 -- 172.21.15.93:0/311839745 shutdown_connections 2023-11-16T16:45:59.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.678+0000 7f93a2fff700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f938c0a1150 0x7f938c0a3600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.678+0000 7f93a2fff700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939c101e10 0x7f939c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.678+0000 7f93a2fff700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f939c100290 0x7f939c1981b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.678+0000 7f93a2fff700 1 --2- 172.21.15.93:0/311839745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f939c101480 0x7f939c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:59.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.678+0000 7f93a2fff700 1 -- 172.21.15.93:0/311839745 >> 172.21.15.93:0/311839745 conn(0x7f939c0fb870 msgr2=0x7f939c106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:59.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.678+0000 7f93a2fff700 1 -- 172.21.15.93:0/311839745 shutdown_connections 2023-11-16T16:45:59.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:45:59.678+0000 7f93a2fff700 1 -- 172.21.15.93:0/311839745 wait complete. 2023-11-16T16:45:59.681 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 1 2023-11-16T16:45:59.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:59 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:45:59.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:59 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:45:59.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:59 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:45:59.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:59 smithi125 ceph-mon[131076]: Service mds.all has no related daemon already placed. 2023-11-16T16:45:59.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:59 smithi125 ceph-mon[131076]: Deploying daemon mds.l on smithi159 2023-11-16T16:45:59.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:59 smithi125 ceph-mon[131076]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:59.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:59 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2135147041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:45:59.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:45:59 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3989222264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:45:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:59 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:45:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:59 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:45:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:59 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:45:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:59 smithi093 ceph-mon[97863]: Service mds.all has no related daemon already placed. 2023-11-16T16:45:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:59 smithi093 ceph-mon[97863]: Deploying daemon mds.l on smithi159 2023-11-16T16:45:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:59 smithi093 ceph-mon[97863]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:59 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2135147041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:45:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:45:59 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3989222264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:45:59.985 INFO:journalctl@ceph.mds.l.smithi159.stdout:Nov 16 16:45:59 smithi159 systemd[1]: Starting Ceph mds.l for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:45:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:59 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:45:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:59 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:45:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:59 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:45:59.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:59 smithi159 ceph-mon[131528]: Service mds.all has no related daemon already placed. 2023-11-16T16:45:59.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:59 smithi159 ceph-mon[131528]: Deploying daemon mds.l on smithi159 2023-11-16T16:45:59.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:59 smithi159 ceph-mon[131528]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:45:59.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:59 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2135147041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:45:59.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:45:59 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3989222264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:46:00.302 INFO:journalctl@ceph.mds.l.smithi159.stdout:Nov 16 16:46:00 smithi159 podman[156619]: 2023-11-16T16:46:00.568 INFO:journalctl@ceph.mds.l.smithi159.stdout:Nov 16 16:46:00 smithi159 ceph-mds[156659]: -- 172.21.15.159:0/646396789 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56358e6c9520 con 0x56358e69ec00 2023-11-16T16:46:00.568 INFO:journalctl@ceph.mds.l.smithi159.stdout:Nov 16 16:46:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-l[156635]: starting mds.l at 2023-11-16T16:46:00.568 INFO:journalctl@ceph.mds.l.smithi159.stdout:Nov 16 16:46:00 smithi159 bash[156619]: 4f8b16423417c1a37908dca495c5704ea7b8a28041b8bcd991224e5f8c6ab8db 2023-11-16T16:46:00.568 INFO:journalctl@ceph.mds.l.smithi159.stdout:Nov 16 16:46:00 smithi159 systemd[1]: Started Ceph mds.l for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:46:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:00 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/311839745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:00 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.159:6832/2711358292,v1:172.21.15.159:6833/2711358292] up:boot 2023-11-16T16:46:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:00 smithi125 ceph-mon[131076]: fsmap 1 up:standby 2023-11-16T16:46:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-16T16:46:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:00 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/311839745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:00 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.159:6832/2711358292,v1:172.21.15.159:6833/2711358292] up:boot 2023-11-16T16:46:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:00 smithi093 ceph-mon[97863]: fsmap 1 up:standby 2023-11-16T16:46:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-16T16:46:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:00.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:00.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:00.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:00 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/311839745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:00 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.159:6832/2711358292,v1:172.21.15.159:6833/2711358292] up:boot 2023-11-16T16:46:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:00 smithi159 ceph-mon[131528]: fsmap 1 up:standby 2023-11-16T16:46:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-16T16:46:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:01.699 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:01.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:01 smithi125 ceph-mon[131076]: Deploying daemon mds.k on smithi125 2023-11-16T16:46:01.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:01 smithi125 ceph-mon[131076]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:01.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:01 smithi093 ceph-mon[97863]: Deploying daemon mds.k on smithi125 2023-11-16T16:46:01.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:01 smithi093 ceph-mon[97863]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:01.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.917+0000 7fee039ff700 1 -- 172.21.15.93:0/2043375002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedfc101e10 msgr2=0x7fedfc1086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:01.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.917+0000 7fee039ff700 1 --2- 172.21.15.93:0/2043375002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedfc101e10 0x7fedfc1086d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fedf400b0a0 tx=0x7fedf400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:46:01.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.917+0000 7fee039ff700 1 -- 172.21.15.93:0/2043375002 shutdown_connections 2023-11-16T16:46:01.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.917+0000 7fee039ff700 1 --2- 172.21.15.93:0/2043375002 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedfc100290 0x7fedfc1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:01.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.917+0000 7fee039ff700 1 --2- 172.21.15.93:0/2043375002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedfc101e10 0x7fedfc1086d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:01.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.917+0000 7fee039ff700 1 --2- 172.21.15.93:0/2043375002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fedfc101480 0x7fedfc1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:01.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.917+0000 7fee039ff700 1 -- 172.21.15.93:0/2043375002 >> 172.21.15.93:0/2043375002 conn(0x7fedfc0fb870 msgr2=0x7fedfc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:01.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.917+0000 7fee039ff700 1 -- 172.21.15.93:0/2043375002 shutdown_connections 2023-11-16T16:46:01.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.917+0000 7fee039ff700 1 -- 172.21.15.93:0/2043375002 wait complete. 2023-11-16T16:46:01.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.918+0000 7fee039ff700 1 Processor -- start 2023-11-16T16:46:01.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.918+0000 7fee039ff700 1 -- start start 2023-11-16T16:46:01.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.918+0000 7fee039ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fedfc100290 0x7fedfc196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:01.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.918+0000 7fee039ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedfc101480 0x7fedfc1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:01.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.918+0000 7fee039ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedfc101e10 0x7fedfc19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:01.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.918+0000 7fee039ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc19dc10 con 0x7fedfc101e10 2023-11-16T16:46:01.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.918+0000 7fee039ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc19dd50 con 0x7fedfc101480 2023-11-16T16:46:01.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.918+0000 7fee039ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc19dee0 con 0x7fedfc100290 2023-11-16T16:46:01.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.919+0000 7fee01f9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedfc101e10 0x7fedfc19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:01.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.919+0000 7fee0179b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fedfc100290 0x7fedfc196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:01.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.919+0000 7fee01f9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedfc101e10 0x7fedfc19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42244/0 (socket says 172.21.15.93:42244) 2023-11-16T16:46:01.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.919+0000 7fee00f9a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedfc101480 0x7fedfc1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:01.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.919+0000 7fee01f9c700 1 -- 172.21.15.93:0/371540781 learned_addr learned my addr 172.21.15.93:0/371540781 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:01.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.919+0000 7fee01f9c700 1 -- 172.21.15.93:0/371540781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fedfc100290 msgr2=0x7fedfc196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:01.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.919+0000 7fee01f9c700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fedfc100290 0x7fedfc196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:01.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.919+0000 7fee01f9c700 1 -- 172.21.15.93:0/371540781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedfc101480 msgr2=0x7fedfc1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:01.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.919+0000 7fee01f9c700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedfc101480 0x7fedfc1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:01.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.919+0000 7fee01f9c700 1 -- 172.21.15.93:0/371540781 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf4009d00 con 0x7fedfc101e10 2023-11-16T16:46:01.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.919+0000 7fee00f9a700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedfc101480 0x7fedfc1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:01.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.919+0000 7fee01f9c700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedfc101e10 0x7fedfc19d640 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fedf400d010 tx=0x7fedf400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:01.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.920+0000 7fedf27fc700 1 -- 172.21.15.93:0/371540781 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf400e040 con 0x7fedfc101e10 2023-11-16T16:46:01.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.920+0000 7fedf27fc700 1 -- 172.21.15.93:0/371540781 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedf4003e10 con 0x7fedfc101e10 2023-11-16T16:46:01.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.920+0000 7fee039ff700 1 -- 172.21.15.93:0/371540781 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedfc10bc70 con 0x7fedfc101e10 2023-11-16T16:46:01.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.920+0000 7fedf27fc700 1 -- 172.21.15.93:0/371540781 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf40052a0 con 0x7fedfc101e10 2023-11-16T16:46:01.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.920+0000 7fee039ff700 1 -- 172.21.15.93:0/371540781 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedfc10bae0 con 0x7fedfc101e10 2023-11-16T16:46:01.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.921+0000 7fee039ff700 1 -- 172.21.15.93:0/371540781 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedfc107770 con 0x7fedfc101e10 2023-11-16T16:46:01.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.923+0000 7fedf27fc700 1 -- 172.21.15.93:0/371540781 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fedf401d480 con 0x7fedfc101e10 2023-11-16T16:46:01.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.924+0000 7fedf27fc700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fede80a1150 0x7fede80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:01.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.924+0000 7fedf27fc700 1 -- 172.21.15.93:0/371540781 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fedf40cd860 con 0x7fedfc101e10 2023-11-16T16:46:01.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.924+0000 7fee0179b700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fede80a1150 0x7fede80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:01.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.925+0000 7fee0179b700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fede80a1150 0x7fede80a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fedfc1012e0 tx=0x7fedf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:01.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:01.928+0000 7fedf27fc700 1 -- 172.21.15.93:0/371540781 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fedf4097440 con 0x7fedfc101e10 2023-11-16T16:46:01.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:01 smithi159 ceph-mon[131528]: Deploying daemon mds.k on smithi125 2023-11-16T16:46:01.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:01 smithi159 ceph-mon[131528]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:02.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.169+0000 7fee039ff700 1 -- 172.21.15.93:0/371540781 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedfc02cfe0 con 0x7fedfc101e10 2023-11-16T16:46:02.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.170+0000 7fedf27fc700 1 -- 172.21.15.93:0/371540781 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7fedf409aa60 con 0x7fedfc101e10 2023-11-16T16:46:02.171 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:02.171 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-16T16:46:02.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.172+0000 7fee039ff700 1 -- 172.21.15.93:0/371540781 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fede80a1150 msgr2=0x7fede80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:02.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.172+0000 7fee039ff700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fede80a1150 0x7fede80a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fedfc1012e0 tx=0x7fedf8009260 comp rx=0 tx=0).stop 2023-11-16T16:46:02.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.172+0000 7fee039ff700 1 -- 172.21.15.93:0/371540781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedfc101e10 msgr2=0x7fedfc19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:02.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.172+0000 7fee039ff700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedfc101e10 0x7fedfc19d640 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fedf400d010 tx=0x7fedf400bc80 comp rx=0 tx=0).stop 2023-11-16T16:46:02.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.172+0000 7fee039ff700 1 -- 172.21.15.93:0/371540781 shutdown_connections 2023-11-16T16:46:02.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.172+0000 7fee039ff700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fede80a1150 0x7fede80a3600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:02.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.172+0000 7fee039ff700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedfc101e10 0x7fedfc19d640 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:02.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.172+0000 7fee039ff700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fedfc100290 0x7fedfc196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:02.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.172+0000 7fee039ff700 1 --2- 172.21.15.93:0/371540781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedfc101480 0x7fedfc1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:02.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.172+0000 7fee039ff700 1 -- 172.21.15.93:0/371540781 >> 172.21.15.93:0/371540781 conn(0x7fedfc0fb870 msgr2=0x7fedfc1048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:02.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.173+0000 7fee039ff700 1 -- 172.21.15.93:0/371540781 shutdown_connections 2023-11-16T16:46:02.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:02.173+0000 7fee039ff700 1 -- 172.21.15.93:0/371540781 wait complete. 2023-11-16T16:46:02.175 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 2 2023-11-16T16:46:02.479 INFO:journalctl@ceph.mds.k.smithi125.stdout:Nov 16 16:46:02 smithi125 systemd[1]: Starting Ceph mds.k for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:46:02.750 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:02 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/371540781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:02.750 INFO:journalctl@ceph.mds.k.smithi125.stdout:Nov 16 16:46:02 smithi125 podman[155508]: 2023-11-16T16:46:02.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:02 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/371540781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:02.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:02 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/371540781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:03.077 INFO:journalctl@ceph.mds.k.smithi125.stdout:Nov 16 16:46:02 smithi125 ceph-mds[155548]: -- 172.21.15.125:0/3468084169 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559e80b1d520 con 0x559e80af2800 2023-11-16T16:46:03.078 INFO:journalctl@ceph.mds.k.smithi125.stdout:Nov 16 16:46:02 smithi125 bash[155508]: 0a638153ce8e7bcdb0ffab3b8eeb5e4b16a1a39c9049ff8e05ea83800bb8dd73 2023-11-16T16:46:03.078 INFO:journalctl@ceph.mds.k.smithi125.stdout:Nov 16 16:46:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-k[155524]: starting mds.k at 2023-11-16T16:46:03.078 INFO:journalctl@ceph.mds.k.smithi125.stdout:Nov 16 16:46:02 smithi125 systemd[1]: Started Ceph mds.k for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:46:04.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:03 smithi125 ceph-mon[131076]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:04.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:04.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:04.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:04.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:04.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:04.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:04.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:03 smithi125 ceph-mon[131076]: Deploying daemon mds.j on smithi093 2023-11-16T16:46:04.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:03 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.125:6832/1299400539,v1:172.21.15.125:6833/1299400539] up:boot 2023-11-16T16:46:04.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:03 smithi125 ceph-mon[131076]: fsmap 2 up:standby 2023-11-16T16:46:04.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:03 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-16T16:46:04.148 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:03 smithi093 ceph-mon[97863]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:04.149 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:04.149 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:04.149 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:04.149 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:04.149 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:04.149 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:04.149 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:03 smithi093 ceph-mon[97863]: Deploying daemon mds.j on smithi093 2023-11-16T16:46:04.149 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:03 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.125:6832/1299400539,v1:172.21.15.125:6833/1299400539] up:boot 2023-11-16T16:46:04.149 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:03 smithi093 ceph-mon[97863]: fsmap 2 up:standby 2023-11-16T16:46:04.149 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:03 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-16T16:46:04.193 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:03 smithi159 ceph-mon[131528]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:03 smithi159 ceph-mon[131528]: Deploying daemon mds.j on smithi093 2023-11-16T16:46:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:03 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.125:6832/1299400539,v1:172.21.15.125:6833/1299400539] up:boot 2023-11-16T16:46:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:03 smithi159 ceph-mon[131528]: fsmap 2 up:standby 2023-11-16T16:46:04.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:03 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-16T16:46:04.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.395+0000 7f6e4fdff700 1 -- 172.21.15.93:0/3491798435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e481002b0 msgr2=0x7f6e481006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:04.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.395+0000 7f6e4fdff700 1 --2- 172.21.15.93:0/3491798435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e481002b0 0x7f6e481006c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009990 tx=0x7f6e44009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:04.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.395+0000 7f6e4fdff700 1 -- 172.21.15.93:0/3491798435 shutdown_connections 2023-11-16T16:46:04.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.395+0000 7f6e4fdff700 1 --2- 172.21.15.93:0/3491798435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48101e30 0x7f6e481086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:04.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.395+0000 7f6e4fdff700 1 --2- 172.21.15.93:0/3491798435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e481002b0 0x7f6e481006c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:04.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.395+0000 7f6e4fdff700 1 --2- 172.21.15.93:0/3491798435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e481014a0 0x7f6e481018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:04.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.395+0000 7f6e4fdff700 1 -- 172.21.15.93:0/3491798435 >> 172.21.15.93:0/3491798435 conn(0x7f6e480fb890 msgr2=0x7f6e480fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:04.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.395+0000 7f6e4fdff700 1 -- 172.21.15.93:0/3491798435 shutdown_connections 2023-11-16T16:46:04.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.395+0000 7f6e4fdff700 1 -- 172.21.15.93:0/3491798435 wait complete. 2023-11-16T16:46:04.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.396+0000 7f6e4fdff700 1 Processor -- start 2023-11-16T16:46:04.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.396+0000 7f6e4fdff700 1 -- start start 2023-11-16T16:46:04.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.396+0000 7f6e4fdff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e481002b0 0x7f6e48198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:04.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.396+0000 7f6e4fdff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e481014a0 0x7f6e48198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:04.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.396+0000 7f6e4fdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48101e30 0x7f6e4819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:04.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.396+0000 7f6e4fdff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4819fd90 con 0x7f6e481002b0 2023-11-16T16:46:04.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.396+0000 7f6e4fdff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4819fed0 con 0x7f6e481014a0 2023-11-16T16:46:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.396+0000 7f6e4fdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e481a0060 con 0x7f6e48101e30 2023-11-16T16:46:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4db9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e481002b0 0x7f6e48198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4db9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e481002b0 0x7f6e48198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42258/0 (socket says 172.21.15.93:42258) 2023-11-16T16:46:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4db9b700 1 -- 172.21.15.93:0/2639365921 learned_addr learned my addr 172.21.15.93:0/2639365921 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4d39a700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e481014a0 0x7f6e48198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4e39c700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48101e30 0x7f6e4819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4db9b700 1 -- 172.21.15.93:0/2639365921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48101e30 msgr2=0x7f6e4819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4db9b700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48101e30 0x7f6e4819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4db9b700 1 -- 172.21.15.93:0/2639365921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e481014a0 msgr2=0x7f6e48198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4db9b700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e481014a0 0x7f6e48198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4db9b700 1 -- 172.21.15.93:0/2639365921 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440095f0 con 0x7f6e481002b0 2023-11-16T16:46:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4d39a700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e481014a0 0x7f6e48198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4db9b700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e481002b0 0x7f6e48198210 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6e44004730 tx=0x7f6e44004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e3effd700 1 -- 172.21.15.93:0/2639365921 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4401d070 con 0x7f6e481002b0 2023-11-16T16:46:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e3effd700 1 -- 172.21.15.93:0/2639365921 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e44021410 con 0x7f6e481002b0 2023-11-16T16:46:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e4fdff700 1 -- 172.21.15.93:0/2639365921 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4810bc80 con 0x7f6e481002b0 2023-11-16T16:46:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.397+0000 7f6e3effd700 1 -- 172.21.15.93:0/2639365921 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44017650 con 0x7f6e481002b0 2023-11-16T16:46:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.398+0000 7f6e4fdff700 1 -- 172.21.15.93:0/2639365921 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4810baf0 con 0x7f6e481002b0 2023-11-16T16:46:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.399+0000 7f6e4fdff700 1 -- 172.21.15.93:0/2639365921 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e481a0660 con 0x7f6e481002b0 2023-11-16T16:46:04.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.401+0000 7f6e3effd700 1 -- 172.21.15.93:0/2639365921 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6e44021580 con 0x7f6e481002b0 2023-11-16T16:46:04.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.402+0000 7f6e3effd700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6e340a11d0 0x7f6e340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:04.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.402+0000 7f6e3effd700 1 -- 172.21.15.93:0/2639365921 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6e440cd870 con 0x7f6e481002b0 2023-11-16T16:46:04.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.402+0000 7f6e4d39a700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6e340a11d0 0x7f6e340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:04.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.403+0000 7f6e4d39a700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6e340a11d0 0x7f6e340a3680 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6e38011490 tx=0x7f6e38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:04.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.406+0000 7f6e3effd700 1 -- 172.21.15.93:0/2639365921 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6e44097450 con 0x7f6e481002b0 2023-11-16T16:46:04.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.687+0000 7f6e4fdff700 1 -- 172.21.15.93:0/2639365921 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e481a0c10 con 0x7f6e481002b0 2023-11-16T16:46:04.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.687+0000 7f6e3effd700 1 -- 172.21.15.93:0/2639365921 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f6e4409aa70 con 0x7f6e481002b0 2023-11-16T16:46:04.689 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:04.689 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-16T16:46:04.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.690+0000 7f6e4fdff700 1 -- 172.21.15.93:0/2639365921 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6e340a11d0 msgr2=0x7f6e340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:04.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.690+0000 7f6e4fdff700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6e340a11d0 0x7f6e340a3680 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6e38011490 tx=0x7f6e38009260 comp rx=0 tx=0).stop 2023-11-16T16:46:04.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.690+0000 7f6e4fdff700 1 -- 172.21.15.93:0/2639365921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e481002b0 msgr2=0x7f6e48198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:04.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.690+0000 7f6e4fdff700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e481002b0 0x7f6e48198210 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6e44004730 tx=0x7f6e44004760 comp rx=0 tx=0).stop 2023-11-16T16:46:04.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.690+0000 7f6e4fdff700 1 -- 172.21.15.93:0/2639365921 shutdown_connections 2023-11-16T16:46:04.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.690+0000 7f6e4fdff700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6e340a11d0 0x7f6e340a3680 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:04.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.690+0000 7f6e4fdff700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e481002b0 0x7f6e48198210 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:04.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.690+0000 7f6e4fdff700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48101e30 0x7f6e4819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:04.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.690+0000 7f6e4fdff700 1 --2- 172.21.15.93:0/2639365921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e481014a0 0x7f6e48198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:04.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.690+0000 7f6e4fdff700 1 -- 172.21.15.93:0/2639365921 >> 172.21.15.93:0/2639365921 conn(0x7f6e480fb890 msgr2=0x7f6e480fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:04.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.690+0000 7f6e4fdff700 1 -- 172.21.15.93:0/2639365921 shutdown_connections 2023-11-16T16:46:04.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:04.690+0000 7f6e4fdff700 1 -- 172.21.15.93:0/2639365921 wait complete. 2023-11-16T16:46:04.693 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 3 2023-11-16T16:46:04.982 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:04 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2639365921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:05.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:04 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2639365921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:05.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:04 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2639365921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:05.400 INFO:journalctl@ceph.mds.j.smithi093.stdout:Nov 16 16:46:04 smithi093 systemd[1]: Starting Ceph mds.j for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:46:05.770 INFO:journalctl@ceph.mds.j.smithi093.stdout:Nov 16 16:46:05 smithi093 podman[130872]: 2023-11-16T16:46:05.770 INFO:journalctl@ceph.mds.j.smithi093.stdout:Nov 16 16:46:05 smithi093 bash[130872]: 1ebf0b3cc28535ae0371f335f98cf1e946e6112e32797f91c681c30fd8ed8756 2023-11-16T16:46:05.770 INFO:journalctl@ceph.mds.j.smithi093.stdout:Nov 16 16:46:05 smithi093 ceph-mds[130912]: -- 172.21.15.93:0/3874267735 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fd7a16d520 con 0x55fd7a144c00 2023-11-16T16:46:05.770 INFO:journalctl@ceph.mds.j.smithi093.stdout:Nov 16 16:46:05 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-j[130888]: starting mds.j at 2023-11-16T16:46:06.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:05 smithi125 ceph-mon[131076]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:06.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:05 smithi093 ceph-mon[97863]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:06.150 INFO:journalctl@ceph.mds.j.smithi093.stdout:Nov 16 16:46:05 smithi093 systemd[1]: Started Ceph mds.j for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:46:06.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:05 smithi159 ceph-mon[131528]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:06.712 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:06.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.920+0000 7f25099ff700 1 -- 172.21.15.93:0/2023152717 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25041014a0 msgr2=0x7f25041018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:06.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.920+0000 7f25099ff700 1 --2- 172.21.15.93:0/2023152717 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25041014a0 0x7f25041018f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009990 tx=0x7f24f8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:06.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.920+0000 7f25099ff700 1 -- 172.21.15.93:0/2023152717 shutdown_connections 2023-11-16T16:46:06.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.920+0000 7f25099ff700 1 --2- 172.21.15.93:0/2023152717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2504101e30 0x7f25041086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:06.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.920+0000 7f25099ff700 1 --2- 172.21.15.93:0/2023152717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25041002b0 0x7f25041006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:06.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.920+0000 7f25099ff700 1 --2- 172.21.15.93:0/2023152717 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25041014a0 0x7f25041018f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:06.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.920+0000 7f25099ff700 1 -- 172.21.15.93:0/2023152717 >> 172.21.15.93:0/2023152717 conn(0x7f25040fb890 msgr2=0x7f25040fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:06.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.920+0000 7f25099ff700 1 -- 172.21.15.93:0/2023152717 shutdown_connections 2023-11-16T16:46:06.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.920+0000 7f25099ff700 1 -- 172.21.15.93:0/2023152717 wait complete. 2023-11-16T16:46:06.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.922+0000 7f25099ff700 1 Processor -- start 2023-11-16T16:46:06.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.922+0000 7f25099ff700 1 -- start start 2023-11-16T16:46:06.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.922+0000 7f25099ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25041002b0 0x7f2504198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:06.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25099ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25041014a0 0x7f25041986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:06.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25099ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2504101e30 0x7f250419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25099ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250419fd10 con 0x7f25041002b0 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25099ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250419fe50 con 0x7f2504101e30 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25099ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250419ffe0 con 0x7f25041014a0 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f2502ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25041002b0 0x7f2504198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f2502ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25041002b0 0x7f2504198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:55742/0 (socket says 172.21.15.93:55742) 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25037fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2504101e30 0x7f250419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f2502ffd700 1 -- 172.21.15.93:0/2147628330 learned_addr learned my addr 172.21.15.93:0/2147628330 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25027fc700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25041014a0 0x7f25041986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25037fe700 1 -- 172.21.15.93:0/2147628330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25041014a0 msgr2=0x7f25041986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25037fe700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25041014a0 0x7f25041986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25037fe700 1 -- 172.21.15.93:0/2147628330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25041002b0 msgr2=0x7f2504198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25037fe700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25041002b0 0x7f2504198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:06.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25037fe700 1 -- 172.21.15.93:0/2147628330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f80095f0 con 0x7f2504101e30 2023-11-16T16:46:06.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.923+0000 7f25027fc700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25041014a0 0x7f25041986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:06.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.924+0000 7f25037fe700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2504101e30 0x7f250419f740 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00efc0 tx=0x7f24fc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:06.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.924+0000 7f25089fd700 1 -- 172.21.15.93:0/2147628330 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc018070 con 0x7f2504101e30 2023-11-16T16:46:06.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.924+0000 7f25099ff700 1 -- 172.21.15.93:0/2147628330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250410bc60 con 0x7f2504101e30 2023-11-16T16:46:06.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.924+0000 7f25089fd700 1 -- 172.21.15.93:0/2147628330 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24fc00f040 con 0x7f2504101e30 2023-11-16T16:46:06.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.924+0000 7f25089fd700 1 -- 172.21.15.93:0/2147628330 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00cd40 con 0x7f2504101e30 2023-11-16T16:46:06.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.924+0000 7f25099ff700 1 -- 172.21.15.93:0/2147628330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250410bad0 con 0x7f2504101e30 2023-11-16T16:46:06.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.925+0000 7f25099ff700 1 -- 172.21.15.93:0/2147628330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25041928f0 con 0x7f2504101e30 2023-11-16T16:46:06.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.927+0000 7f25089fd700 1 -- 172.21.15.93:0/2147628330 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f24fc003a00 con 0x7f2504101e30 2023-11-16T16:46:06.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.928+0000 7f25089fd700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f24ec0a1150 0x7f24ec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:06.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.928+0000 7f25089fd700 1 -- 172.21.15.93:0/2147628330 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f24fc0cc0d0 con 0x7f2504101e30 2023-11-16T16:46:06.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.928+0000 7f2502ffd700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f24ec0a1150 0x7f24ec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:06.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.929+0000 7f2502ffd700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f24ec0a1150 0x7f24ec0a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f24f4005950 tx=0x7f24f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:06.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:06.932+0000 7f25089fd700 1 -- 172.21.15.93:0/2147628330 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f24fc095cb0 con 0x7f2504101e30 2023-11-16T16:46:07.106 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:07.106 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:07.106 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:07.106 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:07.106 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:07.106 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:07.106 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:06 smithi093 ceph-mon[97863]: Deploying daemon mds.d on smithi093 2023-11-16T16:46:07.106 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:06 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.93:6834/278429922,v1:172.21.15.93:6835/278429922] up:boot 2023-11-16T16:46:07.106 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:06 smithi093 ceph-mon[97863]: fsmap 3 up:standby 2023-11-16T16:46:07.107 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:06 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-16T16:46:07.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.197+0000 7f25099ff700 1 -- 172.21.15.93:0/2147628330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f250402cfe0 con 0x7f2504101e30 2023-11-16T16:46:07.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.198+0000 7f25089fd700 1 -- 172.21.15.93:0/2147628330 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7f24fc0992d0 con 0x7f2504101e30 2023-11-16T16:46:07.200 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:07.200 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-16T16:46:07.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.200+0000 7f25099ff700 1 -- 172.21.15.93:0/2147628330 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f24ec0a1150 msgr2=0x7f24ec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:07.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.200+0000 7f25099ff700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f24ec0a1150 0x7f24ec0a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f24f4005950 tx=0x7f24f4009260 comp rx=0 tx=0).stop 2023-11-16T16:46:07.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.200+0000 7f25099ff700 1 -- 172.21.15.93:0/2147628330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2504101e30 msgr2=0x7f250419f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:07.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.200+0000 7f25099ff700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2504101e30 0x7f250419f740 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00efc0 tx=0x7f24fc00c3a0 comp rx=0 tx=0).stop 2023-11-16T16:46:07.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.200+0000 7f25099ff700 1 -- 172.21.15.93:0/2147628330 shutdown_connections 2023-11-16T16:46:07.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.200+0000 7f25099ff700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f24ec0a1150 0x7f24ec0a3600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:07.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.200+0000 7f25099ff700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25041002b0 0x7f2504198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:07.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.200+0000 7f25099ff700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2504101e30 0x7f250419f740 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:07.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.200+0000 7f25099ff700 1 --2- 172.21.15.93:0/2147628330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25041014a0 0x7f25041986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:07.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.200+0000 7f25099ff700 1 -- 172.21.15.93:0/2147628330 >> 172.21.15.93:0/2147628330 conn(0x7f25040fb890 msgr2=0x7f2504106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:07.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.200+0000 7f25099ff700 1 -- 172.21.15.93:0/2147628330 shutdown_connections 2023-11-16T16:46:07.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:07.201+0000 7f25099ff700 1 -- 172.21.15.93:0/2147628330 wait complete. 2023-11-16T16:46:07.204 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 4 2023-11-16T16:46:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:06 smithi159 ceph-mon[131528]: Deploying daemon mds.d on smithi093 2023-11-16T16:46:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:06 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.93:6834/278429922,v1:172.21.15.93:6835/278429922] up:boot 2023-11-16T16:46:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:06 smithi159 ceph-mon[131528]: fsmap 3 up:standby 2023-11-16T16:46:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:06 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-16T16:46:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:06 smithi125 ceph-mon[131076]: Deploying daemon mds.d on smithi093 2023-11-16T16:46:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:06 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.93:6834/278429922,v1:172.21.15.93:6835/278429922] up:boot 2023-11-16T16:46:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:06 smithi125 ceph-mon[131076]: fsmap 3 up:standby 2023-11-16T16:46:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:06 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-16T16:46:08.119 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:07 smithi093 ceph-mon[97863]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:08.119 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:07 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2147628330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:08.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:07 smithi159 ceph-mon[131528]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:08.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:07 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2147628330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:08.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:07 smithi125 ceph-mon[131076]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:08.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:07 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2147628330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:08.400 INFO:journalctl@ceph.mds.d.smithi093.stdout:Nov 16 16:46:08 smithi093 systemd[1]: Starting Ceph mds.d for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:46:08.900 INFO:journalctl@ceph.mds.d.smithi093.stdout:Nov 16 16:46:08 smithi093 podman[131258]: 2023-11-16T16:46:08.901 INFO:journalctl@ceph.mds.d.smithi093.stdout:Nov 16 16:46:08 smithi093 bash[131258]: dc40bbcff651c2cb69f08d192d11cb87975f5e2f455e968d513274c902558335 2023-11-16T16:46:08.901 INFO:journalctl@ceph.mds.d.smithi093.stdout:Nov 16 16:46:08 smithi093 ceph-mds[131298]: -- 172.21.15.93:0/2274778275 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c65fe9b1e0 con 0x55c65fe5e800 2023-11-16T16:46:08.901 INFO:journalctl@ceph.mds.d.smithi093.stdout:Nov 16 16:46:08 smithi093 ceph-mds[131298]: -- 172.21.15.93:0/2274778275 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c65fe87520 con 0x55c65fe5e800 2023-11-16T16:46:08.901 INFO:journalctl@ceph.mds.d.smithi093.stdout:Nov 16 16:46:08 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-d[131274]: starting mds.d at 2023-11-16T16:46:09.224 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:09.287 INFO:journalctl@ceph.mds.d.smithi093.stdout:Nov 16 16:46:08 smithi093 systemd[1]: Started Ceph mds.d for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:46:09.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.423+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/943038835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1a4100290 msgr2=0x7fe1a41006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:09.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.423+0000 7fe1ac3ff700 1 --2- 172.21.15.93:0/943038835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1a4100290 0x7fe1a41006a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009990 tx=0x7fe1a0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:09.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.423+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/943038835 shutdown_connections 2023-11-16T16:46:09.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.423+0000 7fe1ac3ff700 1 --2- 172.21.15.93:0/943038835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1a4101e10 0x7fe1a41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:09.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.423+0000 7fe1ac3ff700 1 --2- 172.21.15.93:0/943038835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1a4100290 0x7fe1a41006a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:09.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.423+0000 7fe1ac3ff700 1 --2- 172.21.15.93:0/943038835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1a4101480 0x7fe1a41018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:09.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.423+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/943038835 >> 172.21.15.93:0/943038835 conn(0x7fe1a40fb870 msgr2=0x7fe1a40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:09.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.424+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/943038835 shutdown_connections 2023-11-16T16:46:09.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.424+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/943038835 wait complete. 2023-11-16T16:46:09.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.424+0000 7fe1ac3ff700 1 Processor -- start 2023-11-16T16:46:09.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.425+0000 7fe1ac3ff700 1 -- start start 2023-11-16T16:46:09.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.425+0000 7fe1ac3ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1a4100290 0x7fe1a4198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:09.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.425+0000 7fe1ac3ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1a4101480 0x7fe1a41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:09.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.425+0000 7fe1ac3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1a4101e10 0x7fe1a419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:09.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.425+0000 7fe1ac3ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a419fce0 con 0x7fe1a4100290 2023-11-16T16:46:09.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.425+0000 7fe1ac3ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a419fe20 con 0x7fe1a4101480 2023-11-16T16:46:09.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.425+0000 7fe1ac3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a419ffb0 con 0x7fe1a4101e10 2023-11-16T16:46:09.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.425+0000 7fe1aa19b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1a4100290 0x7fe1a4198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:09.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.425+0000 7fe1aa19b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1a4100290 0x7fe1a4198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:55786/0 (socket says 172.21.15.93:55786) 2023-11-16T16:46:09.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.425+0000 7fe1aa19b700 1 -- 172.21.15.93:0/2644635325 learned_addr learned my addr 172.21.15.93:0/2644635325 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:09.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.425+0000 7fe1a999a700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1a4101480 0x7fe1a41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:09.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.425+0000 7fe1aa99c700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1a4101e10 0x7fe1a419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:09.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.426+0000 7fe1aa19b700 1 -- 172.21.15.93:0/2644635325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1a4101e10 msgr2=0x7fe1a419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:09.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.426+0000 7fe1aa19b700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1a4101e10 0x7fe1a419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:09.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.426+0000 7fe1aa19b700 1 -- 172.21.15.93:0/2644635325 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1a4101480 msgr2=0x7fe1a41986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:09.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.426+0000 7fe1aa19b700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1a4101480 0x7fe1a41986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:09.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.426+0000 7fe1aa19b700 1 -- 172.21.15.93:0/2644635325 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00095f0 con 0x7fe1a4100290 2023-11-16T16:46:09.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.426+0000 7fe1aa19b700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1a4100290 0x7fe1a4198160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000b680 tx=0x7fe1a0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:09.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.426+0000 7fe19b7fe700 1 -- 172.21.15.93:0/2644635325 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a001d070 con 0x7fe1a4100290 2023-11-16T16:46:09.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.426+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/2644635325 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a410bbe0 con 0x7fe1a4100290 2023-11-16T16:46:09.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.426+0000 7fe19b7fe700 1 -- 172.21.15.93:0/2644635325 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1a0021410 con 0x7fe1a4100290 2023-11-16T16:46:09.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.426+0000 7fe19b7fe700 1 -- 172.21.15.93:0/2644635325 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0017650 con 0x7fe1a4100290 2023-11-16T16:46:09.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.426+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/2644635325 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a410ba50 con 0x7fe1a4100290 2023-11-16T16:46:09.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.427+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/2644635325 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a4192870 con 0x7fe1a4100290 2023-11-16T16:46:09.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.429+0000 7fe19b7fe700 1 -- 172.21.15.93:0/2644635325 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe1a0021580 con 0x7fe1a4100290 2023-11-16T16:46:09.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.430+0000 7fe19b7fe700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe1900a1150 0x7fe1900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:09.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.431+0000 7fe19b7fe700 1 -- 172.21.15.93:0/2644635325 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe1a00cd960 con 0x7fe1a4100290 2023-11-16T16:46:09.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.431+0000 7fe1a999a700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe1900a1150 0x7fe1900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:09.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.431+0000 7fe1a999a700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe1900a1150 0x7fe1900a3600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe194007c60 tx=0x7fe1940073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:09.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.435+0000 7fe19b7fe700 1 -- 172.21.15.93:0/2644635325 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe1a0097540 con 0x7fe1a4100290 2023-11-16T16:46:09.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.674+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/2644635325 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1a402cfe0 con 0x7fe1a4100290 2023-11-16T16:46:09.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.675+0000 7fe19b7fe700 1 -- 172.21.15.93:0/2644635325 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7fe1a009ab60 con 0x7fe1a4100290 2023-11-16T16:46:09.677 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:09.677 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-16T16:46:09.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.677+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/2644635325 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe1900a1150 msgr2=0x7fe1900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:09.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.677+0000 7fe1ac3ff700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe1900a1150 0x7fe1900a3600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe194007c60 tx=0x7fe1940073d0 comp rx=0 tx=0).stop 2023-11-16T16:46:09.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.677+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/2644635325 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1a4100290 msgr2=0x7fe1a4198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:09.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.677+0000 7fe1ac3ff700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1a4100290 0x7fe1a4198160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000b680 tx=0x7fe1a0005740 comp rx=0 tx=0).stop 2023-11-16T16:46:09.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.677+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/2644635325 shutdown_connections 2023-11-16T16:46:09.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.677+0000 7fe1ac3ff700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe1900a1150 0x7fe1900a3600 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:09.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.677+0000 7fe1ac3ff700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1a4100290 0x7fe1a4198160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:09.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.677+0000 7fe1ac3ff700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1a4101e10 0x7fe1a419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:09.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.677+0000 7fe1ac3ff700 1 --2- 172.21.15.93:0/2644635325 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1a4101480 0x7fe1a41986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:09.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.677+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/2644635325 >> 172.21.15.93:0/2644635325 conn(0x7fe1a40fb870 msgr2=0x7fe1a4106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:09.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.677+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/2644635325 shutdown_connections 2023-11-16T16:46:09.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:09.678+0000 7fe1ac3ff700 1 -- 172.21.15.93:0/2644635325 wait complete. 2023-11-16T16:46:09.680 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 4 2023-11-16T16:46:10.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:09 smithi159 ceph-mon[131528]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:10.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:10.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:10.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:10.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:10.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:10.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:10.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:09 smithi159 ceph-mon[131528]: Deploying daemon mds.e on smithi125 2023-11-16T16:46:10.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:09 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2644635325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:10.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:09 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.93:6836/3647861869,v1:172.21.15.93:6837/3647861869] up:boot 2023-11-16T16:46:10.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:09 smithi159 ceph-mon[131528]: fsmap 4 up:standby 2023-11-16T16:46:10.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:09 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-16T16:46:10.301 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:09 smithi125 ceph-mon[131076]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:10.301 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:10.301 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:10.301 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:10.301 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:10.301 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:10.301 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:10.301 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:09 smithi125 ceph-mon[131076]: Deploying daemon mds.e on smithi125 2023-11-16T16:46:10.301 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:09 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2644635325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:10.301 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:09 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.93:6836/3647861869,v1:172.21.15.93:6837/3647861869] up:boot 2023-11-16T16:46:10.301 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:09 smithi125 ceph-mon[131076]: fsmap 4 up:standby 2023-11-16T16:46:10.302 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:09 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-16T16:46:10.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:09 smithi093 ceph-mon[97863]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:10.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:10.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:10.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:10.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:10.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:10.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:10.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:09 smithi093 ceph-mon[97863]: Deploying daemon mds.e on smithi125 2023-11-16T16:46:10.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:09 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2644635325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:10.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:09 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.93:6836/3647861869,v1:172.21.15.93:6837/3647861869] up:boot 2023-11-16T16:46:10.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:09 smithi093 ceph-mon[97863]: fsmap 4 up:standby 2023-11-16T16:46:10.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:09 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-16T16:46:11.078 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:46:10 smithi125 systemd[1]: Starting Ceph mds.e for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:46:11.455 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:11 smithi125 ceph-mon[131076]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:11.455 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:46:11 smithi125 podman[155849]: 2023-11-16T16:46:11.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:11 smithi093 ceph-mon[97863]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:11.699 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:11.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:11 smithi159 ceph-mon[131528]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:11.828 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:46:11 smithi125 ceph-mds[155889]: -- 172.21.15.125:0/3360107843 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56298356b520 con 0x562983541000 2023-11-16T16:46:11.828 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:46:11 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-e[155865]: starting mds.e at 2023-11-16T16:46:11.828 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:46:11 smithi125 bash[155849]: 410e537278fa3982ce872e621d75f9a5602c9112841a554c28bfb5e485a7be7d 2023-11-16T16:46:11.828 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:46:11 smithi125 systemd[1]: Started Ceph mds.e for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:46:11.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.909+0000 7f9de7fff700 1 -- 172.21.15.93:0/4200489665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de01036b0 msgr2=0x7f9de0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:11.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.909+0000 7f9de7fff700 1 --2- 172.21.15.93:0/4200489665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de01036b0 0x7f9de0105a90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009990 tx=0x7f9dd0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:11.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.909+0000 7f9de7fff700 1 -- 172.21.15.93:0/4200489665 shutdown_connections 2023-11-16T16:46:11.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.909+0000 7f9de7fff700 1 --2- 172.21.15.93:0/4200489665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de0105fd0 0x7f9de01083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:11.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.909+0000 7f9de7fff700 1 --2- 172.21.15.93:0/4200489665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9de0100d90 0x7f9de0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:11.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.909+0000 7f9de7fff700 1 --2- 172.21.15.93:0/4200489665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de01036b0 0x7f9de0105a90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:11.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.909+0000 7f9de7fff700 1 -- 172.21.15.93:0/4200489665 >> 172.21.15.93:0/4200489665 conn(0x7f9de00fa750 msgr2=0x7f9de00fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:11.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.909+0000 7f9de7fff700 1 -- 172.21.15.93:0/4200489665 shutdown_connections 2023-11-16T16:46:11.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.909+0000 7f9de7fff700 1 -- 172.21.15.93:0/4200489665 wait complete. 2023-11-16T16:46:11.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.910+0000 7f9de7fff700 1 Processor -- start 2023-11-16T16:46:11.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.910+0000 7f9de7fff700 1 -- start start 2023-11-16T16:46:11.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de7fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9de0100d90 0x7f9de01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:11.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de01036b0 0x7f9de0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:11.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de0105fd0 0x7f9de019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:11.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de7fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de019fd40 con 0x7f9de0105fd0 2023-11-16T16:46:11.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de7fff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de019fe80 con 0x7f9de0100d90 2023-11-16T16:46:11.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de7fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de01a0010 con 0x7f9de01036b0 2023-11-16T16:46:11.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de559a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de01036b0 0x7f9de0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:11.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de0105fd0 0x7f9de019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:11.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de559a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de01036b0 0x7f9de0198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:53056/0 (socket says 172.21.15.93:53056) 2023-11-16T16:46:11.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de559a700 1 -- 172.21.15.93:0/2931642364 learned_addr learned my addr 172.21.15.93:0/2931642364 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:11.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de5d9b700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9de0100d90 0x7f9de01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:11.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de659c700 1 -- 172.21.15.93:0/2931642364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de01036b0 msgr2=0x7f9de0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:11.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de659c700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de01036b0 0x7f9de0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:11.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de659c700 1 -- 172.21.15.93:0/2931642364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9de0100d90 msgr2=0x7f9de01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:11.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de659c700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9de0100d90 0x7f9de01981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:11.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de659c700 1 -- 172.21.15.93:0/2931642364 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd00095f0 con 0x7f9de0105fd0 2023-11-16T16:46:11.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.911+0000 7f9de559a700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de01036b0 0x7f9de0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:11.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.912+0000 7f9de659c700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de0105fd0 0x7f9de019f770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800efc0 tx=0x7f9dd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:11.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.912+0000 7f9dd6ffd700 1 -- 172.21.15.93:0/2931642364 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8018070 con 0x7f9de0105fd0 2023-11-16T16:46:11.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.912+0000 7f9dd6ffd700 1 -- 172.21.15.93:0/2931642364 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dd800f040 con 0x7f9de0105fd0 2023-11-16T16:46:11.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.912+0000 7f9de7fff700 1 -- 172.21.15.93:0/2931642364 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de010bc60 con 0x7f9de0105fd0 2023-11-16T16:46:11.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.912+0000 7f9dd6ffd700 1 -- 172.21.15.93:0/2931642364 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd800cd00 con 0x7f9de0105fd0 2023-11-16T16:46:11.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.912+0000 7f9de7fff700 1 -- 172.21.15.93:0/2931642364 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de010bad0 con 0x7f9de0105fd0 2023-11-16T16:46:11.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.913+0000 7f9de7fff700 1 -- 172.21.15.93:0/2931642364 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de0107b40 con 0x7f9de0105fd0 2023-11-16T16:46:11.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.915+0000 7f9dd6ffd700 1 -- 172.21.15.93:0/2931642364 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9dd8003a00 con 0x7f9de0105fd0 2023-11-16T16:46:11.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.917+0000 7f9dd6ffd700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9dcc0a1150 0x7f9dcc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:11.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.917+0000 7f9dd6ffd700 1 -- 172.21.15.93:0/2931642364 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9dd80cc940 con 0x7f9de0105fd0 2023-11-16T16:46:11.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.917+0000 7f9de5d9b700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9dcc0a1150 0x7f9dcc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:11.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.917+0000 7f9de5d9b700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9dcc0a1150 0x7f9dcc0a3600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9de0068890 tx=0x7f9ddc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:11.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:11.919+0000 7f9dd6ffd700 1 -- 172.21.15.93:0/2931642364 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9dd8096520 con 0x7f9de0105fd0 2023-11-16T16:46:12.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.159+0000 7f9de7fff700 1 -- 172.21.15.93:0/2931642364 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9de002cfe0 con 0x7f9de0105fd0 2023-11-16T16:46:12.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.159+0000 7f9dd6ffd700 1 -- 172.21.15.93:0/2931642364 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3531 (secure 0 0 0) 0x7f9dd800ce60 con 0x7f9de0105fd0 2023-11-16T16:46:12.161 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:12.161 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-16T16:46:12.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.161+0000 7f9de7fff700 1 -- 172.21.15.93:0/2931642364 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9dcc0a1150 msgr2=0x7f9dcc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:12.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.161+0000 7f9de7fff700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9dcc0a1150 0x7f9dcc0a3600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9de0068890 tx=0x7f9ddc009260 comp rx=0 tx=0).stop 2023-11-16T16:46:12.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.161+0000 7f9de7fff700 1 -- 172.21.15.93:0/2931642364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de0105fd0 msgr2=0x7f9de019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:12.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.161+0000 7f9de7fff700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de0105fd0 0x7f9de019f770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800efc0 tx=0x7f9dd800c3a0 comp rx=0 tx=0).stop 2023-11-16T16:46:12.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.161+0000 7f9de7fff700 1 -- 172.21.15.93:0/2931642364 shutdown_connections 2023-11-16T16:46:12.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.161+0000 7f9de7fff700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9dcc0a1150 0x7f9dcc0a3600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:12.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.161+0000 7f9de7fff700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de0105fd0 0x7f9de019f770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:12.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.161+0000 7f9de7fff700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9de0100d90 0x7f9de01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:12.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.161+0000 7f9de7fff700 1 --2- 172.21.15.93:0/2931642364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9de01036b0 0x7f9de0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:12.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.161+0000 7f9de7fff700 1 -- 172.21.15.93:0/2931642364 >> 172.21.15.93:0/2931642364 conn(0x7f9de00fa750 msgr2=0x7f9de00ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:12.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.162+0000 7f9de7fff700 1 -- 172.21.15.93:0/2931642364 shutdown_connections 2023-11-16T16:46:12.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:12.162+0000 7f9de7fff700 1 -- 172.21.15.93:0/2931642364 wait complete. 2023-11-16T16:46:12.165 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 5 2023-11-16T16:46:12.697 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:12.697 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:12.697 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:12.697 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:12.697 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:12.697 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:12.697 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:12 smithi159 ceph-mon[131528]: Deploying daemon mds.f on smithi159 2023-11-16T16:46:12.697 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:12 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2931642364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:12.697 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:12 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.125:6834/2117404468,v1:172.21.15.125:6835/2117404468] up:boot 2023-11-16T16:46:12.698 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:12 smithi159 ceph-mon[131528]: fsmap 5 up:standby 2023-11-16T16:46:12.698 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:12 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:46:12.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:12.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:12.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:12.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:12.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:12.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:12.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:12 smithi125 ceph-mon[131076]: Deploying daemon mds.f on smithi159 2023-11-16T16:46:12.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:12 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2931642364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:12.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:12 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.125:6834/2117404468,v1:172.21.15.125:6835/2117404468] up:boot 2023-11-16T16:46:12.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:12 smithi125 ceph-mon[131076]: fsmap 5 up:standby 2023-11-16T16:46:12.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:12 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:46:12.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:12 smithi093 ceph-mon[97863]: Deploying daemon mds.f on smithi159 2023-11-16T16:46:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:12 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2931642364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:12 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.125:6834/2117404468,v1:172.21.15.125:6835/2117404468] up:boot 2023-11-16T16:46:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:12 smithi093 ceph-mon[97863]: fsmap 5 up:standby 2023-11-16T16:46:12.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:12 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:46:13.235 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:46:13 smithi159 systemd[1]: Starting Ceph mds.f for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:46:13.771 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:13 smithi159 ceph-mon[131528]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:13.772 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:46:13 smithi159 podman[156958]: 2023-11-16T16:46:13.772 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:46:13 smithi159 ceph-mds[156998]: -- 172.21.15.159:0/2962003639 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f91434d520 con 0x55f914322c00 2023-11-16T16:46:13.772 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:46:13 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-f[156974]: starting mds.f at 2023-11-16T16:46:13.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:13 smithi125 ceph-mon[131076]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:13 smithi093 ceph-mon[97863]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:14.183 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:14.235 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:46:13 smithi159 bash[156958]: 5ec490df026fdbf5c9a8ec27cf264ed3dd5ce3b3a3830e4b26981714398a08cc 2023-11-16T16:46:14.235 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:46:13 smithi159 systemd[1]: Started Ceph mds.f for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:46:14.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.387+0000 7f4faefff700 1 -- 172.21.15.93:0/707921436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fa8074b30 msgr2=0x7f4fa8072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.387+0000 7f4faefff700 1 --2- 172.21.15.93:0/707921436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fa8074b30 0x7f4fa8072f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009990 tx=0x7f4f9c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.388+0000 7f4faefff700 1 -- 172.21.15.93:0/707921436 shutdown_connections 2023-11-16T16:46:14.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.388+0000 7f4faefff700 1 --2- 172.21.15.93:0/707921436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fa8074b30 0x7f4fa8072f80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.388+0000 7f4faefff700 1 --2- 172.21.15.93:0/707921436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fa810c860 0x7f4fa810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.388+0000 7f4faefff700 1 --2- 172.21.15.93:0/707921436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fa8073550 0x7f4fa80739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.388+0000 7f4faefff700 1 -- 172.21.15.93:0/707921436 >> 172.21.15.93:0/707921436 conn(0x7f4fa80fb870 msgr2=0x7f4fa80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.388+0000 7f4faefff700 1 -- 172.21.15.93:0/707921436 shutdown_connections 2023-11-16T16:46:14.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.388+0000 7f4faefff700 1 -- 172.21.15.93:0/707921436 wait complete. 2023-11-16T16:46:14.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4faefff700 1 Processor -- start 2023-11-16T16:46:14.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4faefff700 1 -- start start 2023-11-16T16:46:14.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4faefff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fa8073550 0x7f4fa819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4faefff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fa8074b30 0x7f4fa819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4faefff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fa810c860 0x7f4fa81a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4faefff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa81a4180 con 0x7f4fa8073550 2023-11-16T16:46:14.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4faefff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa81a42c0 con 0x7f4fa8074b30 2023-11-16T16:46:14.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4faefff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa81a4450 con 0x7f4fa810c860 2023-11-16T16:46:14.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4facd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fa8073550 0x7f4fa819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4facd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fa8073550 0x7f4fa819c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:55824/0 (socket says 172.21.15.93:55824) 2023-11-16T16:46:14.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4facd9b700 1 -- 172.21.15.93:0/3644368412 learned_addr learned my addr 172.21.15.93:0/3644368412 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:14.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4fa7fff700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fa8074b30 0x7f4fa819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.389+0000 7f4fad59c700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fa810c860 0x7f4fa81a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.390+0000 7f4facd9b700 1 -- 172.21.15.93:0/3644368412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fa810c860 msgr2=0x7f4fa81a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.390+0000 7f4facd9b700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fa810c860 0x7f4fa81a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.390+0000 7f4facd9b700 1 -- 172.21.15.93:0/3644368412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fa8074b30 msgr2=0x7f4fa819cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.390+0000 7f4facd9b700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fa8074b30 0x7f4fa819cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.390+0000 7f4facd9b700 1 -- 172.21.15.93:0/3644368412 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa0009c40 con 0x7f4fa8073550 2023-11-16T16:46:14.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.390+0000 7f4facd9b700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fa8073550 0x7f4fa819c600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c005720 tx=0x7f4f9c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.390+0000 7f4fa5ffb700 1 -- 172.21.15.93:0/3644368412 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c01d070 con 0x7f4fa8073550 2023-11-16T16:46:14.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.390+0000 7f4fa5ffb700 1 -- 172.21.15.93:0/3644368412 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f9c021410 con 0x7f4fa8073550 2023-11-16T16:46:14.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.390+0000 7f4faefff700 1 -- 172.21.15.93:0/3644368412 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c0095f0 con 0x7f4fa8073550 2023-11-16T16:46:14.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.390+0000 7f4fa5ffb700 1 -- 172.21.15.93:0/3644368412 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c017650 con 0x7f4fa8073550 2023-11-16T16:46:14.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.390+0000 7f4faefff700 1 -- 172.21.15.93:0/3644368412 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa8112250 con 0x7f4fa8073550 2023-11-16T16:46:14.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.391+0000 7f4faefff700 1 -- 172.21.15.93:0/3644368412 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa81023b0 con 0x7f4fa8073550 2023-11-16T16:46:14.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.393+0000 7f4fa5ffb700 1 -- 172.21.15.93:0/3644368412 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4f9c021580 con 0x7f4fa8073550 2023-11-16T16:46:14.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.394+0000 7f4fa5ffb700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4f980a1150 0x7f4f980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.395+0000 7f4fa5ffb700 1 -- 172.21.15.93:0/3644368412 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4f9c0cd680 con 0x7f4fa8073550 2023-11-16T16:46:14.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.395+0000 7f4fa7fff700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4f980a1150 0x7f4f980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.395+0000 7f4fa7fff700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4f980a1150 0x7f4f980a3600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8074850 tx=0x7f4f94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.398+0000 7f4fa5ffb700 1 -- 172.21.15.93:0/3644368412 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4f9c097380 con 0x7f4fa8073550 2023-11-16T16:46:14.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.639+0000 7f4faefff700 1 -- 172.21.15.93:0/3644368412 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fa802cfe0 con 0x7f4fa8073550 2023-11-16T16:46:14.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.640+0000 7f4fa5ffb700 1 -- 172.21.15.93:0/3644368412 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5051 (secure 0 0 0) 0x7f4f9c09a9a0 con 0x7f4fa8073550 2023-11-16T16:46:14.641 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:14.642 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-11-16T16:46:14.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.642+0000 7f4faefff700 1 -- 172.21.15.93:0/3644368412 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4f980a1150 msgr2=0x7f4f980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.642+0000 7f4faefff700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4f980a1150 0x7f4f980a3600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8074850 tx=0x7f4f94009260 comp rx=0 tx=0).stop 2023-11-16T16:46:14.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.642+0000 7f4faefff700 1 -- 172.21.15.93:0/3644368412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fa8073550 msgr2=0x7f4fa819c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.642+0000 7f4faefff700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fa8073550 0x7f4fa819c600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c005720 tx=0x7f4f9c009580 comp rx=0 tx=0).stop 2023-11-16T16:46:14.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.642+0000 7f4faefff700 1 -- 172.21.15.93:0/3644368412 shutdown_connections 2023-11-16T16:46:14.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.642+0000 7f4faefff700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4f980a1150 0x7f4f980a3600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.642+0000 7f4faefff700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fa8073550 0x7f4fa819c600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.642+0000 7f4faefff700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fa810c860 0x7f4fa81a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.642+0000 7f4faefff700 1 --2- 172.21.15.93:0/3644368412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fa8074b30 0x7f4fa819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.642+0000 7f4faefff700 1 -- 172.21.15.93:0/3644368412 >> 172.21.15.93:0/3644368412 conn(0x7f4fa80fb870 msgr2=0x7f4fa80fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.643+0000 7f4faefff700 1 -- 172.21.15.93:0/3644368412 shutdown_connections 2023-11-16T16:46:14.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:14.643+0000 7f4faefff700 1 -- 172.21.15.93:0/3644368412 wait complete. 2023-11-16T16:46:14.645 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 7 2023-11-16T16:46:14.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:14 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:14.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:14 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:14.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:14 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:14.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:14 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:14.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:14 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:14.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:14 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:14.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:14 smithi159 ceph-mon[131528]: Deploying daemon mds.c on smithi159 2023-11-16T16:46:14.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:14 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.159:6834/3072074959,v1:172.21.15.159:6835/3072074959] up:boot 2023-11-16T16:46:14.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:14 smithi159 ceph-mon[131528]: fsmap 6 up:standby 2023-11-16T16:46:14.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:14 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:46:14.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:14 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3644368412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:15.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:14 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:15.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:14 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:15.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:14 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:15.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:14 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:15.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:14 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:15.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:14 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:15.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:14 smithi093 ceph-mon[97863]: Deploying daemon mds.c on smithi159 2023-11-16T16:46:15.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:14 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.159:6834/3072074959,v1:172.21.15.159:6835/3072074959] up:boot 2023-11-16T16:46:15.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:14 smithi093 ceph-mon[97863]: fsmap 6 up:standby 2023-11-16T16:46:15.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:14 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:46:15.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:14 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3644368412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:15.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:14 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:15.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:14 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:15.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:14 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:15.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:14 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:15.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:14 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:15.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:14 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:15.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:14 smithi125 ceph-mon[131076]: Deploying daemon mds.c on smithi159 2023-11-16T16:46:15.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:14 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.159:6834/3072074959,v1:172.21.15.159:6835/3072074959] up:boot 2023-11-16T16:46:15.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:14 smithi125 ceph-mon[131076]: fsmap 6 up:standby 2023-11-16T16:46:15.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:14 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:46:15.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:14 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3644368412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:15.784 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:46:15 smithi159 systemd[1]: Starting Ceph mds.c for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:46:15.784 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:46:15 smithi159 podman[157300]: 2023-11-16T16:46:16.094 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:15 smithi159 ceph-mon[131528]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:16.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:15 smithi093 ceph-mon[97863]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:16.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:15 smithi125 ceph-mon[131076]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:16.485 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:46:16 smithi159 ceph-mds[157340]: -- 172.21.15.159:0/1500292561 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dda2589520 con 0x55dda255ec00 2023-11-16T16:46:16.485 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:46:16 smithi159 bash[157300]: c38354ce22645341533ac02861c422e0094bb722ab025f329a87de3086644907 2023-11-16T16:46:16.485 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:46:16 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c[157316]: starting mds.c at 2023-11-16T16:46:16.485 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:46:16 smithi159 systemd[1]: Started Ceph mds.c for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:46:16.664 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:16.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.881+0000 7f0976bff700 1 -- 172.21.15.93:0/1727413007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0970101480 msgr2=0x7f09701018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.881+0000 7f0976bff700 1 --2- 172.21.15.93:0/1727413007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0970101480 0x7f09701018d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f095c009990 tx=0x7f095c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.881+0000 7f0976bff700 1 -- 172.21.15.93:0/1727413007 shutdown_connections 2023-11-16T16:46:16.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.881+0000 7f0976bff700 1 --2- 172.21.15.93:0/1727413007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0970101e10 0x7f09701086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.881+0000 7f0976bff700 1 --2- 172.21.15.93:0/1727413007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0970100290 0x7f09701006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.881+0000 7f0976bff700 1 --2- 172.21.15.93:0/1727413007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0970101480 0x7f09701018d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.881+0000 7f0976bff700 1 -- 172.21.15.93:0/1727413007 >> 172.21.15.93:0/1727413007 conn(0x7f09700fb870 msgr2=0x7f09700fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.881+0000 7f0976bff700 1 -- 172.21.15.93:0/1727413007 shutdown_connections 2023-11-16T16:46:16.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.881+0000 7f0976bff700 1 -- 172.21.15.93:0/1727413007 wait complete. 2023-11-16T16:46:16.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.882+0000 7f0976bff700 1 Processor -- start 2023-11-16T16:46:16.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.882+0000 7f0976bff700 1 -- start start 2023-11-16T16:46:16.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f0976bff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0970100290 0x7f09701981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f0976bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0970101480 0x7f0970198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f0976bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0970101e10 0x7f097019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f0976bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097019fd40 con 0x7f0970101480 2023-11-16T16:46:16.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f0976bff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097019fe80 con 0x7f0970100290 2023-11-16T16:46:16.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f0976bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09701a0010 con 0x7f0970101e10 2023-11-16T16:46:16.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f096ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0970101480 0x7f0970198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f096ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0970101480 0x7f0970198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49462/0 (socket says 172.21.15.93:49462) 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f096ffff700 1 -- 172.21.15.93:0/3020366271 learned_addr learned my addr 172.21.15.93:0/3020366271 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f097499b700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0970100290 0x7f09701981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f097519c700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0970101e10 0x7f097019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f096ffff700 1 -- 172.21.15.93:0/3020366271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0970101e10 msgr2=0x7f097019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f096ffff700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0970101e10 0x7f097019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f096ffff700 1 -- 172.21.15.93:0/3020366271 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0970100290 msgr2=0x7f09701981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f096ffff700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0970100290 0x7f09701981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.883+0000 7f096ffff700 1 -- 172.21.15.93:0/3020366271 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0964009520 con 0x7f0970101480 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.884+0000 7f096ffff700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0970101480 0x7f0970198700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f095c00b620 tx=0x7f095c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.884+0000 7f096dffb700 1 -- 172.21.15.93:0/3020366271 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c01d070 con 0x7f0970101480 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.884+0000 7f096dffb700 1 -- 172.21.15.93:0/3020366271 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f095c022470 con 0x7f0970101480 2023-11-16T16:46:16.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.884+0000 7f096dffb700 1 -- 172.21.15.93:0/3020366271 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c017650 con 0x7f0970101480 2023-11-16T16:46:16.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.884+0000 7f0976bff700 1 -- 172.21.15.93:0/3020366271 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0968009c40 con 0x7f0970101480 2023-11-16T16:46:16.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.884+0000 7f0976bff700 1 -- 172.21.15.93:0/3020366271 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c0095f0 con 0x7f0970101480 2023-11-16T16:46:16.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.885+0000 7f0976bff700 1 -- 172.21.15.93:0/3020366271 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097010bac0 con 0x7f0970101480 2023-11-16T16:46:16.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.887+0000 7f096dffb700 1 -- 172.21.15.93:0/3020366271 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f095c0225e0 con 0x7f0970101480 2023-11-16T16:46:16.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.888+0000 7f096dffb700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f09600a1150 0x7f09600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.888+0000 7f096dffb700 1 -- 172.21.15.93:0/3020366271 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f095c0ce140 con 0x7f0970101480 2023-11-16T16:46:16.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.888+0000 7f097499b700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f09600a1150 0x7f09600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.889+0000 7f097499b700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f09600a1150 0x7f09600a3600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f09701012e0 tx=0x7f0964009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:16.890+0000 7f096dffb700 1 -- 172.21.15.93:0/3020366271 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f095c097d20 con 0x7f0970101480 2023-11-16T16:46:17.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.132+0000 7f0976bff700 1 -- 172.21.15.93:0/3020366271 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09701a0a40 con 0x7f0970101480 2023-11-16T16:46:17.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.132+0000 7f096dffb700 1 -- 172.21.15.93:0/3020366271 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5808 (secure 0 0 0) 0x7f095c09b340 con 0x7f0970101480 2023-11-16T16:46:17.134 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:17.134 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/778982589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":778982589},{"type":"v1","addr":"172.21.15.159:6837","nonce":778982589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-11-16T16:46:17.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.135+0000 7f0976bff700 1 -- 172.21.15.93:0/3020366271 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f09600a1150 msgr2=0x7f09600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:17.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.135+0000 7f0976bff700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f09600a1150 0x7f09600a3600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f09701012e0 tx=0x7f0964009260 comp rx=0 tx=0).stop 2023-11-16T16:46:17.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.135+0000 7f0976bff700 1 -- 172.21.15.93:0/3020366271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0970101480 msgr2=0x7f0970198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:17.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.135+0000 7f0976bff700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0970101480 0x7f0970198700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f095c00b620 tx=0x7f095c005740 comp rx=0 tx=0).stop 2023-11-16T16:46:17.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.135+0000 7f0976bff700 1 -- 172.21.15.93:0/3020366271 shutdown_connections 2023-11-16T16:46:17.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.135+0000 7f0976bff700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f09600a1150 0x7f09600a3600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:17.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.135+0000 7f0976bff700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0970101e10 0x7f097019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:17.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.135+0000 7f0976bff700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0970100290 0x7f09701981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:17.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.135+0000 7f0976bff700 1 --2- 172.21.15.93:0/3020366271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0970101480 0x7f0970198700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:17.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.135+0000 7f0976bff700 1 -- 172.21.15.93:0/3020366271 >> 172.21.15.93:0/3020366271 conn(0x7f09700fb870 msgr2=0x7f09700fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:17.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.135+0000 7f0976bff700 1 -- 172.21.15.93:0/3020366271 shutdown_connections 2023-11-16T16:46:17.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:17.135+0000 7f0976bff700 1 -- 172.21.15.93:0/3020366271 wait complete. 2023-11-16T16:46:17.138 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 8 2023-11-16T16:46:17.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:17.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:17.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:17.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:17.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:17.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:17.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:17 smithi125 ceph-mon[131076]: Deploying daemon mds.h on smithi125 2023-11-16T16:46:17.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:17 smithi125 ceph-mon[131076]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:17.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:17 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.159:6836/778982589,v1:172.21.15.159:6837/778982589] up:boot 2023-11-16T16:46:17.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:17 smithi125 ceph-mon[131076]: fsmap 7 up:standby 2023-11-16T16:46:17.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:46:17.329 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:17 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3020366271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:17.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:17.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:17.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:17.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:17.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:17.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:17.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:17 smithi159 ceph-mon[131528]: Deploying daemon mds.h on smithi125 2023-11-16T16:46:17.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:17 smithi159 ceph-mon[131528]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:17.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:17 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.159:6836/778982589,v1:172.21.15.159:6837/778982589] up:boot 2023-11-16T16:46:17.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:17 smithi159 ceph-mon[131528]: fsmap 7 up:standby 2023-11-16T16:46:17.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:46:17.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:17 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3020366271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:17.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:17.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:17.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:17.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:17 smithi093 ceph-mon[97863]: Deploying daemon mds.h on smithi125 2023-11-16T16:46:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:17 smithi093 ceph-mon[97863]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:17 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.159:6836/778982589,v1:172.21.15.159:6837/778982589] up:boot 2023-11-16T16:46:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:17 smithi093 ceph-mon[97863]: fsmap 7 up:standby 2023-11-16T16:46:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:46:17.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:17 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3020366271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:18.078 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:46:17 smithi125 systemd[1]: Starting Ceph mds.h for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:46:18.472 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:46:18 smithi125 podman[156186]: 2023-11-16T16:46:18.827 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:46:18 smithi125 ceph-mds[156226]: -- 172.21.15.125:0/3152708589 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a789961520 con 0x55a789936800 2023-11-16T16:46:18.828 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:46:18 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-h[156202]: starting mds.h at 2023-11-16T16:46:18.828 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:46:18 smithi125 bash[156186]: 8cf7b457af4a604b71a5b1f09eb044a54c0e62236eeec0fa5abdeb424d5dda42 2023-11-16T16:46:18.828 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:46:18 smithi125 systemd[1]: Started Ceph mds.h for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:46:19.157 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:19.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.373+0000 7fd8c5bff700 1 -- 172.21.15.93:0/4286097575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c0101e10 msgr2=0x7fd8c01086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:19.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.373+0000 7fd8c5bff700 1 --2- 172.21.15.93:0/4286097575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c0101e10 0x7fd8c01086d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009f90 tx=0x7fd8b800b330 comp rx=0 tx=0).stop 2023-11-16T16:46:19.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.373+0000 7fd8c5bff700 1 -- 172.21.15.93:0/4286097575 shutdown_connections 2023-11-16T16:46:19.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.373+0000 7fd8c5bff700 1 --2- 172.21.15.93:0/4286097575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c0101e10 0x7fd8c01086d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:19.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.373+0000 7fd8c5bff700 1 --2- 172.21.15.93:0/4286097575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8c0100290 0x7fd8c01006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:19.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.373+0000 7fd8c5bff700 1 --2- 172.21.15.93:0/4286097575 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8c0101480 0x7fd8c01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:19.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.373+0000 7fd8c5bff700 1 -- 172.21.15.93:0/4286097575 >> 172.21.15.93:0/4286097575 conn(0x7fd8c00fb870 msgr2=0x7fd8c00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:19.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.373+0000 7fd8c5bff700 1 -- 172.21.15.93:0/4286097575 shutdown_connections 2023-11-16T16:46:19.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.373+0000 7fd8c5bff700 1 -- 172.21.15.93:0/4286097575 wait complete. 2023-11-16T16:46:19.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.375+0000 7fd8c5bff700 1 Processor -- start 2023-11-16T16:46:19.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.375+0000 7fd8c5bff700 1 -- start start 2023-11-16T16:46:19.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.375+0000 7fd8c5bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c0100290 0x7fd8c0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:19.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.375+0000 7fd8c5bff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8c0101480 0x7fd8c01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:19.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.375+0000 7fd8c5bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8c0101e10 0x7fd8c019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:19.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.375+0000 7fd8c5bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c019fe10 con 0x7fd8c0100290 2023-11-16T16:46:19.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.375+0000 7fd8c5bff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c019ff50 con 0x7fd8c0101480 2023-11-16T16:46:19.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.375+0000 7fd8c5bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01a00e0 con 0x7fd8c0101e10 2023-11-16T16:46:19.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8bf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c0100290 0x7fd8c0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:19.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8bf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c0100290 0x7fd8c0198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49488/0 (socket says 172.21.15.93:49488) 2023-11-16T16:46:19.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8bf7fe700 1 -- 172.21.15.93:0/3925575075 learned_addr learned my addr 172.21.15.93:0/3925575075 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:19.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8beffd700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8c0101480 0x7fd8c01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:19.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8bffff700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8c0101e10 0x7fd8c019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:19.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8bf7fe700 1 -- 172.21.15.93:0/3925575075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8c0101e10 msgr2=0x7fd8c019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:19.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8bf7fe700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8c0101e10 0x7fd8c019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:19.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8bf7fe700 1 -- 172.21.15.93:0/3925575075 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8c0101480 msgr2=0x7fd8c01987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:19.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8bf7fe700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8c0101480 0x7fd8c01987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:19.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8bf7fe700 1 -- 172.21.15.93:0/3925575075 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b8009c40 con 0x7fd8c0100290 2023-11-16T16:46:19.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8bffff700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8c0101e10 0x7fd8c019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:19.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8bf7fe700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c0100290 0x7fd8c0198290 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000e970 tx=0x7fd8b000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:19.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.376+0000 7fd8bcff9700 1 -- 172.21.15.93:0/3925575075 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0009790 con 0x7fd8c0100290 2023-11-16T16:46:19.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.377+0000 7fd8bcff9700 1 -- 172.21.15.93:0/3925575075 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8b000c7c0 con 0x7fd8c0100290 2023-11-16T16:46:19.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.377+0000 7fd8c5bff700 1 -- 172.21.15.93:0/3925575075 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c010bc70 con 0x7fd8c0100290 2023-11-16T16:46:19.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.377+0000 7fd8bcff9700 1 -- 172.21.15.93:0/3925575075 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0003c20 con 0x7fd8c0100290 2023-11-16T16:46:19.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.377+0000 7fd8c5bff700 1 -- 172.21.15.93:0/3925575075 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c010bae0 con 0x7fd8c0100290 2023-11-16T16:46:19.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.378+0000 7fd8c5bff700 1 -- 172.21.15.93:0/3925575075 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c01a0830 con 0x7fd8c0100290 2023-11-16T16:46:19.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.380+0000 7fd8bcff9700 1 -- 172.21.15.93:0/3925575075 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd8b000c930 con 0x7fd8c0100290 2023-11-16T16:46:19.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.381+0000 7fd8bcff9700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8a80a1150 0x7fd8a80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:19.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.381+0000 7fd8bcff9700 1 -- 172.21.15.93:0/3925575075 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd8b0014070 con 0x7fd8c0100290 2023-11-16T16:46:19.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.381+0000 7fd8beffd700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8a80a1150 0x7fd8a80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:19.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.382+0000 7fd8beffd700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8a80a1150 0x7fd8a80a3600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4005fd0 tx=0x7fd8b4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:19.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.385+0000 7fd8bcff9700 1 -- 172.21.15.93:0/3925575075 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd8b00961f0 con 0x7fd8c0100290 2023-11-16T16:46:19.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.628+0000 7fd8c5bff700 1 -- 172.21.15.93:0/3925575075 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8c01a0de0 con 0x7fd8c0100290 2023-11-16T16:46:19.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.628+0000 7fd8bcff9700 1 -- 172.21.15.93:0/3925575075 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6568 (secure 0 0 0) 0x7fd8b0099810 con 0x7fd8c0100290 2023-11-16T16:46:19.630 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:19.630 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/778982589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":778982589},{"type":"v1","addr":"172.21.15.159:6837","nonce":778982589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-16T16:46:19.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.631+0000 7fd8c5bff700 1 -- 172.21.15.93:0/3925575075 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8a80a1150 msgr2=0x7fd8a80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:19.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.631+0000 7fd8c5bff700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8a80a1150 0x7fd8a80a3600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4005fd0 tx=0x7fd8b4009310 comp rx=0 tx=0).stop 2023-11-16T16:46:19.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.631+0000 7fd8c5bff700 1 -- 172.21.15.93:0/3925575075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c0100290 msgr2=0x7fd8c0198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:19.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.631+0000 7fd8c5bff700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c0100290 0x7fd8c0198290 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000e970 tx=0x7fd8b000ec80 comp rx=0 tx=0).stop 2023-11-16T16:46:19.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.631+0000 7fd8c5bff700 1 -- 172.21.15.93:0/3925575075 shutdown_connections 2023-11-16T16:46:19.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.631+0000 7fd8c5bff700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8a80a1150 0x7fd8a80a3600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:19.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.631+0000 7fd8c5bff700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c0100290 0x7fd8c0198290 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:19.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.631+0000 7fd8c5bff700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8c0101e10 0x7fd8c019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:19.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.631+0000 7fd8c5bff700 1 --2- 172.21.15.93:0/3925575075 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8c0101480 0x7fd8c01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:19.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.631+0000 7fd8c5bff700 1 -- 172.21.15.93:0/3925575075 >> 172.21.15.93:0/3925575075 conn(0x7fd8c00fb870 msgr2=0x7fd8c00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:19.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.631+0000 7fd8c5bff700 1 -- 172.21.15.93:0/3925575075 shutdown_connections 2023-11-16T16:46:19.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:19.631+0000 7fd8c5bff700 1 -- 172.21.15.93:0/3925575075 wait complete. 2023-11-16T16:46:19.634 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 9 2023-11-16T16:46:19.695 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:19 smithi159 ceph-mon[131528]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:19.696 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:19 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] up:boot 2023-11-16T16:46:19.696 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:19 smithi159 ceph-mon[131528]: fsmap 8 up:standby 2023-11-16T16:46:19.696 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:46:19.696 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:19.696 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:19.696 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:19.696 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:19.696 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:19.696 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:19.696 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:19 smithi159 ceph-mon[131528]: Deploying daemon mds.i on smithi159 2023-11-16T16:46:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:19 smithi125 ceph-mon[131076]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:19 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] up:boot 2023-11-16T16:46:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:19 smithi125 ceph-mon[131076]: fsmap 8 up:standby 2023-11-16T16:46:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:46:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:19.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:19 smithi125 ceph-mon[131076]: Deploying daemon mds.i on smithi159 2023-11-16T16:46:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:19 smithi093 ceph-mon[97863]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:19 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] up:boot 2023-11-16T16:46:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:19 smithi093 ceph-mon[97863]: fsmap 8 up:standby 2023-11-16T16:46:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:46:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:19 smithi093 ceph-mon[97863]: Deploying daemon mds.i on smithi159 2023-11-16T16:46:20.457 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:46:20 smithi159 systemd[1]: Starting Ceph mds.i for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:46:20.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:20 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3925575075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:20.735 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:46:20 smithi159 podman[157638]: 2023-11-16T16:46:20.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:20 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3925575075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:20 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3925575075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:21.235 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:46:20 smithi159 ceph-mds[157678]: -- 172.21.15.159:0/3814449433 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563b13073520 con 0x563b13048800 2023-11-16T16:46:21.235 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:46:20 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-i[157654]: starting mds.i at 2023-11-16T16:46:21.235 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:46:20 smithi159 bash[157638]: 416d8f2fa1500a1512f03d80081131dd5844ccfb656a4a1cff7467944403711b 2023-11-16T16:46:21.235 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:46:20 smithi159 systemd[1]: Started Ceph mds.i for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:46:21.653 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:21.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.869+0000 7fbe0a99d700 1 -- 172.21.15.93:0/575026570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe04101340 msgr2=0x7fbe041017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:21.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.869+0000 7fbe0a99d700 1 --2- 172.21.15.93:0/575026570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe04101340 0x7fbe041017b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009990 tx=0x7fbdf4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:21.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.869+0000 7fbe0a99d700 1 -- 172.21.15.93:0/575026570 shutdown_connections 2023-11-16T16:46:21.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.869+0000 7fbe0a99d700 1 --2- 172.21.15.93:0/575026570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe04101cf0 0x7fbe04108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:21.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.869+0000 7fbe0a99d700 1 --2- 172.21.15.93:0/575026570 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe041000b0 0x7fbe041004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:21.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.869+0000 7fbe0a99d700 1 --2- 172.21.15.93:0/575026570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe04101340 0x7fbe041017b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:21.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.869+0000 7fbe0a99d700 1 -- 172.21.15.93:0/575026570 >> 172.21.15.93:0/575026570 conn(0x7fbe040fb6d0 msgr2=0x7fbe040fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:21.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.869+0000 7fbe0a99d700 1 -- 172.21.15.93:0/575026570 shutdown_connections 2023-11-16T16:46:21.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.869+0000 7fbe0a99d700 1 -- 172.21.15.93:0/575026570 wait complete. 2023-11-16T16:46:21.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.870+0000 7fbe0a99d700 1 Processor -- start 2023-11-16T16:46:21.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.870+0000 7fbe0a99d700 1 -- start start 2023-11-16T16:46:21.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0a99d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe041000b0 0x7fbe04197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:21.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0a99d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe04101340 0x7fbe041984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:21.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0a99d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe04101cf0 0x7fbe0419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:21.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0a99d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0419fb30 con 0x7fbe041000b0 2023-11-16T16:46:21.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0a99d700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0419fc70 con 0x7fbe04101cf0 2023-11-16T16:46:21.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0a99d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0419fe00 con 0x7fbe04101340 2023-11-16T16:46:21.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0999b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe041000b0 0x7fbe04197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:21.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0999b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe041000b0 0x7fbe04197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49502/0 (socket says 172.21.15.93:49502) 2023-11-16T16:46:21.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0999b700 1 -- 172.21.15.93:0/2971894652 learned_addr learned my addr 172.21.15.93:0/2971894652 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:21.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0919a700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe04101340 0x7fbe041984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:21.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0a19c700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe04101cf0 0x7fbe0419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:21.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0999b700 1 -- 172.21.15.93:0/2971894652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe04101340 msgr2=0x7fbe041984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:21.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0999b700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe04101340 0x7fbe041984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:21.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0999b700 1 -- 172.21.15.93:0/2971894652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe04101cf0 msgr2=0x7fbe0419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:21.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0999b700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe04101cf0 0x7fbe0419f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:21.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.871+0000 7fbe0999b700 1 -- 172.21.15.93:0/2971894652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf40095f0 con 0x7fbe041000b0 2023-11-16T16:46:21.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.872+0000 7fbe0999b700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe041000b0 0x7fbe04197fb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e9a0 tx=0x7fbe0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:21.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.872+0000 7fbdfaffd700 1 -- 172.21.15.93:0/2971894652 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00009790 con 0x7fbe041000b0 2023-11-16T16:46:21.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.872+0000 7fbdfaffd700 1 -- 172.21.15.93:0/2971894652 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe00010430 con 0x7fbe041000b0 2023-11-16T16:46:21.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.872+0000 7fbe0a99d700 1 -- 172.21.15.93:0/2971894652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0410bb00 con 0x7fbe041000b0 2023-11-16T16:46:21.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.872+0000 7fbdfaffd700 1 -- 172.21.15.93:0/2971894652 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00018680 con 0x7fbe041000b0 2023-11-16T16:46:21.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.872+0000 7fbe0a99d700 1 -- 172.21.15.93:0/2971894652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0410b970 con 0x7fbe041000b0 2023-11-16T16:46:21.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.873+0000 7fbe0a99d700 1 -- 172.21.15.93:0/2971894652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe04192730 con 0x7fbe041000b0 2023-11-16T16:46:21.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.875+0000 7fbdfaffd700 1 -- 172.21.15.93:0/2971894652 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbe000105a0 con 0x7fbe041000b0 2023-11-16T16:46:21.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.876+0000 7fbdfaffd700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbdf00a1150 0x7fbdf00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:21.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.876+0000 7fbdfaffd700 1 -- 172.21.15.93:0/2971894652 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbe00014070 con 0x7fbe041000b0 2023-11-16T16:46:21.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.876+0000 7fbe0919a700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbdf00a1150 0x7fbdf00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:21.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.877+0000 7fbe0919a700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbdf00a1150 0x7fbdf00a3600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400b2e0 tx=0x7fbdf4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:21.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:21.880+0000 7fbdfaffd700 1 -- 172.21.15.93:0/2971894652 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbe00051730 con 0x7fbe041000b0 2023-11-16T16:46:22.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:21 smithi093 ceph-mon[97863]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:22.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:22.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:22.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:22.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:22.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:22.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:22.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:21 smithi093 ceph-mon[97863]: Deploying daemon mds.g on smithi093 2023-11-16T16:46:22.004 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:21 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] up:boot 2023-11-16T16:46:22.004 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:21 smithi093 ceph-mon[97863]: fsmap 9 up:standby 2023-11-16T16:46:22.004 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:46:22.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.146+0000 7fbe0a99d700 1 -- 172.21.15.93:0/2971894652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe0402cfe0 con 0x7fbe041000b0 2023-11-16T16:46:30.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.147+0000 7fbdfaffd700 1 -- 172.21.15.93:0/2971894652 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7327 (secure 0 0 0) 0x7fbe00096220 con 0x7fbe041000b0 2023-11-16T16:46:30.442 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:30.442 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/778982589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":778982589},{"type":"v1","addr":"172.21.15.159:6837","nonce":778982589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/149300707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":149300707},{"type":"v1","addr":"172.21.15.159:6839","nonce":149300707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:21 smithi125 ceph-mon[131076]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:21 smithi125 ceph-mon[131076]: Deploying daemon mds.g on smithi093 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:21 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] up:boot 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:21 smithi125 ceph-mon[131076]: fsmap 9 up:standby 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:22 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2971894652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:23 smithi125 ceph-mon[131076]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:25 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:25 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:25 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:25 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:25 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:25 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:25 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.93:6838/3614592201,v1:172.21.15.93:6839/3614592201] up:boot 2023-11-16T16:46:30.444 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:25 smithi125 ceph-mon[131076]: fsmap 10 up:standby 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:25 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:26 smithi125 ceph-mon[131076]: Deploying daemon mds.a on smithi093 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:26 smithi125 ceph-mon[131076]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:28 smithi125 ceph-mon[131076]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:28 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.93:6840/4068789362,v1:172.21.15.93:6841/4068789362] up:boot 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:28 smithi125 ceph-mon[131076]: fsmap 11 up:standby 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:30.445 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:28 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:30.446 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:46:28 smithi125 systemd[1]: Starting Ceph mds.b for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:46:30.446 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:21 smithi159 ceph-mon[131528]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.446 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.446 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.446 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.446 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:30.446 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:30.446 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:21 smithi159 ceph-mon[131528]: Deploying daemon mds.g on smithi093 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:21 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] up:boot 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:21 smithi159 ceph-mon[131528]: fsmap 9 up:standby 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:22 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2971894652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:23 smithi159 ceph-mon[131528]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:25 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:25 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:25 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:25 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:25 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:25 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:30.447 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:25 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.93:6838/3614592201,v1:172.21.15.93:6839/3614592201] up:boot 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:25 smithi159 ceph-mon[131528]: fsmap 10 up:standby 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:25 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:26 smithi159 ceph-mon[131528]: Deploying daemon mds.a on smithi093 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:26 smithi159 ceph-mon[131528]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:28 smithi159 ceph-mon[131528]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:28 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.93:6840/4068789362,v1:172.21.15.93:6841/4068789362] up:boot 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:28 smithi159 ceph-mon[131528]: fsmap 11 up:standby 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:30.448 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:30.449 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:28 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:30.449 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:29 smithi159 ceph-mon[131528]: Deploying daemon mds.b on smithi125 2023-11-16T16:46:30.449 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:29 smithi125 ceph-mon[131076]: Deploying daemon mds.b on smithi125 2023-11-16T16:46:30.470 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:46:29 smithi125 podman[156524]: 2023-11-16T16:46:30.470 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:46:29 smithi125 ceph-mds[156564]: -- 172.21.15.125:0/3403914454 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5610ddee5520 con 0x5610ddebb000 2023-11-16T16:46:30.470 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:46:29 smithi125 bash[156524]: f719c66ca2430c31b6e2f263017860127e461651463801bd6b736f5a00ff4fb6 2023-11-16T16:46:30.470 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:46:29 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-b[156540]: starting mds.b at 2023-11-16T16:46:30.470 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:46:29 smithi125 systemd[1]: Started Ceph mds.b for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:46:30.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.149+0000 7fbe0a99d700 1 -- 172.21.15.93:0/2971894652 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbdf00a1150 msgr2=0x7fbdf00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:30.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.149+0000 7fbe0a99d700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbdf00a1150 0x7fbdf00a3600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400b2e0 tx=0x7fbdf4005fb0 comp rx=0 tx=0).stop 2023-11-16T16:46:30.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.149+0000 7fbe0a99d700 1 -- 172.21.15.93:0/2971894652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe041000b0 msgr2=0x7fbe04197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:30.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.149+0000 7fbe0a99d700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe041000b0 0x7fbe04197fb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e9a0 tx=0x7fbe0000ecb0 comp rx=0 tx=0).stop 2023-11-16T16:46:30.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.149+0000 7fbe0a99d700 1 -- 172.21.15.93:0/2971894652 shutdown_connections 2023-11-16T16:46:30.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.149+0000 7fbe0a99d700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbdf00a1150 0x7fbdf00a3600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:30.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.149+0000 7fbe0a99d700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe041000b0 0x7fbe04197fb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:30.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.149+0000 7fbe0a99d700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe04101cf0 0x7fbe0419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:30.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.149+0000 7fbe0a99d700 1 --2- 172.21.15.93:0/2971894652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe04101340 0x7fbe041984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:30.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.149+0000 7fbe0a99d700 1 -- 172.21.15.93:0/2971894652 >> 172.21.15.93:0/2971894652 conn(0x7fbe040fb6d0 msgr2=0x7fbe04106c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:30.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.150+0000 7fbe0a99d700 1 -- 172.21.15.93:0/2971894652 shutdown_connections 2023-11-16T16:46:30.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:22.150+0000 7fbe0a99d700 1 -- 172.21.15.93:0/2971894652 wait complete. 2023-11-16T16:46:30.471 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 10 2023-11-16T16:46:30.472 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:22 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2971894652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:30.472 INFO:journalctl@ceph.mds.g.smithi093.stdout:Nov 16 16:46:23 smithi093 systemd[1]: Starting Ceph mds.g for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:46:30.472 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:23 smithi093 ceph-mon[97863]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.473 INFO:journalctl@ceph.mds.g.smithi093.stdout:Nov 16 16:46:23 smithi093 podman[131871]: 2023-11-16T16:46:30.473 INFO:journalctl@ceph.mds.g.smithi093.stdout:Nov 16 16:46:23 smithi093 ceph-mds[131911]: -- 172.21.15.93:0/1299923360 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5595fae41520 con 0x5595fae18c00 2023-11-16T16:46:30.473 INFO:journalctl@ceph.mds.g.smithi093.stdout:Nov 16 16:46:23 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-g[131887]: starting mds.g at 2023-11-16T16:46:30.473 INFO:journalctl@ceph.mds.g.smithi093.stdout:Nov 16 16:46:23 smithi093 bash[131871]: cf1b72894d54b600a1e9020c9a447999c1dd59ff67387225b9d7634c26ec0a88 2023-11-16T16:46:30.473 INFO:journalctl@ceph.mds.g.smithi093.stdout:Nov 16 16:46:24 smithi093 systemd[1]: Started Ceph mds.g for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:46:30.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:25 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:25 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:25 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:25 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:30.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:25 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:30.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:25 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:30.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:25 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.93:6838/3614592201,v1:172.21.15.93:6839/3614592201] up:boot 2023-11-16T16:46:30.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:25 smithi093 ceph-mon[97863]: fsmap 10 up:standby 2023-11-16T16:46:30.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:25 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-16T16:46:30.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:26 smithi093 ceph-mon[97863]: Deploying daemon mds.a on smithi093 2023-11-16T16:46:30.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:26 smithi093 ceph-mon[97863]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.475 INFO:journalctl@ceph.mds.a.smithi093.stdout:Nov 16 16:46:26 smithi093 systemd[1]: Starting Ceph mds.a for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:46:30.475 INFO:journalctl@ceph.mds.a.smithi093.stdout:Nov 16 16:46:26 smithi093 podman[132216]: 2023-11-16T16:46:30.475 INFO:journalctl@ceph.mds.a.smithi093.stdout:Nov 16 16:46:27 smithi093 bash[132216]: 9e751868326443cf9a6e3b352c6c6dafe033fe2bcd9aee5a2f4426e29dc2da8f 2023-11-16T16:46:30.475 INFO:journalctl@ceph.mds.a.smithi093.stdout:Nov 16 16:46:27 smithi093 ceph-mds[132256]: -- 172.21.15.93:0/2604077952 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ee89337520 con 0x55ee8930e800 2023-11-16T16:46:30.475 INFO:journalctl@ceph.mds.a.smithi093.stdout:Nov 16 16:46:27 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-a[132232]: starting mds.a at 2023-11-16T16:46:30.475 INFO:journalctl@ceph.mds.a.smithi093.stdout:Nov 16 16:46:27 smithi093 systemd[1]: Started Ceph mds.a for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:46:30.476 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:28 smithi093 ceph-mon[97863]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.476 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:28 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.93:6840/4068789362,v1:172.21.15.93:6841/4068789362] up:boot 2023-11-16T16:46:30.476 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:28 smithi093 ceph-mon[97863]: fsmap 11 up:standby 2023-11-16T16:46:30.476 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-16T16:46:30.476 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.476 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.476 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.476 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:46:30.476 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:46:30.476 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:28 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:30.476 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:29 smithi093 ceph-mon[97863]: Deploying daemon mds.b on smithi125 2023-11-16T16:46:30.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:30 smithi159 ceph-mon[131528]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:30 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:30 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:30 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:30 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:30 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:30 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:46:30.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:30 smithi125 ceph-mon[131076]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:30 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:30 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:30 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:30 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:30 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:30 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:46:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:30 smithi093 ceph-mon[97863]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:30 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:30 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:30 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:30 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:30 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:30.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:30 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:46:31.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:31 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] up:boot 2023-11-16T16:46:31.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:31 smithi159 ceph-mon[131528]: fsmap 12 up:standby 2023-11-16T16:46:31.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:31 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:46:31.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:31 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:31.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:31 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] up:boot 2023-11-16T16:46:31.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:31 smithi125 ceph-mon[131076]: fsmap 12 up:standby 2023-11-16T16:46:31.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:31 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:46:31.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:31 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:31 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] up:boot 2023-11-16T16:46:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:31 smithi093 ceph-mon[97863]: fsmap 12 up:standby 2023-11-16T16:46:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:31 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:46:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:31 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:32.473 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:32.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:32 smithi159 ceph-mon[131528]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:32.540 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:32 smithi093 ceph-mon[97863]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:32.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:32 smithi125 ceph-mon[131076]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:32.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.688+0000 7fa744dff700 1 -- 172.21.15.93:0/292435344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa740100290 msgr2=0x7fa7401006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:32.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.688+0000 7fa744dff700 1 --2- 172.21.15.93:0/292435344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa740100290 0x7fa7401006a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa730009990 tx=0x7fa730009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:32.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.688+0000 7fa744dff700 1 -- 172.21.15.93:0/292435344 shutdown_connections 2023-11-16T16:46:32.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.688+0000 7fa744dff700 1 --2- 172.21.15.93:0/292435344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa740100290 0x7fa7401006a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:32.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.688+0000 7fa744dff700 1 --2- 172.21.15.93:0/292435344 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa740101e10 0x7fa7401086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:32.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.688+0000 7fa744dff700 1 --2- 172.21.15.93:0/292435344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa740101480 0x7fa7401018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:32.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.688+0000 7fa744dff700 1 -- 172.21.15.93:0/292435344 >> 172.21.15.93:0/292435344 conn(0x7fa7400fb870 msgr2=0x7fa7400fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:32.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.688+0000 7fa744dff700 1 -- 172.21.15.93:0/292435344 shutdown_connections 2023-11-16T16:46:32.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.688+0000 7fa744dff700 1 -- 172.21.15.93:0/292435344 wait complete. 2023-11-16T16:46:32.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.689+0000 7fa744dff700 1 Processor -- start 2023-11-16T16:46:32.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.689+0000 7fa744dff700 1 -- start start 2023-11-16T16:46:32.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.689+0000 7fa744dff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa740100290 0x7fa7401981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:32.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.689+0000 7fa744dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa740101480 0x7fa740198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:32.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa744dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa740101e10 0x7fa74019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:32.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa744dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74019fd40 con 0x7fa740101480 2023-11-16T16:46:32.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa744dff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74019fe80 con 0x7fa740100290 2023-11-16T16:46:32.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa744dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7401a0010 con 0x7fa740101e10 2023-11-16T16:46:32.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa73e59c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa740100290 0x7fa7401981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:32.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa73e59c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa740100290 0x7fa7401981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:35180/0 (socket says 172.21.15.93:35180) 2023-11-16T16:46:32.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa73e59c700 1 -- 172.21.15.93:0/1698022721 learned_addr learned my addr 172.21.15.93:0/1698022721 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:32.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa737fff700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa740101480 0x7fa740198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:32.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa73ed9d700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa740101e10 0x7fa74019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:32.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa73e59c700 1 -- 172.21.15.93:0/1698022721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa740101e10 msgr2=0x7fa74019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:32.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa73e59c700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa740101e10 0x7fa74019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:32.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa73e59c700 1 -- 172.21.15.93:0/1698022721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa740101480 msgr2=0x7fa740198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:32.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa73e59c700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa740101480 0x7fa740198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:32.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa73e59c700 1 -- 172.21.15.93:0/1698022721 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7300095f0 con 0x7fa740100290 2023-11-16T16:46:32.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.690+0000 7fa73e59c700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa740100290 0x7fa7401981c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa730005740 tx=0x7fa730005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:32.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.691+0000 7fa7377fe700 1 -- 172.21.15.93:0/1698022721 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73001d070 con 0x7fa740100290 2023-11-16T16:46:32.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.691+0000 7fa744dff700 1 -- 172.21.15.93:0/1698022721 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74010bc40 con 0x7fa740100290 2023-11-16T16:46:32.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.691+0000 7fa7377fe700 1 -- 172.21.15.93:0/1698022721 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa730021410 con 0x7fa740100290 2023-11-16T16:46:32.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.691+0000 7fa7377fe700 1 -- 172.21.15.93:0/1698022721 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa730017650 con 0x7fa740100290 2023-11-16T16:46:32.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.691+0000 7fa744dff700 1 -- 172.21.15.93:0/1698022721 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74010bab0 con 0x7fa740100290 2023-11-16T16:46:32.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.692+0000 7fa744dff700 1 -- 172.21.15.93:0/1698022721 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7401928d0 con 0x7fa740100290 2023-11-16T16:46:32.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.693+0000 7fa7377fe700 1 -- 172.21.15.93:0/1698022721 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa730021580 con 0x7fa740100290 2023-11-16T16:46:32.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.695+0000 7fa7377fe700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa72c0a1150 0x7fa72c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:32.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.695+0000 7fa7377fe700 1 -- 172.21.15.93:0/1698022721 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7300cd5e0 con 0x7fa740100290 2023-11-16T16:46:32.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.695+0000 7fa737fff700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa72c0a1150 0x7fa72c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:32.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.695+0000 7fa737fff700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa72c0a1150 0x7fa72c0a3600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa728009c60 tx=0x7fa728009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:32.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.697+0000 7fa7377fe700 1 -- 172.21.15.93:0/1698022721 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa730097240 con 0x7fa740100290 2023-11-16T16:46:32.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.953+0000 7fa744dff700 1 -- 172.21.15.93:0/1698022721 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa740061700 con 0x7fa740100290 2023-11-16T16:46:32.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.954+0000 7fa7377fe700 1 -- 172.21.15.93:0/1698022721 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9604 (secure 0 0 0) 0x7fa73009a860 con 0x7fa740100290 2023-11-16T16:46:32.956 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:32.956 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3614592201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3614592201},{"type":"v1","addr":"172.21.15.93:6839","nonce":3614592201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4068789362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4068789362},{"type":"v1","addr":"172.21.15.93:6841","nonce":4068789362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/778982589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":778982589},{"type":"v1","addr":"172.21.15.159:6837","nonce":778982589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/149300707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":149300707},{"type":"v1","addr":"172.21.15.159:6839","nonce":149300707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6839/3590027328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":3590027328},{"type":"v1","addr":"172.21.15.125:6839","nonce":3590027328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-11-16T16:46:32.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.956+0000 7fa744dff700 1 -- 172.21.15.93:0/1698022721 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa72c0a1150 msgr2=0x7fa72c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:32.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.956+0000 7fa744dff700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa72c0a1150 0x7fa72c0a3600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa728009c60 tx=0x7fa728009260 comp rx=0 tx=0).stop 2023-11-16T16:46:32.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.956+0000 7fa744dff700 1 -- 172.21.15.93:0/1698022721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa740100290 msgr2=0x7fa7401981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:32.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.956+0000 7fa744dff700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa740100290 0x7fa7401981c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa730005740 tx=0x7fa730005770 comp rx=0 tx=0).stop 2023-11-16T16:46:32.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.956+0000 7fa744dff700 1 -- 172.21.15.93:0/1698022721 shutdown_connections 2023-11-16T16:46:32.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.956+0000 7fa744dff700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa72c0a1150 0x7fa72c0a3600 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:32.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.956+0000 7fa744dff700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa740101e10 0x7fa74019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:32.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.956+0000 7fa744dff700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa740100290 0x7fa7401981c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:32.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.956+0000 7fa744dff700 1 --2- 172.21.15.93:0/1698022721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa740101480 0x7fa740198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:32.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.956+0000 7fa744dff700 1 -- 172.21.15.93:0/1698022721 >> 172.21.15.93:0/1698022721 conn(0x7fa7400fb870 msgr2=0x7fa740106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:32.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.957+0000 7fa744dff700 1 -- 172.21.15.93:0/1698022721 shutdown_connections 2023-11-16T16:46:32.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:32.957+0000 7fa744dff700 1 -- 172.21.15.93:0/1698022721 wait complete. 2023-11-16T16:46:32.959 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 13 2023-11-16T16:46:32.978 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:46:33.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.180+0000 7f59449ff700 1 -- 172.21.15.93:0/669676533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5940101480 msgr2=0x7f59401018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:33.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.180+0000 7f59449ff700 1 --2- 172.21.15.93:0/669676533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5940101480 0x7f59401018d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5934009990 tx=0x7f5934009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.180+0000 7f59449ff700 1 -- 172.21.15.93:0/669676533 shutdown_connections 2023-11-16T16:46:33.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.180+0000 7f59449ff700 1 --2- 172.21.15.93:0/669676533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5940101e10 0x7f59401086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.180+0000 7f59449ff700 1 --2- 172.21.15.93:0/669676533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5940100290 0x7f59401006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.180+0000 7f59449ff700 1 --2- 172.21.15.93:0/669676533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5940101480 0x7f59401018d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.180+0000 7f59449ff700 1 -- 172.21.15.93:0/669676533 >> 172.21.15.93:0/669676533 conn(0x7f59400fb870 msgr2=0x7f59400fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:33.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.180+0000 7f59449ff700 1 -- 172.21.15.93:0/669676533 shutdown_connections 2023-11-16T16:46:33.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.180+0000 7f59449ff700 1 -- 172.21.15.93:0/669676533 wait complete. 2023-11-16T16:46:33.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.181+0000 7f59449ff700 1 Processor -- start 2023-11-16T16:46:33.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.181+0000 7f59449ff700 1 -- start start 2023-11-16T16:46:33.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.181+0000 7f59449ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5940100290 0x7f5940196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:33.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.181+0000 7f59449ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5940101480 0x7f59401965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:33.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.181+0000 7f59449ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5940101e10 0x7f594019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:33.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.181+0000 7f59449ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594019dbf0 con 0x7f5940100290 2023-11-16T16:46:33.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.181+0000 7f59449ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594019dd30 con 0x7f5940101e10 2023-11-16T16:46:33.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.181+0000 7f59449ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594019dec0 con 0x7f5940101480 2023-11-16T16:46:33.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.182+0000 7f593e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5940100290 0x7f5940196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:33.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.182+0000 7f593e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5940100290 0x7f5940196070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40006/0 (socket says 172.21.15.93:40006) 2023-11-16T16:46:33.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.182+0000 7f593e59c700 1 -- 172.21.15.93:0/4108999804 learned_addr learned my addr 172.21.15.93:0/4108999804 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:33.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.182+0000 7f593ed9d700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5940101e10 0x7f594019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:33.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.182+0000 7f593dd9b700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5940101480 0x7f59401965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:33.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.182+0000 7f593e59c700 1 -- 172.21.15.93:0/4108999804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5940101480 msgr2=0x7f59401965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:33.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.182+0000 7f593e59c700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5940101480 0x7f59401965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.182+0000 7f593e59c700 1 -- 172.21.15.93:0/4108999804 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5940101e10 msgr2=0x7f594019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:33.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.182+0000 7f593e59c700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5940101e10 0x7f594019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.182+0000 7f593e59c700 1 -- 172.21.15.93:0/4108999804 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59340095f0 con 0x7f5940100290 2023-11-16T16:46:33.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.182+0000 7f593e59c700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5940100290 0x7f5940196070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f593000e970 tx=0x7f593000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:33.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.182+0000 7f592f7fe700 1 -- 172.21.15.93:0/4108999804 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930009790 con 0x7f5940100290 2023-11-16T16:46:33.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.183+0000 7f592f7fe700 1 -- 172.21.15.93:0/4108999804 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f593000c7e0 con 0x7f5940100290 2023-11-16T16:46:33.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.183+0000 7f59449ff700 1 -- 172.21.15.93:0/4108999804 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594010bc50 con 0x7f5940100290 2023-11-16T16:46:33.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.183+0000 7f592f7fe700 1 -- 172.21.15.93:0/4108999804 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930003c70 con 0x7f5940100290 2023-11-16T16:46:33.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.183+0000 7f59449ff700 1 -- 172.21.15.93:0/4108999804 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594010bac0 con 0x7f5940100290 2023-11-16T16:46:33.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.184+0000 7f59449ff700 1 -- 172.21.15.93:0/4108999804 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59401078f0 con 0x7f5940100290 2023-11-16T16:46:33.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.186+0000 7f592f7fe700 1 -- 172.21.15.93:0/4108999804 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f593000cc50 con 0x7f5940100290 2023-11-16T16:46:33.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.188+0000 7f592f7fe700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f59280a11d0 0x7f59280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:33.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.188+0000 7f592f7fe700 1 -- 172.21.15.93:0/4108999804 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5930014070 con 0x7f5940100290 2023-11-16T16:46:33.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.188+0000 7f593dd9b700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f59280a11d0 0x7f59280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:33.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.189+0000 7f593dd9b700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f59280a11d0 0x7f59280a3680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f593400b2e0 tx=0x7f5934005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:33.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.191+0000 7f592f7fe700 1 -- 172.21.15.93:0/4108999804 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f59300961a0 con 0x7f5940100290 2023-11-16T16:46:33.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.413+0000 7f59449ff700 1 -- 172.21.15.93:0/4108999804 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f594002cfe0 con 0x7f5940100290 2023-11-16T16:46:33.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.414+0000 7f592f7fe700 1 -- 172.21.15.93:0/4108999804 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18432 (secure 0 0 0) 0x7f5930003dd0 con 0x7f5940100290 2023-11-16T16:46:33.416 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:33.416 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":80,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:45:47.266367+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:46:33.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.417+0000 7f59449ff700 1 -- 172.21.15.93:0/4108999804 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f59280a11d0 msgr2=0x7f59280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:33.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.417+0000 7f59449ff700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f59280a11d0 0x7f59280a3680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f593400b2e0 tx=0x7f5934005fb0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.417+0000 7f59449ff700 1 -- 172.21.15.93:0/4108999804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5940100290 msgr2=0x7f5940196070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:33.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.417+0000 7f59449ff700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5940100290 0x7f5940196070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f593000e970 tx=0x7f593000ec80 comp rx=0 tx=0).stop 2023-11-16T16:46:33.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.417+0000 7f59449ff700 1 -- 172.21.15.93:0/4108999804 shutdown_connections 2023-11-16T16:46:33.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.417+0000 7f59449ff700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f59280a11d0 0x7f59280a3680 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.417+0000 7f59449ff700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5940100290 0x7f5940196070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.417+0000 7f59449ff700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5940101e10 0x7f594019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.417+0000 7f59449ff700 1 --2- 172.21.15.93:0/4108999804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5940101480 0x7f59401965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.417+0000 7f59449ff700 1 -- 172.21.15.93:0/4108999804 >> 172.21.15.93:0/4108999804 conn(0x7f59400fb870 msgr2=0x7f5940104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:33.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.417+0000 7f59449ff700 1 -- 172.21.15.93:0/4108999804 shutdown_connections 2023-11-16T16:46:33.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.417+0000 7f59449ff700 1 -- 172.21.15.93:0/4108999804 wait complete. 2023-11-16T16:46:33.441 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:42:13.029135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:46:33.441 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:46:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.645+0000 7f71a71ff700 1 -- 172.21.15.93:0/982581666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71a01002c0 msgr2=0x7f71a01006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.645+0000 7f71a71ff700 1 --2- 172.21.15.93:0/982581666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71a01002c0 0x7f71a01006d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7194009990 tx=0x7f7194009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.645+0000 7f71a71ff700 1 -- 172.21.15.93:0/982581666 shutdown_connections 2023-11-16T16:46:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.645+0000 7f71a71ff700 1 --2- 172.21.15.93:0/982581666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71a01002c0 0x7f71a01006d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.645+0000 7f71a71ff700 1 --2- 172.21.15.93:0/982581666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a0101e40 0x7f71a0108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.645+0000 7f71a71ff700 1 --2- 172.21.15.93:0/982581666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a01014b0 0x7f71a0101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.645+0000 7f71a71ff700 1 -- 172.21.15.93:0/982581666 >> 172.21.15.93:0/982581666 conn(0x7f71a00fb8a0 msgr2=0x7f71a00fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.645+0000 7f71a71ff700 1 -- 172.21.15.93:0/982581666 shutdown_connections 2023-11-16T16:46:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.645+0000 7f71a71ff700 1 -- 172.21.15.93:0/982581666 wait complete. 2023-11-16T16:46:33.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.646+0000 7f71a71ff700 1 Processor -- start 2023-11-16T16:46:33.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.646+0000 7f71a71ff700 1 -- start start 2023-11-16T16:46:33.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f71a71ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a01002c0 0x7f71a0196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:33.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f71a71ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71a01014b0 0x7f71a01965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f71a71ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a0101e40 0x7f71a019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f71a71ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a019dbf0 con 0x7f71a01014b0 2023-11-16T16:46:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f71a71ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a019dd30 con 0x7f71a01002c0 2023-11-16T16:46:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f71a71ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a019dec0 con 0x7f71a0101e40 2023-11-16T16:46:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f719ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71a01014b0 0x7f71a01965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f719ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71a01014b0 0x7f71a01965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40028/0 (socket says 172.21.15.93:40028) 2023-11-16T16:46:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f719ffff700 1 -- 172.21.15.93:0/1719474437 learned_addr learned my addr 172.21.15.93:0/1719474437 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f71a4f9b700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a01002c0 0x7f71a0196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f71a579c700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a0101e40 0x7f71a019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f719ffff700 1 -- 172.21.15.93:0/1719474437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a0101e40 msgr2=0x7f71a019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f719ffff700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a0101e40 0x7f71a019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f719ffff700 1 -- 172.21.15.93:0/1719474437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a01002c0 msgr2=0x7f71a0196070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:33.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f719ffff700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a01002c0 0x7f71a0196070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f719ffff700 1 -- 172.21.15.93:0/1719474437 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71940095f0 con 0x7f71a01014b0 2023-11-16T16:46:33.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.647+0000 7f719ffff700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71a01014b0 0x7f71a01965b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f718c00e9a0 tx=0x7f718c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:33.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.648+0000 7f719dffb700 1 -- 172.21.15.93:0/1719474437 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c009790 con 0x7f71a01014b0 2023-11-16T16:46:33.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.648+0000 7f719dffb700 1 -- 172.21.15.93:0/1719474437 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f718c00cbd0 con 0x7f71a01014b0 2023-11-16T16:46:33.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.648+0000 7f719dffb700 1 -- 172.21.15.93:0/1719474437 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c018680 con 0x7f71a01014b0 2023-11-16T16:46:33.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.648+0000 7f71a71ff700 1 -- 172.21.15.93:0/1719474437 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a010bc90 con 0x7f71a01014b0 2023-11-16T16:46:33.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.648+0000 7f71a71ff700 1 -- 172.21.15.93:0/1719474437 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a010bb00 con 0x7f71a01014b0 2023-11-16T16:46:33.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.649+0000 7f71a71ff700 1 -- 172.21.15.93:0/1719474437 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a0107860 con 0x7f71a01014b0 2023-11-16T16:46:33.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.651+0000 7f719dffb700 1 -- 172.21.15.93:0/1719474437 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f718c00cd40 con 0x7f71a01014b0 2023-11-16T16:46:33.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.652+0000 7f719dffb700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f71900a11d0 0x7f71900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:33.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.652+0000 7f719dffb700 1 -- 172.21.15.93:0/1719474437 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f718c014070 con 0x7f71a01014b0 2023-11-16T16:46:33.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.652+0000 7f71a4f9b700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f71900a11d0 0x7f71900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:33.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.653+0000 7f71a4f9b700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f71900a11d0 0x7f71900a3680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f719400b2e0 tx=0x7f7194005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:33.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.656+0000 7f719dffb700 1 -- 172.21.15.93:0/1719474437 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f718c051730 con 0x7f71a01014b0 2023-11-16T16:46:33.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:33 smithi125 ceph-mon[131076]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:33 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1698022721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:33 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/4108999804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:46:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:33 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.862+0000 7f71a71ff700 1 -- 172.21.15.93:0/1719474437 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71a0061700 con 0x7f71a01014b0 2023-11-16T16:46:33.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.863+0000 7f719dffb700 1 -- 172.21.15.93:0/1719474437 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f718c096230 con 0x7f71a01014b0 2023-11-16T16:46:33.864 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-11-16T16:46:33.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.865+0000 7f71a71ff700 1 -- 172.21.15.93:0/1719474437 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f71900a11d0 msgr2=0x7f71900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:33.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.865+0000 7f71a71ff700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f71900a11d0 0x7f71900a3680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f719400b2e0 tx=0x7f7194005fb0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.865+0000 7f71a71ff700 1 -- 172.21.15.93:0/1719474437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71a01014b0 msgr2=0x7f71a01965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:33.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.865+0000 7f71a71ff700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71a01014b0 0x7f71a01965b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f718c00e9a0 tx=0x7f718c00ecb0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.865+0000 7f71a71ff700 1 -- 172.21.15.93:0/1719474437 shutdown_connections 2023-11-16T16:46:33.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.865+0000 7f71a71ff700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f71900a11d0 0x7f71900a3680 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.865+0000 7f71a71ff700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a0101e40 0x7f71a019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.865+0000 7f71a71ff700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a01002c0 0x7f71a0196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.865+0000 7f71a71ff700 1 --2- 172.21.15.93:0/1719474437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71a01014b0 0x7f71a01965b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:33.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.865+0000 7f71a71ff700 1 -- 172.21.15.93:0/1719474437 >> 172.21.15.93:0/1719474437 conn(0x7f71a00fb8a0 msgr2=0x7f71a01049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:33.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.866+0000 7f71a71ff700 1 -- 172.21.15.93:0/1719474437 shutdown_connections 2023-11-16T16:46:33.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:33.866+0000 7f71a71ff700 1 -- 172.21.15.93:0/1719474437 wait complete. 2023-11-16T16:46:33.884 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.884 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:33 smithi093 ceph-mon[97863]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:33.884 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.884 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.885 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.885 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:33 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1698022721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:33.885 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.885 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:33 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/4108999804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:33.885 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.885 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:33.885 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:46:33.885 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:33 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.886 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-16T16:46:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:33 smithi159 ceph-mon[131528]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:33 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1698022721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:33 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/4108999804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:33.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:46:33.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:33 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:34.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.087+0000 7f381a7ff700 1 -- 172.21.15.93:0/1037998081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f381410c860 msgr2=0x7f381410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:34.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.087+0000 7f381a7ff700 1 --2- 172.21.15.93:0/1037998081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f381410c860 0x7f381410ec40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b0a0 tx=0x7f380c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.087+0000 7f381a7ff700 1 -- 172.21.15.93:0/1037998081 shutdown_connections 2023-11-16T16:46:34.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.087+0000 7f381a7ff700 1 --2- 172.21.15.93:0/1037998081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f381410c860 0x7f381410ec40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.087+0000 7f381a7ff700 1 --2- 172.21.15.93:0/1037998081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3814074b30 0x7f3814072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.087+0000 7f381a7ff700 1 --2- 172.21.15.93:0/1037998081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3814073550 0x7f38140739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.087+0000 7f381a7ff700 1 -- 172.21.15.93:0/1037998081 >> 172.21.15.93:0/1037998081 conn(0x7f38140fb870 msgr2=0x7f38140fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:34.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.087+0000 7f381a7ff700 1 -- 172.21.15.93:0/1037998081 shutdown_connections 2023-11-16T16:46:34.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.087+0000 7f381a7ff700 1 -- 172.21.15.93:0/1037998081 wait complete. 2023-11-16T16:46:34.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.088+0000 7f381a7ff700 1 Processor -- start 2023-11-16T16:46:34.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.088+0000 7f381a7ff700 1 -- start start 2023-11-16T16:46:34.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.088+0000 7f381a7ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814073550 0x7f381419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:34.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.088+0000 7f381a7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3814074b30 0x7f381419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:34.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.088+0000 7f381a7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f381410c860 0x7f38141a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:34.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.088+0000 7f381a7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141a4250 con 0x7f381410c860 2023-11-16T16:46:34.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.088+0000 7f381a7ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141a4390 con 0x7f3814073550 2023-11-16T16:46:34.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.088+0000 7f381a7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141a4520 con 0x7f3814074b30 2023-11-16T16:46:34.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f3813fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814073550 0x7f381419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:34.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f38137fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3814074b30 0x7f381419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:34.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f38137fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3814074b30 0x7f381419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:47556/0 (socket says 172.21.15.93:47556) 2023-11-16T16:46:34.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f3813fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814073550 0x7f381419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:35222/0 (socket says 172.21.15.93:35222) 2023-11-16T16:46:34.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f3813fff700 1 -- 172.21.15.93:0/2357403106 learned_addr learned my addr 172.21.15.93:0/2357403106 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:34.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f3818d9c700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f381410c860 0x7f38141a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:34.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f3813fff700 1 -- 172.21.15.93:0/2357403106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3814074b30 msgr2=0x7f381419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:34.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f3813fff700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3814074b30 0x7f381419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f3813fff700 1 -- 172.21.15.93:0/2357403106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f381410c860 msgr2=0x7f38141a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:34.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f3813fff700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f381410c860 0x7f38141a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f3813fff700 1 -- 172.21.15.93:0/2357403106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3808009520 con 0x7f3814073550 2023-11-16T16:46:34.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f3813fff700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814073550 0x7f381419c6d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f380400e9f0 tx=0x7f380400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:34.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.089+0000 7f38117fa700 1 -- 172.21.15.93:0/2357403106 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804009750 con 0x7f3814073550 2023-11-16T16:46:34.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.090+0000 7f38117fa700 1 -- 172.21.15.93:0/2357403106 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f380400c7c0 con 0x7f3814073550 2023-11-16T16:46:34.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.090+0000 7f381a7ff700 1 -- 172.21.15.93:0/2357403106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380c009d00 con 0x7f3814073550 2023-11-16T16:46:34.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.090+0000 7f38117fa700 1 -- 172.21.15.93:0/2357403106 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38040054d0 con 0x7f3814073550 2023-11-16T16:46:34.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.090+0000 7f381a7ff700 1 -- 172.21.15.93:0/2357403106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3814112290 con 0x7f3814073550 2023-11-16T16:46:34.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.091+0000 7f381a7ff700 1 -- 172.21.15.93:0/2357403106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3814112420 con 0x7f3814073550 2023-11-16T16:46:34.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.092+0000 7f38117fa700 1 -- 172.21.15.93:0/2357403106 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f380400c930 con 0x7f3814073550 2023-11-16T16:46:34.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.094+0000 7f38117fa700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f37fc0a1150 0x7f37fc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:34.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.094+0000 7f38117fa700 1 -- 172.21.15.93:0/2357403106 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3804014070 con 0x7f3814073550 2023-11-16T16:46:34.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.094+0000 7f38137fe700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f37fc0a1150 0x7f37fc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:34.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.094+0000 7f38137fe700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f37fc0a1150 0x7f37fc0a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3814074850 tx=0x7f3808009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:34.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.098+0000 7f38117fa700 1 -- 172.21.15.93:0/2357403106 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3804096280 con 0x7f3814073550 2023-11-16T16:46:34.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.307+0000 7f381a7ff700 1 -- 172.21.15.93:0/2357403106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f381402d030 con 0x7f3814073550 2023-11-16T16:46:34.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.307+0000 7f38117fa700 1 -- 172.21.15.93:0/2357403106 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f38040998a0 con 0x7f3814073550 2023-11-16T16:46:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:34.309 INFO:teuthology.orchestra.run.smithi093.stdout:[ 2023-11-16T16:46:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:46:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "poolnum": 1, 2023-11-16T16:46:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "poolname": ".mgr" 2023-11-16T16:46:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:46:34.309 INFO:teuthology.orchestra.run.smithi093.stdout:] 2023-11-16T16:46:34.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.310+0000 7f381a7ff700 1 -- 172.21.15.93:0/2357403106 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f37fc0a1150 msgr2=0x7f37fc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:34.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.310+0000 7f381a7ff700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f37fc0a1150 0x7f37fc0a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3814074850 tx=0x7f3808009260 comp rx=0 tx=0).stop 2023-11-16T16:46:34.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.310+0000 7f381a7ff700 1 -- 172.21.15.93:0/2357403106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814073550 msgr2=0x7f381419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:34.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.310+0000 7f381a7ff700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814073550 0x7f381419c6d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f380400e9f0 tx=0x7f380400ed00 comp rx=0 tx=0).stop 2023-11-16T16:46:34.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.310+0000 7f381a7ff700 1 -- 172.21.15.93:0/2357403106 shutdown_connections 2023-11-16T16:46:34.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.310+0000 7f381a7ff700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f37fc0a1150 0x7f37fc0a3600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.310+0000 7f381a7ff700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f381410c860 0x7f38141a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.310+0000 7f381a7ff700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814073550 0x7f381419c6d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.310+0000 7f381a7ff700 1 --2- 172.21.15.93:0/2357403106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3814074b30 0x7f381419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.310+0000 7f381a7ff700 1 -- 172.21.15.93:0/2357403106 >> 172.21.15.93:0/2357403106 conn(0x7f38140fb870 msgr2=0x7f38140fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:34.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.310+0000 7f381a7ff700 1 -- 172.21.15.93:0/2357403106 shutdown_connections 2023-11-16T16:46:34.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.310+0000 7f381a7ff700 1 -- 172.21.15.93:0/2357403106 wait complete. 2023-11-16T16:46:34.332 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-16T16:46:34.332 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-16T16:46:34.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.539+0000 7f70d95ff700 1 -- 172.21.15.93:0/4207170882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70d41036d0 msgr2=0x7f70d4103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:34.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.539+0000 7f70d95ff700 1 --2- 172.21.15.93:0/4207170882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70d41036d0 0x7f70d4103b20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009990 tx=0x7f70bc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.540+0000 7f70d95ff700 1 -- 172.21.15.93:0/4207170882 shutdown_connections 2023-11-16T16:46:34.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.540+0000 7f70d95ff700 1 --2- 172.21.15.93:0/4207170882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70d41024e0 0x7f70d41028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.540+0000 7f70d95ff700 1 --2- 172.21.15.93:0/4207170882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70d4104060 0x7f70d410a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.540+0000 7f70d95ff700 1 --2- 172.21.15.93:0/4207170882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70d41036d0 0x7f70d4103b20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.540+0000 7f70d95ff700 1 -- 172.21.15.93:0/4207170882 >> 172.21.15.93:0/4207170882 conn(0x7f70d40fdb20 msgr2=0x7f70d40fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:34.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.540+0000 7f70d95ff700 1 -- 172.21.15.93:0/4207170882 shutdown_connections 2023-11-16T16:46:34.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.540+0000 7f70d95ff700 1 -- 172.21.15.93:0/4207170882 wait complete. 2023-11-16T16:46:34.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70d95ff700 1 Processor -- start 2023-11-16T16:46:34.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70d95ff700 1 -- start start 2023-11-16T16:46:34.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70d95ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70d41024e0 0x7f70d4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:34.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70d95ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70d41036d0 0x7f70d41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:34.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70d95ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70d4104060 0x7f70d419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:34.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70d95ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d419fde0 con 0x7f70d4104060 2023-11-16T16:46:34.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70d95ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d419ff20 con 0x7f70d41024e0 2023-11-16T16:46:34.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70d95ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d41a00b0 con 0x7f70d41036d0 2023-11-16T16:46:34.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70d2ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70d41024e0 0x7f70d4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:34.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70d2ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70d41024e0 0x7f70d4198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:35232/0 (socket says 172.21.15.93:35232) 2023-11-16T16:46:34.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70d2ffd700 1 -- 172.21.15.93:0/1098467049 learned_addr learned my addr 172.21.15.93:0/1098467049 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:34.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70cbfff700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70d41036d0 0x7f70d41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:34.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.541+0000 7f70d37fe700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70d4104060 0x7f70d419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:34.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.542+0000 7f70d2ffd700 1 -- 172.21.15.93:0/1098467049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70d41036d0 msgr2=0x7f70d41987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:34.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.542+0000 7f70d2ffd700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70d41036d0 0x7f70d41987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.542+0000 7f70d2ffd700 1 -- 172.21.15.93:0/1098467049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70d4104060 msgr2=0x7f70d419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:34.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.542+0000 7f70d2ffd700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70d4104060 0x7f70d419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:34.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.542+0000 7f70d2ffd700 1 -- 172.21.15.93:0/1098467049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc0095f0 con 0x7f70d41024e0 2023-11-16T16:46:34.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.542+0000 7f70d37fe700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70d4104060 0x7f70d419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:34.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.542+0000 7f70d2ffd700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70d41024e0 0x7f70d4198260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70c400e970 tx=0x7f70c400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:34.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.542+0000 7f70d0ff9700 1 -- 172.21.15.93:0/1098467049 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4009790 con 0x7f70d41024e0 2023-11-16T16:46:34.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.542+0000 7f70d0ff9700 1 -- 172.21.15.93:0/1098467049 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70c400c7e0 con 0x7f70d41024e0 2023-11-16T16:46:34.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.542+0000 7f70d95ff700 1 -- 172.21.15.93:0/1098467049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d410dea0 con 0x7f70d41024e0 2023-11-16T16:46:34.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.542+0000 7f70d0ff9700 1 -- 172.21.15.93:0/1098467049 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4003c70 con 0x7f70d41024e0 2023-11-16T16:46:34.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.542+0000 7f70d95ff700 1 -- 172.21.15.93:0/1098467049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d410dd10 con 0x7f70d41024e0 2023-11-16T16:46:34.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.543+0000 7f70d95ff700 1 -- 172.21.15.93:0/1098467049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d4109410 con 0x7f70d41024e0 2023-11-16T16:46:34.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.545+0000 7f70d0ff9700 1 -- 172.21.15.93:0/1098467049 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f70c400cc50 con 0x7f70d41024e0 2023-11-16T16:46:34.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.546+0000 7f70d0ff9700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f70c00a1150 0x7f70c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:34.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.547+0000 7f70d0ff9700 1 -- 172.21.15.93:0/1098467049 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f70c4014070 con 0x7f70d41024e0 2023-11-16T16:46:34.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.547+0000 7f70cbfff700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f70c00a1150 0x7f70c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:34.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.547+0000 7f70cbfff700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f70c00a1150 0x7f70c00a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00b2e0 tx=0x7f70bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:34.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.551+0000 7f70d0ff9700 1 -- 172.21.15.93:0/1098467049 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f70c40961f0 con 0x7f70d41024e0 2023-11-16T16:46:34.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:34 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:46:34.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:34 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:46:34.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:34 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:46:34.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:34 smithi093 ceph-mon[97863]: Service mds.all has no related daemon already placed. 2023-11-16T16:46:34.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:34 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:46:34.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:34 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1719474437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:46:34.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:34 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2357403106' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-16T16:46:34.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:34.767+0000 7f70d95ff700 1 -- 172.21.15.93:0/1098467049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f70d402cfe0 con 0x7f70d41024e0 2023-11-16T16:46:34.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:34 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:46:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:34 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:46:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:34 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:46:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:34 smithi125 ceph-mon[131076]: Service mds.all has no related daemon already placed. 2023-11-16T16:46:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:34 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:46:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:34 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1719474437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:46:34.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:34 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2357403106' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-16T16:46:34.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:34 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:46:34.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:34 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:46:34.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:34 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:46:34.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:34 smithi159 ceph-mon[131528]: Service mds.all has no related daemon already placed. 2023-11-16T16:46:34.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:34 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:46:34.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:34 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1719474437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:46:34.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:34 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2357403106' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-16T16:46:35.580 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.579+0000 7f70d0ff9700 1 -- 172.21.15.93:0/1098467049 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f70c4003dd0 con 0x7f70d41024e0 2023-11-16T16:46:35.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.583+0000 7f70d95ff700 1 -- 172.21.15.93:0/1098467049 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f70c00a1150 msgr2=0x7f70c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:35.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.583+0000 7f70d95ff700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f70c00a1150 0x7f70c00a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00b2e0 tx=0x7f70bc005fb0 comp rx=0 tx=0).stop 2023-11-16T16:46:35.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.583+0000 7f70d95ff700 1 -- 172.21.15.93:0/1098467049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70d41024e0 msgr2=0x7f70d4198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:35.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.583+0000 7f70d95ff700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70d41024e0 0x7f70d4198260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70c400e970 tx=0x7f70c400ec80 comp rx=0 tx=0).stop 2023-11-16T16:46:35.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.583+0000 7f70d95ff700 1 -- 172.21.15.93:0/1098467049 shutdown_connections 2023-11-16T16:46:35.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.583+0000 7f70d95ff700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f70c00a1150 0x7f70c00a3600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:35.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.583+0000 7f70d95ff700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70d4104060 0x7f70d419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:35.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.583+0000 7f70d95ff700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70d41024e0 0x7f70d4198260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:35.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.583+0000 7f70d95ff700 1 --2- 172.21.15.93:0/1098467049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70d41036d0 0x7f70d41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:35.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.583+0000 7f70d95ff700 1 -- 172.21.15.93:0/1098467049 >> 172.21.15.93:0/1098467049 conn(0x7f70d40fdb20 msgr2=0x7f70d40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:35.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.583+0000 7f70d95ff700 1 -- 172.21.15.93:0/1098467049 shutdown_connections 2023-11-16T16:46:35.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.584+0000 7f70d95ff700 1 -- 172.21.15.93:0/1098467049 wait complete. 2023-11-16T16:46:35.586 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_metadata' created 2023-11-16T16:46:35.606 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-16T16:46:35.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.820+0000 7f160f7ff700 1 -- 172.21.15.93:0/649701961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16081024e0 msgr2=0x7f16081028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:35.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.820+0000 7f160f7ff700 1 --2- 172.21.15.93:0/649701961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16081024e0 0x7f16081028f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1604009990 tx=0x7f1604009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:35.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.820+0000 7f160f7ff700 1 -- 172.21.15.93:0/649701961 shutdown_connections 2023-11-16T16:46:35.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.820+0000 7f160f7ff700 1 --2- 172.21.15.93:0/649701961 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1608104060 0x7f160810a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:35.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.820+0000 7f160f7ff700 1 --2- 172.21.15.93:0/649701961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16081024e0 0x7f16081028f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:35.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.820+0000 7f160f7ff700 1 --2- 172.21.15.93:0/649701961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16081036d0 0x7f1608103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:35.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.820+0000 7f160f7ff700 1 -- 172.21.15.93:0/649701961 >> 172.21.15.93:0/649701961 conn(0x7f16080fdb20 msgr2=0x7f16080fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:35.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.820+0000 7f160f7ff700 1 -- 172.21.15.93:0/649701961 shutdown_connections 2023-11-16T16:46:35.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.820+0000 7f160f7ff700 1 -- 172.21.15.93:0/649701961 wait complete. 2023-11-16T16:46:35.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.821+0000 7f160f7ff700 1 Processor -- start 2023-11-16T16:46:35.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.821+0000 7f160f7ff700 1 -- start start 2023-11-16T16:46:35.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.821+0000 7f160f7ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16081024e0 0x7f1608198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:35.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.821+0000 7f160f7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16081036d0 0x7f1608198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:35.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.821+0000 7f160f7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1608104060 0x7f160819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:35.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160f7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160819fd90 con 0x7f1608104060 2023-11-16T16:46:35.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160f7ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160819fed0 con 0x7f16081024e0 2023-11-16T16:46:35.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160f7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16081a0060 con 0x7f16081036d0 2023-11-16T16:46:35.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160d59b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16081024e0 0x7f1608198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:35.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160d59b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16081024e0 0x7f1608198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:35256/0 (socket says 172.21.15.93:35256) 2023-11-16T16:46:35.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160d59b700 1 -- 172.21.15.93:0/3804701111 learned_addr learned my addr 172.21.15.93:0/3804701111 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:35.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160dd9c700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1608104060 0x7f160819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:35.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160cd9a700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16081036d0 0x7f1608198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:35.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160d59b700 1 -- 172.21.15.93:0/3804701111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16081036d0 msgr2=0x7f1608198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:35.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160d59b700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16081036d0 0x7f1608198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:35.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160d59b700 1 -- 172.21.15.93:0/3804701111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1608104060 msgr2=0x7f160819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:35.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160d59b700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1608104060 0x7f160819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:35.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160d59b700 1 -- 172.21.15.93:0/3804701111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16040095f0 con 0x7f16081024e0 2023-11-16T16:46:35.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160dd9c700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1608104060 0x7f160819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:35.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f160d59b700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16081024e0 0x7f1608198210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f16040095c0 tx=0x7f1604005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:35.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.822+0000 7f15fe7fc700 1 -- 172.21.15.93:0/3804701111 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160401d070 con 0x7f16081024e0 2023-11-16T16:46:35.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.823+0000 7f160f7ff700 1 -- 172.21.15.93:0/3804701111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160810de30 con 0x7f16081024e0 2023-11-16T16:46:35.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.823+0000 7f15fe7fc700 1 -- 172.21.15.93:0/3804701111 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1604022470 con 0x7f16081024e0 2023-11-16T16:46:35.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.823+0000 7f15fe7fc700 1 -- 172.21.15.93:0/3804701111 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604017650 con 0x7f16081024e0 2023-11-16T16:46:35.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.823+0000 7f160f7ff700 1 -- 172.21.15.93:0/3804701111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160810dca0 con 0x7f16081024e0 2023-11-16T16:46:35.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.824+0000 7f160f7ff700 1 -- 172.21.15.93:0/3804701111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1608109a80 con 0x7f16081024e0 2023-11-16T16:46:35.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.825+0000 7f15fe7fc700 1 -- 172.21.15.93:0/3804701111 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f16040225e0 con 0x7f16081024e0 2023-11-16T16:46:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:35 smithi125 ceph-mon[131076]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:35 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1098467049' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-16T16:46:35.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:35 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-16T16:46:35.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.827+0000 7f15fe7fc700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15f40a1150 0x7f15f40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:35.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.827+0000 7f15fe7fc700 1 -- 172.21.15.93:0/3804701111 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f16040cdca0 con 0x7f16081024e0 2023-11-16T16:46:35.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.827+0000 7f160cd9a700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15f40a1150 0x7f15f40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:35.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.827+0000 7f160cd9a700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15f40a1150 0x7f15f40a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009c60 tx=0x7f15f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:35.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:35.829+0000 7f15fe7fc700 1 -- 172.21.15.93:0/3804701111 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1604097710 con 0x7f16081024e0 2023-11-16T16:46:35.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:35 smithi093 ceph-mon[97863]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:35.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:35 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1098467049' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-16T16:46:35.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:35 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-16T16:46:35.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:35 smithi159 ceph-mon[131528]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:35.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:35 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1098467049' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-16T16:46:35.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:35 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-16T16:46:36.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.038+0000 7f160f7ff700 1 -- 172.21.15.93:0/3804701111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f160802cfe0 con 0x7f16081024e0 2023-11-16T16:46:36.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.584+0000 7f15fe7fc700 1 -- 172.21.15.93:0/3804701111 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) v1 ==== 171+0+0 (secure 0 0 0) 0x7f160409ad30 con 0x7f16081024e0 2023-11-16T16:46:36.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.593+0000 7f160f7ff700 1 -- 172.21.15.93:0/3804701111 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15f40a1150 msgr2=0x7f15f40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:36.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.593+0000 7f160f7ff700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15f40a1150 0x7f15f40a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009c60 tx=0x7f15f8009260 comp rx=0 tx=0).stop 2023-11-16T16:46:36.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.593+0000 7f160f7ff700 1 -- 172.21.15.93:0/3804701111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16081024e0 msgr2=0x7f1608198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:36.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.593+0000 7f160f7ff700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16081024e0 0x7f1608198210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f16040095c0 tx=0x7f1604005720 comp rx=0 tx=0).stop 2023-11-16T16:46:36.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.593+0000 7f160f7ff700 1 -- 172.21.15.93:0/3804701111 shutdown_connections 2023-11-16T16:46:36.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.593+0000 7f160f7ff700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15f40a1150 0x7f15f40a3600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:36.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.593+0000 7f160f7ff700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1608104060 0x7f160819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:36.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.593+0000 7f160f7ff700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16081024e0 0x7f1608198210 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f16040095c0 tx=0x7f1604005720 comp rx=0 tx=0).stop 2023-11-16T16:46:36.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.593+0000 7f160f7ff700 1 --2- 172.21.15.93:0/3804701111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16081036d0 0x7f1608198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:36.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.593+0000 7f160f7ff700 1 -- 172.21.15.93:0/3804701111 >> 172.21.15.93:0/3804701111 conn(0x7f16080fdb20 msgr2=0x7f1608106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.595+0000 7f160f7ff700 1 -- 172.21.15.93:0/3804701111 shutdown_connections 2023-11-16T16:46:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.595+0000 7f160f7ff700 1 -- 172.21.15.93:0/3804701111 wait complete. 2023-11-16T16:46:36.604 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_data' created 2023-11-16T16:46:36.629 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-16T16:46:36.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.851+0000 7f6bb57ff700 1 -- 172.21.15.93:0/4082658984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bb01024e0 msgr2=0x7f6bb01028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:36.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.851+0000 7f6bb57ff700 1 --2- 172.21.15.93:0/4082658984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bb01024e0 0x7f6bb01028f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009990 tx=0x7f6ba0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:36.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.852+0000 7f6bb57ff700 1 -- 172.21.15.93:0/4082658984 shutdown_connections 2023-11-16T16:46:36.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.852+0000 7f6bb57ff700 1 --2- 172.21.15.93:0/4082658984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bb01024e0 0x7f6bb01028f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:36.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.852+0000 7f6bb57ff700 1 --2- 172.21.15.93:0/4082658984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb0104060 0x7f6bb010a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:36.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.852+0000 7f6bb57ff700 1 --2- 172.21.15.93:0/4082658984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb01036d0 0x7f6bb0103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:36.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.852+0000 7f6bb57ff700 1 -- 172.21.15.93:0/4082658984 >> 172.21.15.93:0/4082658984 conn(0x7f6bb00fdb20 msgr2=0x7f6bb00fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:36.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.852+0000 7f6bb57ff700 1 -- 172.21.15.93:0/4082658984 shutdown_connections 2023-11-16T16:46:36.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.852+0000 7f6bb57ff700 1 -- 172.21.15.93:0/4082658984 wait complete. 2023-11-16T16:46:36.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6bb57ff700 1 Processor -- start 2023-11-16T16:46:36.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6bb57ff700 1 -- start start 2023-11-16T16:46:36.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6bb57ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bb01024e0 0x7f6bb0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:36.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6bb57ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb01036d0 0x7f6bb01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:36.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6bb57ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb0104060 0x7f6bb019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:36.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6bb57ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb019fe10 con 0x7f6bb01024e0 2023-11-16T16:46:36.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6bb57ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb019ff50 con 0x7f6bb01036d0 2023-11-16T16:46:36.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6bb57ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb01a00e0 con 0x7f6bb0104060 2023-11-16T16:46:36.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6baeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bb01024e0 0x7f6bb0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:36.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6ba7fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb01036d0 0x7f6bb01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:36.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6ba7fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb01036d0 0x7f6bb01987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:35278/0 (socket says 172.21.15.93:35278) 2023-11-16T16:46:36.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6ba7fff700 1 -- 172.21.15.93:0/1516356379 learned_addr learned my addr 172.21.15.93:0/1516356379 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:36.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.853+0000 7f6baf7fe700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb0104060 0x7f6bb019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:36.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.854+0000 7f6baeffd700 1 -- 172.21.15.93:0/1516356379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb0104060 msgr2=0x7f6bb019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:36.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.854+0000 7f6baeffd700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb0104060 0x7f6bb019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:36.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.854+0000 7f6baeffd700 1 -- 172.21.15.93:0/1516356379 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb01036d0 msgr2=0x7f6bb01987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:36.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.854+0000 7f6baeffd700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb01036d0 0x7f6bb01987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:36.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.854+0000 7f6baeffd700 1 -- 172.21.15.93:0/1516356379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba00095f0 con 0x7f6bb01024e0 2023-11-16T16:46:36.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.854+0000 7f6baeffd700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bb01024e0 0x7f6bb0198290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00098e0 tx=0x7f6ba0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:36.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.854+0000 7f6bacff9700 1 -- 172.21.15.93:0/1516356379 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba001d070 con 0x7f6bb01024e0 2023-11-16T16:46:36.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.854+0000 7f6bacff9700 1 -- 172.21.15.93:0/1516356379 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ba0021410 con 0x7f6bb01024e0 2023-11-16T16:46:36.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.854+0000 7f6bb57ff700 1 -- 172.21.15.93:0/1516356379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb010de90 con 0x7f6bb01024e0 2023-11-16T16:46:36.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.854+0000 7f6bacff9700 1 -- 172.21.15.93:0/1516356379 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0017650 con 0x7f6bb01024e0 2023-11-16T16:46:36.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.854+0000 7f6bb57ff700 1 -- 172.21.15.93:0/1516356379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb010dd00 con 0x7f6bb01024e0 2023-11-16T16:46:36.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.855+0000 7f6bb57ff700 1 -- 172.21.15.93:0/1516356379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb0109a70 con 0x7f6bb01024e0 2023-11-16T16:46:36.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.857+0000 7f6bacff9700 1 -- 172.21.15.93:0/1516356379 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6ba0021580 con 0x7f6bb01024e0 2023-11-16T16:46:36.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.859+0000 7f6bacff9700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6b9c0a11d0 0x7f6b9c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:36.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.860+0000 7f6bacff9700 1 -- 172.21.15.93:0/1516356379 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f6ba00cd9b0 con 0x7f6bb01024e0 2023-11-16T16:46:36.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.860+0000 7f6ba7fff700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6b9c0a11d0 0x7f6b9c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:36.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.860+0000 7f6ba7fff700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6b9c0a11d0 0x7f6b9c0a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009d20 tx=0x7f6b98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:36.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:36.864+0000 7f6bacff9700 1 -- 172.21.15.93:0/1516356379 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6ba00972a0 con 0x7f6bb01024e0 2023-11-16T16:46:36.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:36 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-16T16:46:36.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:36 smithi093 ceph-mon[97863]: osdmap e81: 12 total, 12 up, 12 in 2023-11-16T16:46:36.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:36 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3804701111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-16T16:46:36.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:36 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-16T16:46:36.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:36 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:36.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:36 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:36.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:36 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:36.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:36 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:36.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:36 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-16T16:46:36.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:36 smithi159 ceph-mon[131528]: osdmap e81: 12 total, 12 up, 12 in 2023-11-16T16:46:36.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:36 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3804701111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-16T16:46:36.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:36 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-16T16:46:36.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:36 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:36.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:36 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:36.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:36 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:36.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:36 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:37.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:36 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-16T16:46:37.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:36 smithi125 ceph-mon[131076]: osdmap e81: 12 total, 12 up, 12 in 2023-11-16T16:46:37.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:36 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3804701111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-16T16:46:37.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:36 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-16T16:46:37.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:36 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:37.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:36 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:37.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:36 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:37.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:36 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:37.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.122+0000 7f6bb57ff700 1 -- 172.21.15.93:0/1516356379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6bb002cfe0 con 0x7f6bb01024e0 2023-11-16T16:46:37.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.130+0000 7f6bacff9700 1 -- 172.21.15.93:0/1516356379 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6ba009a8c0 con 0x7f6bb01024e0 2023-11-16T16:46:37.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.135+0000 7f6bb57ff700 1 -- 172.21.15.93:0/1516356379 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6b9c0a11d0 msgr2=0x7f6b9c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:37.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.135+0000 7f6bb57ff700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6b9c0a11d0 0x7f6b9c0a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009d20 tx=0x7f6b98009260 comp rx=0 tx=0).stop 2023-11-16T16:46:37.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.135+0000 7f6bb57ff700 1 -- 172.21.15.93:0/1516356379 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bb01024e0 msgr2=0x7f6bb0198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:37.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.135+0000 7f6bb57ff700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bb01024e0 0x7f6bb0198290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00098e0 tx=0x7f6ba0005740 comp rx=0 tx=0).stop 2023-11-16T16:46:37.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.136+0000 7f6bb57ff700 1 -- 172.21.15.93:0/1516356379 shutdown_connections 2023-11-16T16:46:37.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.136+0000 7f6bb57ff700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6b9c0a11d0 0x7f6b9c0a3680 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:37.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.136+0000 7f6bb57ff700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bb01024e0 0x7f6bb0198290 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:37.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.136+0000 7f6bb57ff700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb0104060 0x7f6bb019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:37.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.136+0000 7f6bb57ff700 1 --2- 172.21.15.93:0/1516356379 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb01036d0 0x7f6bb01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:37.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.136+0000 7f6bb57ff700 1 -- 172.21.15.93:0/1516356379 >> 172.21.15.93:0/1516356379 conn(0x7f6bb00fdb20 msgr2=0x7f6bb0106bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:37.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.138+0000 7f6bb57ff700 1 -- 172.21.15.93:0/1516356379 shutdown_connections 2023-11-16T16:46:37.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.138+0000 7f6bb57ff700 1 -- 172.21.15.93:0/1516356379 wait complete. 2023-11-16T16:46:37.142 INFO:teuthology.orchestra.run.smithi093.stderr:new fs with metadata pool 2 and data pool 3 2023-11-16T16:46:37.175 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-16T16:46:37.175 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-16T16:46:37.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.391+0000 7f07353ff700 1 -- 172.21.15.93:0/3135019094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07301002d0 msgr2=0x7f07301006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:37.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.391+0000 7f07353ff700 1 --2- 172.21.15.93:0/3135019094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07301002d0 0x7f07301006e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f072000b0a0 tx=0x7f072000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:46:37.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.391+0000 7f07353ff700 1 -- 172.21.15.93:0/3135019094 shutdown_connections 2023-11-16T16:46:37.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.391+0000 7f07353ff700 1 --2- 172.21.15.93:0/3135019094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07301002d0 0x7f07301006e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:37.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.391+0000 7f07353ff700 1 --2- 172.21.15.93:0/3135019094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0730101e50 0x7f0730108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:37.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.391+0000 7f07353ff700 1 --2- 172.21.15.93:0/3135019094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07301014c0 0x7f0730101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:37.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.391+0000 7f07353ff700 1 -- 172.21.15.93:0/3135019094 >> 172.21.15.93:0/3135019094 conn(0x7f07300fb8d0 msgr2=0x7f07300fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:37.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.392+0000 7f07353ff700 1 -- 172.21.15.93:0/3135019094 shutdown_connections 2023-11-16T16:46:37.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.392+0000 7f07353ff700 1 -- 172.21.15.93:0/3135019094 wait complete. 2023-11-16T16:46:37.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f07353ff700 1 Processor -- start 2023-11-16T16:46:37.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f07353ff700 1 -- start start 2023-11-16T16:46:37.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f07353ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07301002d0 0x7f07300748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:37.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f07353ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07301014c0 0x7f0730072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:37.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f07353ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0730101e50 0x7f0730073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:37.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f07353ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0730073bc0 con 0x7f0730101e50 2023-11-16T16:46:37.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f07353ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0730073d30 con 0x7f07301014c0 2023-11-16T16:46:37.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f07353ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07301a0240 con 0x7f07301002d0 2023-11-16T16:46:37.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f072f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0730101e50 0x7f0730073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:37.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f072e7fc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07301014c0 0x7f0730072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:37.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f072f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0730101e50 0x7f0730073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58782/0 (socket says 172.21.15.93:58782) 2023-11-16T16:46:37.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f072e7fc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07301014c0 0x7f0730072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:35062/0 (socket says 172.21.15.93:35062) 2023-11-16T16:46:37.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.393+0000 7f072f7fe700 1 -- 172.21.15.93:0/735803479 learned_addr learned my addr 172.21.15.93:0/735803479 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:37.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.394+0000 7f072effd700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07301002d0 0x7f07300748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:37.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.394+0000 7f072e7fc700 1 -- 172.21.15.93:0/735803479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07301002d0 msgr2=0x7f07300748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:37.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.394+0000 7f072e7fc700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07301002d0 0x7f07300748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:37.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.394+0000 7f072e7fc700 1 -- 172.21.15.93:0/735803479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0730101e50 msgr2=0x7f0730073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:37.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.394+0000 7f072e7fc700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0730101e50 0x7f0730073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:37.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.394+0000 7f072e7fc700 1 -- 172.21.15.93:0/735803479 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720009d00 con 0x7f07301014c0 2023-11-16T16:46:37.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.394+0000 7f072e7fc700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07301014c0 0x7f0730072ee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f072400e9f0 tx=0x7f072400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:37.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.394+0000 7f0717fff700 1 -- 172.21.15.93:0/735803479 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072400c9c0 con 0x7f07301014c0 2023-11-16T16:46:37.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.394+0000 7f0717fff700 1 -- 172.21.15.93:0/735803479 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0724010430 con 0x7f07301014c0 2023-11-16T16:46:37.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.394+0000 7f07353ff700 1 -- 172.21.15.93:0/735803479 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073010bca0 con 0x7f07301014c0 2023-11-16T16:46:37.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.394+0000 7f0717fff700 1 -- 172.21.15.93:0/735803479 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724018680 con 0x7f07301014c0 2023-11-16T16:46:37.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.395+0000 7f07353ff700 1 -- 172.21.15.93:0/735803479 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073010bb10 con 0x7f07301014c0 2023-11-16T16:46:37.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.395+0000 7f07353ff700 1 -- 172.21.15.93:0/735803479 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07301a0930 con 0x7f07301014c0 2023-11-16T16:46:37.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.397+0000 7f0717fff700 1 -- 172.21.15.93:0/735803479 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0724010860 con 0x7f07301014c0 2023-11-16T16:46:37.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.399+0000 7f0717fff700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f07180a1150 0x7f07180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:37.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.399+0000 7f0717fff700 1 -- 172.21.15.93:0/735803479 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0724014070 con 0x7f07301014c0 2023-11-16T16:46:37.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.399+0000 7f072effd700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f07180a1150 0x7f07180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:37.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.400+0000 7f072effd700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f07180a1150 0x7f07180a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f07200079f0 tx=0x7f072000b740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:37.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.401+0000 7f0717fff700 1 -- 172.21.15.93:0/735803479 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0724096130 con 0x7f07301014c0 2023-11-16T16:46:37.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:37.628+0000 7f07353ff700 1 -- 172.21.15.93:0/735803479 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f0730107180 con 0x7f07301014c0 2023-11-16T16:46:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: pgmap v274: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-11-16T16:46:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: osdmap e82: 12 total, 12 up, 12 in 2023-11-16T16:46:37.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1516356379' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-16T16:46:37.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-16T16:46:37.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: osdmap e83: 12 total, 12 up, 12 in 2023-11-16T16:46:37.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1516356379' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-16T16:46:37.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-16T16:46:37.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-16T16:46:37.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:37.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:37.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:37.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:46:37.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:37 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: pgmap v274: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-11-16T16:46:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: osdmap e82: 12 total, 12 up, 12 in 2023-11-16T16:46:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1516356379' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-16T16:46:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-16T16:46:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: osdmap e83: 12 total, 12 up, 12 in 2023-11-16T16:46:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1516356379' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-16T16:46:37.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-16T16:46:37.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-16T16:46:37.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:37.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:37.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:37.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:46:37.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:37 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:38.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: pgmap v274: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:38.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-11-16T16:46:38.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: osdmap e82: 12 total, 12 up, 12 in 2023-11-16T16:46:38.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1516356379' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-16T16:46:38.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-16T16:46:38.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: osdmap e83: 12 total, 12 up, 12 in 2023-11-16T16:46:38.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1516356379' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-16T16:46:38.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-16T16:46:38.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-16T16:46:38.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:38.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:38.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:38.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:46:38.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:37 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:46:38.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.136+0000 7f0717fff700 1 -- 172.21.15.93:0/735803479 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) v1 ==== 157+0+0 (secure 0 0 0) 0x7f0724099750 con 0x7f07301014c0 2023-11-16T16:46:38.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.138+0000 7f07353ff700 1 -- 172.21.15.93:0/735803479 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f07180a1150 msgr2=0x7f07180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:38.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.138+0000 7f07353ff700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f07180a1150 0x7f07180a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f07200079f0 tx=0x7f072000b740 comp rx=0 tx=0).stop 2023-11-16T16:46:38.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.138+0000 7f07353ff700 1 -- 172.21.15.93:0/735803479 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07301014c0 msgr2=0x7f0730072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:38.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.138+0000 7f07353ff700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07301014c0 0x7f0730072ee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f072400e9f0 tx=0x7f072400ed00 comp rx=0 tx=0).stop 2023-11-16T16:46:38.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.138+0000 7f07353ff700 1 -- 172.21.15.93:0/735803479 shutdown_connections 2023-11-16T16:46:38.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.138+0000 7f07353ff700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f07180a1150 0x7f07180a3600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:38.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.138+0000 7f07353ff700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0730101e50 0x7f0730073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:38.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.138+0000 7f07353ff700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07301002d0 0x7f07300748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:38.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.138+0000 7f07353ff700 1 --2- 172.21.15.93:0/735803479 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07301014c0 0x7f0730072ee0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:38.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.138+0000 7f07353ff700 1 -- 172.21.15.93:0/735803479 >> 172.21.15.93:0/735803479 conn(0x7f07300fb8d0 msgr2=0x7f07300fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:38.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.138+0000 7f07353ff700 1 -- 172.21.15.93:0/735803479 shutdown_connections 2023-11-16T16:46:38.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.139+0000 7f07353ff700 1 -- 172.21.15.93:0/735803479 wait complete. 2023-11-16T16:46:38.160 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-16T16:46:38.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.363+0000 7f9287fff700 1 -- 172.21.15.93:0/420869650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f928010c8a0 msgr2=0x7f928010ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:38.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.363+0000 7f9287fff700 1 --2- 172.21.15.93:0/420869650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f928010c8a0 0x7f928010ec80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f927800b0a0 tx=0x7f927800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:46:38.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.363+0000 7f9287fff700 1 -- 172.21.15.93:0/420869650 shutdown_connections 2023-11-16T16:46:38.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.363+0000 7f9287fff700 1 --2- 172.21.15.93:0/420869650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f928010c8a0 0x7f928010ec80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:38.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.363+0000 7f9287fff700 1 --2- 172.21.15.93:0/420869650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9280074b30 0x7f9280072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:38.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.363+0000 7f9287fff700 1 --2- 172.21.15.93:0/420869650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9280073550 0x7f92800739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:38.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.363+0000 7f9287fff700 1 -- 172.21.15.93:0/420869650 >> 172.21.15.93:0/420869650 conn(0x7f92800fb8b0 msgr2=0x7f92800fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:38.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.364+0000 7f9287fff700 1 -- 172.21.15.93:0/420869650 shutdown_connections 2023-11-16T16:46:38.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.364+0000 7f9287fff700 1 -- 172.21.15.93:0/420869650 wait complete. 2023-11-16T16:46:38.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.365+0000 7f9287fff700 1 Processor -- start 2023-11-16T16:46:38.367 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.365+0000 7f9287fff700 1 -- start start 2023-11-16T16:46:38.368 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f9287fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9280073550 0x7f928019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:38.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f9287fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9280074b30 0x7f928019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:38.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f9287fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f928010c8a0 0x7f92801a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:38.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f9287fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92801a4300 con 0x7f928010c8a0 2023-11-16T16:46:38.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f9287fff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92801a4440 con 0x7f9280074b30 2023-11-16T16:46:38.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f9287fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92801a45d0 con 0x7f9280073550 2023-11-16T16:46:38.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f928659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f928010c8a0 0x7f92801a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:38.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f928559a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9280074b30 0x7f928019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:38.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f9285d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9280073550 0x7f928019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:38.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f928559a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9280074b30 0x7f928019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:35080/0 (socket says 172.21.15.93:35080) 2023-11-16T16:46:38.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f928559a700 1 -- 172.21.15.93:0/1577875521 learned_addr learned my addr 172.21.15.93:0/1577875521 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:38.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f928559a700 1 -- 172.21.15.93:0/1577875521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9280073550 msgr2=0x7f928019c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:38.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f928559a700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9280073550 0x7f928019c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:38.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f928559a700 1 -- 172.21.15.93:0/1577875521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f928010c8a0 msgr2=0x7f92801a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:38.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f928559a700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f928010c8a0 0x7f92801a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:38.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f928559a700 1 -- 172.21.15.93:0/1577875521 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c009520 con 0x7f9280074b30 2023-11-16T16:46:38.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.366+0000 7f928659c700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f928010c8a0 0x7f92801a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:38.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.367+0000 7f928559a700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9280074b30 0x7f928019ccc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f927000e970 tx=0x7f927000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:38.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.367+0000 7f9276ffd700 1 -- 172.21.15.93:0/1577875521 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270009790 con 0x7f9280074b30 2023-11-16T16:46:38.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.367+0000 7f9276ffd700 1 -- 172.21.15.93:0/1577875521 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f927000c7e0 con 0x7f9280074b30 2023-11-16T16:46:38.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.367+0000 7f9287fff700 1 -- 172.21.15.93:0/1577875521 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9278009d00 con 0x7f9280074b30 2023-11-16T16:46:38.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.367+0000 7f9276ffd700 1 -- 172.21.15.93:0/1577875521 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270003c70 con 0x7f9280074b30 2023-11-16T16:46:38.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.367+0000 7f9287fff700 1 -- 172.21.15.93:0/1577875521 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92801122b0 con 0x7f9280074b30 2023-11-16T16:46:38.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.368+0000 7f9287fff700 1 -- 172.21.15.93:0/1577875521 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92801023f0 con 0x7f9280074b30 2023-11-16T16:46:38.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.370+0000 7f9276ffd700 1 -- 172.21.15.93:0/1577875521 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f927000cc50 con 0x7f9280074b30 2023-11-16T16:46:38.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.371+0000 7f9276ffd700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f926c0a1150 0x7f926c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:38.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.371+0000 7f9276ffd700 1 -- 172.21.15.93:0/1577875521 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f9270014070 con 0x7f9280074b30 2023-11-16T16:46:38.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.372+0000 7f9285d9b700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f926c0a1150 0x7f926c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:38.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.372+0000 7f9285d9b700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f926c0a1150 0x7f926c0a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9280074f00 tx=0x7f927c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:38.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.373+0000 7f9276ffd700 1 -- 172.21.15.93:0/1577875521 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9270096370 con 0x7f9280074b30 2023-11-16T16:46:38.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:38.571+0000 7f9287fff700 1 -- 172.21.15.93:0/1577875521 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9280067b00 con 0x7f9280074b30 2023-11-16T16:46:38.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:46:38.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:46:38.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:46:38.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: Service mds.all has no related daemon already placed. 2023-11-16T16:46:38.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/735803479' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-16T16:46:38.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-16T16:46:38.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] up:active 2023-11-16T16:46:38.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:46:38.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-11-16T16:46:38.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: osdmap e84: 12 total, 12 up, 12 in 2023-11-16T16:46:38.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:46:38.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1577875521' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-16T16:46:38.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:38 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-16T16:46:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:46:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:46:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:46:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: Service mds.all has no related daemon already placed. 2023-11-16T16:46:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/735803479' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-16T16:46:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-16T16:46:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] up:active 2023-11-16T16:46:38.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:46:38.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-11-16T16:46:38.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: osdmap e84: 12 total, 12 up, 12 in 2023-11-16T16:46:38.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:46:38.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1577875521' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-16T16:46:38.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:38 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-16T16:46:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:46:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:46:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:46:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: Service mds.all has no related daemon already placed. 2023-11-16T16:46:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/735803479' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-16T16:46:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-16T16:46:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] up:active 2023-11-16T16:46:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:46:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-11-16T16:46:39.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: osdmap e84: 12 total, 12 up, 12 in 2023-11-16T16:46:39.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:46:39.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1577875521' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-16T16:46:39.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:38 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-16T16:46:40.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.154+0000 7f9276ffd700 1 -- 172.21.15.93:0/1577875521 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) v1 ==== 228+0+0 (secure 0 0 0) 0x7f9270099990 con 0x7f9280074b30 2023-11-16T16:46:40.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.159+0000 7f9287fff700 1 -- 172.21.15.93:0/1577875521 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f926c0a1150 msgr2=0x7f926c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:40.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.159+0000 7f9287fff700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f926c0a1150 0x7f926c0a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9280074f00 tx=0x7f927c009260 comp rx=0 tx=0).stop 2023-11-16T16:46:40.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.159+0000 7f9287fff700 1 -- 172.21.15.93:0/1577875521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9280074b30 msgr2=0x7f928019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:40.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.159+0000 7f9287fff700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9280074b30 0x7f928019ccc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f927000e970 tx=0x7f927000ec80 comp rx=0 tx=0).stop 2023-11-16T16:46:40.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.159+0000 7f9287fff700 1 -- 172.21.15.93:0/1577875521 shutdown_connections 2023-11-16T16:46:40.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.159+0000 7f9287fff700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f926c0a1150 0x7f926c0a3600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:40.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.159+0000 7f9287fff700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f928010c8a0 0x7f92801a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:40.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.159+0000 7f9287fff700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9280073550 0x7f928019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:40.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.159+0000 7f9287fff700 1 --2- 172.21.15.93:0/1577875521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9280074b30 0x7f928019ccc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:40.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.159+0000 7f9287fff700 1 -- 172.21.15.93:0/1577875521 >> 172.21.15.93:0/1577875521 conn(0x7f92800fb8b0 msgr2=0x7f92800fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:40.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.159+0000 7f9287fff700 1 -- 172.21.15.93:0/1577875521 shutdown_connections 2023-11-16T16:46:40.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.159+0000 7f9287fff700 1 -- 172.21.15.93:0/1577875521 wait complete. 2023-11-16T16:46:40.162 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_data_ec' created 2023-11-16T16:46:40.180 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-16T16:46:40.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.381+0000 7fd681bff700 1 -- 172.21.15.93:0/2286437280 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd67c1014b0 msgr2=0x7fd67c101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:40.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.381+0000 7fd681bff700 1 --2- 172.21.15.93:0/2286437280 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd67c1014b0 0x7fd67c101900 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd664009990 tx=0x7fd664009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:40.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.381+0000 7fd681bff700 1 -- 172.21.15.93:0/2286437280 shutdown_connections 2023-11-16T16:46:40.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.381+0000 7fd681bff700 1 --2- 172.21.15.93:0/2286437280 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd67c101e40 0x7fd67c108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:40.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.381+0000 7fd681bff700 1 --2- 172.21.15.93:0/2286437280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd67c1002c0 0x7fd67c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:40.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.381+0000 7fd681bff700 1 --2- 172.21.15.93:0/2286437280 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd67c1014b0 0x7fd67c101900 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:40.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.381+0000 7fd681bff700 1 -- 172.21.15.93:0/2286437280 >> 172.21.15.93:0/2286437280 conn(0x7fd67c0fb8a0 msgr2=0x7fd67c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:40.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.381+0000 7fd681bff700 1 -- 172.21.15.93:0/2286437280 shutdown_connections 2023-11-16T16:46:40.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.381+0000 7fd681bff700 1 -- 172.21.15.93:0/2286437280 wait complete. 2023-11-16T16:46:40.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.382+0000 7fd681bff700 1 Processor -- start 2023-11-16T16:46:40.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.382+0000 7fd681bff700 1 -- start start 2023-11-16T16:46:40.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.382+0000 7fd681bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd67c1002c0 0x7fd67c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:40.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.382+0000 7fd681bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd67c1014b0 0x7fd67c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:40.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.382+0000 7fd681bff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd67c101e40 0x7fd67c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:40.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.382+0000 7fd681bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c19fdc0 con 0x7fd67c1002c0 2023-11-16T16:46:40.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.382+0000 7fd681bff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c19ff00 con 0x7fd67c101e40 2023-11-16T16:46:40.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.382+0000 7fd681bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c1a0090 con 0x7fd67c1014b0 2023-11-16T16:46:40.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.382+0000 7fd67b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd67c1002c0 0x7fd67c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:40.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.382+0000 7fd67b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd67c1002c0 0x7fd67c198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58830/0 (socket says 172.21.15.93:58830) 2023-11-16T16:46:40.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.382+0000 7fd67b7fe700 1 -- 172.21.15.93:0/213682084 learned_addr learned my addr 172.21.15.93:0/213682084 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:40.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd67bfff700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd67c101e40 0x7fd67c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:40.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd673fff700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd67c1014b0 0x7fd67c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:40.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd67b7fe700 1 -- 172.21.15.93:0/213682084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd67c1014b0 msgr2=0x7fd67c198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:40.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd67b7fe700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd67c1014b0 0x7fd67c198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:40.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd67b7fe700 1 -- 172.21.15.93:0/213682084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd67c101e40 msgr2=0x7fd67c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:40.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd67b7fe700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd67c101e40 0x7fd67c19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:40.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd67b7fe700 1 -- 172.21.15.93:0/213682084 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6640095f0 con 0x7fd67c1002c0 2023-11-16T16:46:40.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd673fff700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd67c1014b0 0x7fd67c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:40.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd67bfff700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd67c101e40 0x7fd67c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:40.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd67b7fe700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd67c1002c0 0x7fd67c198240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00e970 tx=0x7fd66c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:40.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd6797fa700 1 -- 172.21.15.93:0/213682084 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c009790 con 0x7fd67c1002c0 2023-11-16T16:46:40.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd681bff700 1 -- 172.21.15.93:0/213682084 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67c10bc70 con 0x7fd67c1002c0 2023-11-16T16:46:40.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd6797fa700 1 -- 172.21.15.93:0/213682084 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd66c0044e0 con 0x7fd67c1002c0 2023-11-16T16:46:40.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd6797fa700 1 -- 172.21.15.93:0/213682084 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c003e30 con 0x7fd67c1002c0 2023-11-16T16:46:40.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.383+0000 7fd681bff700 1 -- 172.21.15.93:0/213682084 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67c10bae0 con 0x7fd67c1002c0 2023-11-16T16:46:40.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.384+0000 7fd681bff700 1 -- 172.21.15.93:0/213682084 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67c1929d0 con 0x7fd67c1002c0 2023-11-16T16:46:40.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.388+0000 7fd6797fa700 1 -- 172.21.15.93:0/213682084 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd66c004000 con 0x7fd67c1002c0 2023-11-16T16:46:40.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.389+0000 7fd6797fa700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd6680a1150 0x7fd6680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:40.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.389+0000 7fd6797fa700 1 -- 172.21.15.93:0/213682084 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fd66c014070 con 0x7fd67c1002c0 2023-11-16T16:46:40.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.389+0000 7fd673fff700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd6680a1150 0x7fd6680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:40.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.390+0000 7fd673fff700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd6680a1150 0x7fd6680a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd66400b2e0 tx=0x7fd664005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:40.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.392+0000 7fd6797fa700 1 -- 172.21.15.93:0/213682084 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd66c0964a0 con 0x7fd67c1002c0 2023-11-16T16:46:40.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:40 smithi093 ceph-mon[97863]: pgmap v278: 97 pgs: 9 creating+peering, 26 active+clean, 62 unknown; 577 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2023-11-16T16:46:40.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:40 smithi093 ceph-mon[97863]: osdmap e85: 12 total, 12 up, 12 in 2023-11-16T16:46:40.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:40 smithi159 ceph-mon[131528]: pgmap v278: 97 pgs: 9 creating+peering, 26 active+clean, 62 unknown; 577 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2023-11-16T16:46:40.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:40 smithi159 ceph-mon[131528]: osdmap e85: 12 total, 12 up, 12 in 2023-11-16T16:46:40.577 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:46:40 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-b[156540]: 2023-11-16T16:46:40.154+0000 7fb7a87ca700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-16T16:46:40.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:40 smithi125 ceph-mon[131076]: pgmap v278: 97 pgs: 9 creating+peering, 26 active+clean, 62 unknown; 577 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2023-11-16T16:46:40.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:40 smithi125 ceph-mon[131076]: osdmap e85: 12 total, 12 up, 12 in 2023-11-16T16:46:40.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:40.590+0000 7fd681bff700 1 -- 172.21.15.93:0/213682084 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fd67c02cfe0 con 0x7fd67c1002c0 2023-11-16T16:46:41.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.151+0000 7fd6797fa700 1 -- 172.21.15.93:0/213682084 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd66c01f340 con 0x7fd67c1002c0 2023-11-16T16:46:41.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.156+0000 7fd681bff700 1 -- 172.21.15.93:0/213682084 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd6680a1150 msgr2=0x7fd6680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:41.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.156+0000 7fd681bff700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd6680a1150 0x7fd6680a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd66400b2e0 tx=0x7fd664005fd0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.156+0000 7fd681bff700 1 -- 172.21.15.93:0/213682084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd67c1002c0 msgr2=0x7fd67c198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:41.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.156+0000 7fd681bff700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd67c1002c0 0x7fd67c198240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00e970 tx=0x7fd66c00ed30 comp rx=0 tx=0).stop 2023-11-16T16:46:41.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.156+0000 7fd681bff700 1 -- 172.21.15.93:0/213682084 shutdown_connections 2023-11-16T16:46:41.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.156+0000 7fd681bff700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd6680a1150 0x7fd6680a3600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.156+0000 7fd681bff700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd67c1002c0 0x7fd67c198240 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.156+0000 7fd681bff700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd67c101e40 0x7fd67c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.156+0000 7fd681bff700 1 --2- 172.21.15.93:0/213682084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd67c1014b0 0x7fd67c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.156+0000 7fd681bff700 1 -- 172.21.15.93:0/213682084 >> 172.21.15.93:0/213682084 conn(0x7fd67c0fb8a0 msgr2=0x7fd67c106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:41.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.156+0000 7fd681bff700 1 -- 172.21.15.93:0/213682084 shutdown_connections 2023-11-16T16:46:41.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.156+0000 7fd681bff700 1 -- 172.21.15.93:0/213682084 wait complete. 2023-11-16T16:46:41.159 INFO:teuthology.orchestra.run.smithi093.stderr:set pool 4 allow_ec_overwrites to true 2023-11-16T16:46:41.191 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-16T16:46:41.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.392+0000 7fea5f3ff700 1 -- 172.21.15.93:0/1939406393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea581014c0 msgr2=0x7fea58101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:41.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.392+0000 7fea5f3ff700 1 --2- 172.21.15.93:0/1939406393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea581014c0 0x7fea58101910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fea48009990 tx=0x7fea48009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.392+0000 7fea5f3ff700 1 -- 172.21.15.93:0/1939406393 shutdown_connections 2023-11-16T16:46:41.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.392+0000 7fea5f3ff700 1 --2- 172.21.15.93:0/1939406393 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea58101e50 0x7fea58108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.392+0000 7fea5f3ff700 1 --2- 172.21.15.93:0/1939406393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea581002d0 0x7fea581006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.392+0000 7fea5f3ff700 1 --2- 172.21.15.93:0/1939406393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea581014c0 0x7fea58101910 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.392+0000 7fea5f3ff700 1 -- 172.21.15.93:0/1939406393 >> 172.21.15.93:0/1939406393 conn(0x7fea580fb8b0 msgr2=0x7fea580fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:41.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.392+0000 7fea5f3ff700 1 -- 172.21.15.93:0/1939406393 shutdown_connections 2023-11-16T16:46:41.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.392+0000 7fea5f3ff700 1 -- 172.21.15.93:0/1939406393 wait complete. 2023-11-16T16:46:41.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.393+0000 7fea5f3ff700 1 Processor -- start 2023-11-16T16:46:41.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.393+0000 7fea5f3ff700 1 -- start start 2023-11-16T16:46:41.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.393+0000 7fea5f3ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea581002d0 0x7fea580748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:41.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.393+0000 7fea5f3ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea581014c0 0x7fea58072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:41.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.393+0000 7fea5f3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea58101e50 0x7fea58073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:41.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.393+0000 7fea5f3ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea58073bc0 con 0x7fea581002d0 2023-11-16T16:46:41.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.393+0000 7fea5f3ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea58073d30 con 0x7fea581014c0 2023-11-16T16:46:41.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.393+0000 7fea5f3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea581a0090 con 0x7fea58101e50 2023-11-16T16:46:41.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea5d19b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea581002d0 0x7fea580748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:41.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea5d19b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea581002d0 0x7fea580748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58852/0 (socket says 172.21.15.93:58852) 2023-11-16T16:46:41.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea5d99c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea58101e50 0x7fea58073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:41.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea5d19b700 1 -- 172.21.15.93:0/2491807227 learned_addr learned my addr 172.21.15.93:0/2491807227 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:41.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea5c99a700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea581014c0 0x7fea58072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:41.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea5d19b700 1 -- 172.21.15.93:0/2491807227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea58101e50 msgr2=0x7fea58073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:41.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea5d19b700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea58101e50 0x7fea58073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea5d19b700 1 -- 172.21.15.93:0/2491807227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea581014c0 msgr2=0x7fea58072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:41.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea5d19b700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea581014c0 0x7fea58072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea5d19b700 1 -- 172.21.15.93:0/2491807227 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea480095f0 con 0x7fea581002d0 2023-11-16T16:46:41.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea5d19b700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea581002d0 0x7fea580748a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea5400e970 tx=0x7fea5400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:41.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea4e7fc700 1 -- 172.21.15.93:0/2491807227 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54009790 con 0x7fea581002d0 2023-11-16T16:46:41.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.394+0000 7fea4e7fc700 1 -- 172.21.15.93:0/2491807227 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea5400c7e0 con 0x7fea581002d0 2023-11-16T16:46:41.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.395+0000 7fea5f3ff700 1 -- 172.21.15.93:0/2491807227 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5810bc30 con 0x7fea581002d0 2023-11-16T16:46:41.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.395+0000 7fea4e7fc700 1 -- 172.21.15.93:0/2491807227 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54003c70 con 0x7fea581002d0 2023-11-16T16:46:41.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.395+0000 7fea5f3ff700 1 -- 172.21.15.93:0/2491807227 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5810baa0 con 0x7fea581002d0 2023-11-16T16:46:41.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.396+0000 7fea5f3ff700 1 -- 172.21.15.93:0/2491807227 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea58107200 con 0x7fea581002d0 2023-11-16T16:46:41.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.397+0000 7fea4e7fc700 1 -- 172.21.15.93:0/2491807227 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fea5400cc50 con 0x7fea581002d0 2023-11-16T16:46:41.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.399+0000 7fea4e7fc700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fea440a1150 0x7fea440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:41.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.399+0000 7fea4e7fc700 1 -- 172.21.15.93:0/2491807227 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fea54014070 con 0x7fea581002d0 2023-11-16T16:46:41.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.399+0000 7fea5c99a700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fea440a1150 0x7fea440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:41.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.400+0000 7fea5c99a700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fea440a1150 0x7fea440a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea4800b2e0 tx=0x7fea48005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:41.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.403+0000 7fea4e7fc700 1 -- 172.21.15.93:0/2491807227 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fea54096210 con 0x7fea581002d0 2023-11-16T16:46:41.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:41 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-16T16:46:41.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:41 smithi159 ceph-mon[131528]: osdmap e86: 12 total, 12 up, 12 in 2023-11-16T16:46:41.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:41 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/213682084' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-11-16T16:46:41.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:41 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-16T16:46:41.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:41 smithi125 ceph-mon[131076]: osdmap e86: 12 total, 12 up, 12 in 2023-11-16T16:46:41.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:41 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/213682084' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-11-16T16:46:41.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:41 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-16T16:46:41.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:41 smithi093 ceph-mon[97863]: osdmap e86: 12 total, 12 up, 12 in 2023-11-16T16:46:41.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:41 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/213682084' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-11-16T16:46:41.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.649+0000 7fea5f3ff700 1 -- 172.21.15.93:0/2491807227 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fea5802cfe0 con 0x7fea581002d0 2023-11-16T16:46:41.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.658+0000 7fea4e7fc700 1 -- 172.21.15.93:0/2491807227 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fea54003dd0 con 0x7fea581002d0 2023-11-16T16:46:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.661+0000 7fea5f3ff700 1 -- 172.21.15.93:0/2491807227 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fea440a1150 msgr2=0x7fea440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.661+0000 7fea5f3ff700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fea440a1150 0x7fea440a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea4800b2e0 tx=0x7fea48005fb0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.661+0000 7fea5f3ff700 1 -- 172.21.15.93:0/2491807227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea581002d0 msgr2=0x7fea580748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.661+0000 7fea5f3ff700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea581002d0 0x7fea580748a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea5400e970 tx=0x7fea5400ec80 comp rx=0 tx=0).stop 2023-11-16T16:46:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.661+0000 7fea5f3ff700 1 -- 172.21.15.93:0/2491807227 shutdown_connections 2023-11-16T16:46:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.661+0000 7fea5f3ff700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fea440a1150 0x7fea440a3600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.661+0000 7fea5f3ff700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea581002d0 0x7fea580748a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.661+0000 7fea5f3ff700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea58101e50 0x7fea58073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.661+0000 7fea5f3ff700 1 --2- 172.21.15.93:0/2491807227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea581014c0 0x7fea58072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.661+0000 7fea5f3ff700 1 -- 172.21.15.93:0/2491807227 >> 172.21.15.93:0/2491807227 conn(0x7fea580fb8b0 msgr2=0x7fea580fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.661+0000 7fea5f3ff700 1 -- 172.21.15.93:0/2491807227 shutdown_connections 2023-11-16T16:46:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.661+0000 7fea5f3ff700 1 -- 172.21.15.93:0/2491807227 wait complete. 2023-11-16T16:46:41.664 INFO:teuthology.orchestra.run.smithi093.stderr:added data pool 4 to fsmap 2023-11-16T16:46:41.681 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:41.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.878+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3152314918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b74101480 msgr2=0x7f3b741018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:41.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.878+0000 7f3b7b7ff700 1 --2- 172.21.15.93:0/3152314918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b74101480 0x7f3b741018d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009990 tx=0x7f3b64009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.879+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3152314918 shutdown_connections 2023-11-16T16:46:41.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.879+0000 7f3b7b7ff700 1 --2- 172.21.15.93:0/3152314918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b74101e10 0x7f3b741086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.879+0000 7f3b7b7ff700 1 --2- 172.21.15.93:0/3152314918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b74100290 0x7f3b741006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.879+0000 7f3b7b7ff700 1 --2- 172.21.15.93:0/3152314918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b74101480 0x7f3b741018d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.879+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3152314918 >> 172.21.15.93:0/3152314918 conn(0x7f3b740fb870 msgr2=0x7f3b740fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:41.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.879+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3152314918 shutdown_connections 2023-11-16T16:46:41.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.879+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3152314918 wait complete. 2023-11-16T16:46:41.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.880+0000 7f3b7b7ff700 1 Processor -- start 2023-11-16T16:46:41.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.880+0000 7f3b7b7ff700 1 -- start start 2023-11-16T16:46:41.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.880+0000 7f3b7b7ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b74100290 0x7f3b740748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:41.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.880+0000 7f3b7b7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b74101480 0x7f3b74072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:41.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.880+0000 7f3b7b7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b74101e10 0x7f3b74073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:41.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.880+0000 7f3b7b7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b74073bc0 con 0x7f3b74101e10 2023-11-16T16:46:41.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.880+0000 7f3b7b7ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b74073d30 con 0x7f3b74100290 2023-11-16T16:46:41.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.880+0000 7f3b7b7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7419fff0 con 0x7f3b74101480 2023-11-16T16:46:41.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.880+0000 7f3b79d9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b74101e10 0x7f3b74073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.880+0000 7f3b79d9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b74101e10 0x7f3b74073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58874/0 (socket says 172.21.15.93:58874) 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.880+0000 7f3b79d9c700 1 -- 172.21.15.93:0/3644548701 learned_addr learned my addr 172.21.15.93:0/3644548701 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b78d9a700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b74101480 0x7f3b74072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b7959b700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b74100290 0x7f3b740748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b79d9c700 1 -- 172.21.15.93:0/3644548701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b74101480 msgr2=0x7f3b74072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b79d9c700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b74101480 0x7f3b74072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b79d9c700 1 -- 172.21.15.93:0/3644548701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b74100290 msgr2=0x7f3b740748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b79d9c700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b74100290 0x7f3b740748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b79d9c700 1 -- 172.21.15.93:0/3644548701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b640095f0 con 0x7f3b74101e10 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b79d9c700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b74101e10 0x7f3b74073420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00ef30 tx=0x7f3b6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b6a7fc700 1 -- 172.21.15.93:0/3644548701 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c018070 con 0x7f3b74101e10 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b6a7fc700 1 -- 172.21.15.93:0/3644548701 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b6c00f040 con 0x7f3b74101e10 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3644548701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7410bc50 con 0x7f3b74101e10 2023-11-16T16:46:41.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b6a7fc700 1 -- 172.21.15.93:0/3644548701 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c00cd00 con 0x7f3b74101e10 2023-11-16T16:46:41.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.881+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3644548701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7410bac0 con 0x7f3b74101e10 2023-11-16T16:46:41.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.882+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3644548701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b741a06d0 con 0x7f3b74101e10 2023-11-16T16:46:41.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.884+0000 7f3b6a7fc700 1 -- 172.21.15.93:0/3644548701 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3b6c003a00 con 0x7f3b74101e10 2023-11-16T16:46:41.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.885+0000 7f3b6a7fc700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3b600a11d0 0x7f3b600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:41.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.886+0000 7f3b6a7fc700 1 -- 172.21.15.93:0/3644548701 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3b6c0cc570 con 0x7f3b74101e10 2023-11-16T16:46:41.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.886+0000 7f3b7959b700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3b600a11d0 0x7f3b600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:41.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.886+0000 7f3b7959b700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3b600a11d0 0x7f3b600a3680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3b741012e0 tx=0x7f3b70007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:41.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:41.889+0000 7f3b6a7fc700 1 -- 172.21.15.93:0/3644548701 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3b6c095b00 con 0x7f3b74101e10 2023-11-16T16:46:42.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.132+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3644548701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b741071d0 con 0x7f3b74101e10 2023-11-16T16:46:42.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.133+0000 7f3b6a7fc700 1 -- 172.21.15.93:0/3644548701 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10750 (secure 0 0 0) 0x7f3b6c099120 con 0x7f3b74101e10 2023-11-16T16:46:42.134 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:42.135 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3614592201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3614592201},{"type":"v1","addr":"172.21.15.93:6839","nonce":3614592201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4068789362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4068789362},{"type":"v1","addr":"172.21.15.93:6841","nonce":4068789362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/778982589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":778982589},{"type":"v1","addr":"172.21.15.159:6837","nonce":778982589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/149300707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":149300707},{"type":"v1","addr":"172.21.15.159:6839","nonce":149300707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-16T16:46:37.124338+0000","modified":"2023-11-16T16:46:41.651539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/3590027328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":3590027328},{"type":"v1","addr":"172.21.15.125:6839","nonce":3590027328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-16T16:46:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.135+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3644548701 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3b600a11d0 msgr2=0x7f3b600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.135+0000 7f3b7b7ff700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3b600a11d0 0x7f3b600a3680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3b741012e0 tx=0x7f3b70007400 comp rx=0 tx=0).stop 2023-11-16T16:46:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.135+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3644548701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b74101e10 msgr2=0x7f3b74073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.135+0000 7f3b7b7ff700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b74101e10 0x7f3b74073420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00ef30 tx=0x7f3b6c00c3a0 comp rx=0 tx=0).stop 2023-11-16T16:46:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.136+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3644548701 shutdown_connections 2023-11-16T16:46:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.136+0000 7f3b7b7ff700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3b600a11d0 0x7f3b600a3680 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.136+0000 7f3b7b7ff700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b74101e10 0x7f3b74073420 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.136+0000 7f3b7b7ff700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b74100290 0x7f3b740748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.136+0000 7f3b7b7ff700 1 --2- 172.21.15.93:0/3644548701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b74101480 0x7f3b74072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.136+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3644548701 >> 172.21.15.93:0/3644548701 conn(0x7f3b740fb870 msgr2=0x7f3b740fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.136+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3644548701 shutdown_connections 2023-11-16T16:46:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:42.136+0000 7f3b7b7ff700 1 -- 172.21.15.93:0/3644548701 wait complete. 2023-11-16T16:46:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 17 2023-11-16T16:46:42.156 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:46:44.304 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:42 smithi125 ceph-mon[131076]: pgmap v281: 129 pgs: 10 creating+peering, 74 active+clean, 45 unknown; 578 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 8 op/s 2023-11-16T16:46:44.304 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:42 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/213682084' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-11-16T16:46:44.304 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:42 smithi125 ceph-mon[131076]: osdmap e87: 12 total, 12 up, 12 in 2023-11-16T16:46:44.304 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:42 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2491807227' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-11-16T16:46:44.304 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:42 smithi125 ceph-mon[131076]: osdmap e88: 12 total, 12 up, 12 in 2023-11-16T16:46:44.304 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:42 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2491807227' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-11-16T16:46:44.304 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:42 smithi125 ceph-mon[131076]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:46:44.304 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:42 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3644548701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:44.305 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:42 smithi159 ceph-mon[131528]: pgmap v281: 129 pgs: 10 creating+peering, 74 active+clean, 45 unknown; 578 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 8 op/s 2023-11-16T16:46:44.305 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:42 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/213682084' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-11-16T16:46:44.305 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:42 smithi159 ceph-mon[131528]: osdmap e87: 12 total, 12 up, 12 in 2023-11-16T16:46:44.305 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:42 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2491807227' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-11-16T16:46:44.305 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:42 smithi159 ceph-mon[131528]: osdmap e88: 12 total, 12 up, 12 in 2023-11-16T16:46:44.305 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:42 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2491807227' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-11-16T16:46:44.305 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:42 smithi159 ceph-mon[131528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:46:44.305 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:42 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3644548701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:44.305 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:42 smithi093 ceph-mon[97863]: pgmap v281: 129 pgs: 10 creating+peering, 74 active+clean, 45 unknown; 578 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 8 op/s 2023-11-16T16:46:44.305 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:42 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/213682084' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-11-16T16:46:44.305 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:42 smithi093 ceph-mon[97863]: osdmap e87: 12 total, 12 up, 12 in 2023-11-16T16:46:44.305 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:42 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2491807227' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-11-16T16:46:44.306 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:42 smithi093 ceph-mon[97863]: osdmap e88: 12 total, 12 up, 12 in 2023-11-16T16:46:44.306 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:42 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2491807227' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-11-16T16:46:44.306 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:42 smithi093 ceph-mon[97863]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:46:44.306 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:42 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3644548701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:44.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:44 smithi159 ceph-mon[131528]: pgmap v284: 129 pgs: 1 creating+activating, 4 creating+peering, 111 active+clean, 13 unknown; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2023-11-16T16:46:44.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.508+0000 7f969adff700 1 -- 172.21.15.93:0/3029926603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9694074ad0 msgr2=0x7f9694072f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:44.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.508+0000 7f969adff700 1 --2- 172.21.15.93:0/3029926603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9694074ad0 0x7f9694072f20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9688009990 tx=0x7f9688009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.509+0000 7f969adff700 1 -- 172.21.15.93:0/3029926603 shutdown_connections 2023-11-16T16:46:44.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.509+0000 7f969adff700 1 --2- 172.21.15.93:0/3029926603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9694074ad0 0x7f9694072f20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.509+0000 7f969adff700 1 --2- 172.21.15.93:0/3029926603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9694108140 0x7f969410a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.509+0000 7f969adff700 1 --2- 172.21.15.93:0/3029926603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96940734f0 0x7f9694073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.509+0000 7f969adff700 1 -- 172.21.15.93:0/3029926603 >> 172.21.15.93:0/3029926603 conn(0x7f96940fb890 msgr2=0x7f96940fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:44.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.509+0000 7f969adff700 1 -- 172.21.15.93:0/3029926603 shutdown_connections 2023-11-16T16:46:44.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.509+0000 7f969adff700 1 -- 172.21.15.93:0/3029926603 wait complete. 2023-11-16T16:46:44.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.510+0000 7f969adff700 1 Processor -- start 2023-11-16T16:46:44.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.510+0000 7f969adff700 1 -- start start 2023-11-16T16:46:44.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.510+0000 7f969adff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96940734f0 0x7f9694198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:44.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.510+0000 7f969adff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9694074ad0 0x7f96941986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:44.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.510+0000 7f969adff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9694108140 0x7f969419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:44.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.510+0000 7f969adff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969419fd10 con 0x7f96940734f0 2023-11-16T16:46:44.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.510+0000 7f969adff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969419fe50 con 0x7f9694074ad0 2023-11-16T16:46:44.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.510+0000 7f969adff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969419ffe0 con 0x7f9694108140 2023-11-16T16:46:44.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.510+0000 7f9698b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96940734f0 0x7f9694198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:44.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.510+0000 7f9698b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96940734f0 0x7f9694198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58890/0 (socket says 172.21.15.93:58890) 2023-11-16T16:46:44.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.510+0000 7f9698b9b700 1 -- 172.21.15.93:0/90784778 learned_addr learned my addr 172.21.15.93:0/90784778 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:44.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f9693fff700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9694074ad0 0x7f96941986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:44.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f969939c700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9694108140 0x7f969419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:44.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f9698b9b700 1 -- 172.21.15.93:0/90784778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9694108140 msgr2=0x7f969419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:44.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f9698b9b700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9694108140 0x7f969419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f9698b9b700 1 -- 172.21.15.93:0/90784778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9694074ad0 msgr2=0x7f96941986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:44.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f9698b9b700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9694074ad0 0x7f96941986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f9698b9b700 1 -- 172.21.15.93:0/90784778 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96880095f0 con 0x7f96940734f0 2023-11-16T16:46:44.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f969939c700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9694108140 0x7f969419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:44.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f9698b9b700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96940734f0 0x7f9694198190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9688004740 tx=0x7f9688009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:44.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f9691ffb700 1 -- 172.21.15.93:0/90784778 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968801d070 con 0x7f96940734f0 2023-11-16T16:46:44.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f969adff700 1 -- 172.21.15.93:0/90784778 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969410de50 con 0x7f96940734f0 2023-11-16T16:46:44.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f9691ffb700 1 -- 172.21.15.93:0/90784778 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9688004830 con 0x7f96940734f0 2023-11-16T16:46:44.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f9691ffb700 1 -- 172.21.15.93:0/90784778 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688005450 con 0x7f96940734f0 2023-11-16T16:46:44.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.511+0000 7f969adff700 1 -- 172.21.15.93:0/90784778 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969410dcc0 con 0x7f96940734f0 2023-11-16T16:46:44.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.512+0000 7f969adff700 1 -- 172.21.15.93:0/90784778 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9694106880 con 0x7f96940734f0 2023-11-16T16:46:44.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.514+0000 7f9691ffb700 1 -- 172.21.15.93:0/90784778 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f968800fb00 con 0x7f96940734f0 2023-11-16T16:46:44.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.516+0000 7f9691ffb700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f96840a1150 0x7f96840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:44.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.516+0000 7f9691ffb700 1 -- 172.21.15.93:0/90784778 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f96880cdd50 con 0x7f96940734f0 2023-11-16T16:46:44.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.516+0000 7f9693fff700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f96840a1150 0x7f96840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:44.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.516+0000 7f9693fff700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f96840a1150 0x7f96840a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9680009c60 tx=0x7f9680009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:44.520 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.519+0000 7f9691ffb700 1 -- 172.21.15.93:0/90784778 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f968805a730 con 0x7f96940734f0 2023-11-16T16:46:44.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:44 smithi125 ceph-mon[131076]: pgmap v284: 129 pgs: 1 creating+activating, 4 creating+peering, 111 active+clean, 13 unknown; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2023-11-16T16:46:44.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:44 smithi093 ceph-mon[97863]: pgmap v284: 129 pgs: 1 creating+activating, 4 creating+peering, 111 active+clean, 13 unknown; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2023-11-16T16:46:44.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.723+0000 7f969adff700 1 -- 172.21.15.93:0/90784778 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f969402cfe0 con 0x7f96940734f0 2023-11-16T16:46:44.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.725+0000 7f9691ffb700 1 -- 172.21.15.93:0/90784778 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23970 (secure 0 0 0) 0x7f96880972e0 con 0x7f96940734f0 2023-11-16T16:46:44.727 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:44.727 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":88,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:46:41.651293+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:46:44.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.727+0000 7f969adff700 1 -- 172.21.15.93:0/90784778 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f96840a1150 msgr2=0x7f96840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:44.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.727+0000 7f969adff700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f96840a1150 0x7f96840a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9680009c60 tx=0x7f9680009260 comp rx=0 tx=0).stop 2023-11-16T16:46:44.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.727+0000 7f969adff700 1 -- 172.21.15.93:0/90784778 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96940734f0 msgr2=0x7f9694198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:44.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.727+0000 7f969adff700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96940734f0 0x7f9694198190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9688004740 tx=0x7f9688009580 comp rx=0 tx=0).stop 2023-11-16T16:46:44.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.727+0000 7f969adff700 1 -- 172.21.15.93:0/90784778 shutdown_connections 2023-11-16T16:46:44.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.727+0000 7f969adff700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f96840a1150 0x7f96840a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.727+0000 7f969adff700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96940734f0 0x7f9694198190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.727+0000 7f969adff700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9694108140 0x7f969419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.727+0000 7f969adff700 1 --2- 172.21.15.93:0/90784778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9694074ad0 0x7f96941986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.727+0000 7f969adff700 1 -- 172.21.15.93:0/90784778 >> 172.21.15.93:0/90784778 conn(0x7f96940fb890 msgr2=0x7f96940fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:44.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.727+0000 7f969adff700 1 -- 172.21.15.93:0/90784778 shutdown_connections 2023-11-16T16:46:44.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.727+0000 7f969adff700 1 -- 172.21.15.93:0/90784778 wait complete. 2023-11-16T16:46:44.749 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:46:44.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.944+0000 7f0f733ff700 1 -- 172.21.15.93:0/3645373906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c100290 msgr2=0x7f0f6c1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.944+0000 7f0f733ff700 1 --2- 172.21.15.93:0/3645373906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c100290 0x7f0f6c1006a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009990 tx=0x7f0f68009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.944+0000 7f0f733ff700 1 -- 172.21.15.93:0/3645373906 shutdown_connections 2023-11-16T16:46:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.944+0000 7f0f733ff700 1 --2- 172.21.15.93:0/3645373906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c100290 0x7f0f6c1006a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.944+0000 7f0f733ff700 1 --2- 172.21.15.93:0/3645373906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f6c101e10 0x7f0f6c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.944+0000 7f0f733ff700 1 --2- 172.21.15.93:0/3645373906 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f6c101480 0x7f0f6c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.944+0000 7f0f733ff700 1 -- 172.21.15.93:0/3645373906 >> 172.21.15.93:0/3645373906 conn(0x7f0f6c0fb870 msgr2=0x7f0f6c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.944+0000 7f0f733ff700 1 -- 172.21.15.93:0/3645373906 shutdown_connections 2023-11-16T16:46:44.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.944+0000 7f0f733ff700 1 -- 172.21.15.93:0/3645373906 wait complete. 2023-11-16T16:46:44.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.945+0000 7f0f733ff700 1 Processor -- start 2023-11-16T16:46:44.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.945+0000 7f0f733ff700 1 -- start start 2023-11-16T16:46:44.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.945+0000 7f0f733ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c100290 0x7f0f6c195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:44.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.945+0000 7f0f733ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f6c101480 0x7f0f6c1964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:44.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.945+0000 7f0f733ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f6c101e10 0x7f0f6c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:44.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.945+0000 7f0f733ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c19db30 con 0x7f0f6c100290 2023-11-16T16:46:44.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.945+0000 7f0f733ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c19dc70 con 0x7f0f6c101e10 2023-11-16T16:46:44.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.945+0000 7f0f733ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c19de00 con 0x7f0f6c101480 2023-11-16T16:46:44.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f7119b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c100290 0x7f0f6c195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:44.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f7119b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c100290 0x7f0f6c195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58906/0 (socket says 172.21.15.93:58906) 2023-11-16T16:46:44.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f7119b700 1 -- 172.21.15.93:0/1545445614 learned_addr learned my addr 172.21.15.93:0/1545445614 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f7199c700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f6c101e10 0x7f0f6c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f7099a700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f6c101480 0x7f0f6c1964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f7119b700 1 -- 172.21.15.93:0/1545445614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f6c101480 msgr2=0x7f0f6c1964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f7119b700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f6c101480 0x7f0f6c1964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f7119b700 1 -- 172.21.15.93:0/1545445614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f6c101e10 msgr2=0x7f0f6c19d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f7119b700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f6c101e10 0x7f0f6c19d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f7119b700 1 -- 172.21.15.93:0/1545445614 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f680095f0 con 0x7f0f6c100290 2023-11-16T16:46:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f7119b700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c100290 0x7f0f6c195fb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0f68000c00 tx=0x7f0f68005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f627fc700 1 -- 172.21.15.93:0/1545445614 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6801d070 con 0x7f0f6c100290 2023-11-16T16:46:44.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f733ff700 1 -- 172.21.15.93:0/1545445614 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6c10bc40 con 0x7f0f6c100290 2023-11-16T16:46:44.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f627fc700 1 -- 172.21.15.93:0/1545445614 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f68003f60 con 0x7f0f6c100290 2023-11-16T16:46:44.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.946+0000 7f0f627fc700 1 -- 172.21.15.93:0/1545445614 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68022b50 con 0x7f0f6c100290 2023-11-16T16:46:44.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.947+0000 7f0f733ff700 1 -- 172.21.15.93:0/1545445614 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6c10bab0 con 0x7f0f6c100290 2023-11-16T16:46:44.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.948+0000 7f0f733ff700 1 -- 172.21.15.93:0/1545445614 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f6c107880 con 0x7f0f6c100290 2023-11-16T16:46:44.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.950+0000 7f0f627fc700 1 -- 172.21.15.93:0/1545445614 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0f680040d0 con 0x7f0f6c100290 2023-11-16T16:46:44.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.951+0000 7f0f627fc700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f0f580a1150 0x7f0f580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:44.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.951+0000 7f0f627fc700 1 -- 172.21.15.93:0/1545445614 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0f680ce8e0 con 0x7f0f6c100290 2023-11-16T16:46:44.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.951+0000 7f0f7099a700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f0f580a1150 0x7f0f580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:44.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.952+0000 7f0f7099a700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f0f580a1150 0x7f0f580a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c007c60 tx=0x7f0f5c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:44.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:44.953+0000 7f0f627fc700 1 -- 172.21.15.93:0/1545445614 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0f68097e70 con 0x7f0f6c100290 2023-11-16T16:46:45.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.155+0000 7f0f733ff700 1 -- 172.21.15.93:0/1545445614 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f6c02cfe0 con 0x7f0f6c100290 2023-11-16T16:46:45.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.157+0000 7f0f627fc700 1 -- 172.21.15.93:0/1545445614 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23970 (secure 0 0 0) 0x7f0f6809b490 con 0x7f0f6c100290 2023-11-16T16:46:45.159 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:45.159 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":88,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:46:41.651293+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:46:45.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.160+0000 7f0f733ff700 1 -- 172.21.15.93:0/1545445614 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f0f580a1150 msgr2=0x7f0f580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:45.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.160+0000 7f0f733ff700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f0f580a1150 0x7f0f580a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c007c60 tx=0x7f0f5c0073d0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.160+0000 7f0f733ff700 1 -- 172.21.15.93:0/1545445614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c100290 msgr2=0x7f0f6c195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:45.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.160+0000 7f0f733ff700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c100290 0x7f0f6c195fb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0f68000c00 tx=0x7f0f68005740 comp rx=0 tx=0).stop 2023-11-16T16:46:45.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.160+0000 7f0f733ff700 1 -- 172.21.15.93:0/1545445614 shutdown_connections 2023-11-16T16:46:45.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.160+0000 7f0f733ff700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f0f580a1150 0x7f0f580a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.160+0000 7f0f733ff700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c100290 0x7f0f6c195fb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.160+0000 7f0f733ff700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f6c101e10 0x7f0f6c19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.160+0000 7f0f733ff700 1 --2- 172.21.15.93:0/1545445614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f6c101480 0x7f0f6c1964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.160+0000 7f0f733ff700 1 -- 172.21.15.93:0/1545445614 >> 172.21.15.93:0/1545445614 conn(0x7f0f6c0fb870 msgr2=0x7f0f6c1049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:45.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.160+0000 7f0f733ff700 1 -- 172.21.15.93:0/1545445614 shutdown_connections 2023-11-16T16:46:45.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.160+0000 7f0f733ff700 1 -- 172.21.15.93:0/1545445614 wait complete. 2023-11-16T16:46:45.195 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:45.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.390+0000 7fc6fafff700 1 -- 172.21.15.93:0/3505350738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f4100d20 msgr2=0x7fc6f4103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:45.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.390+0000 7fc6fafff700 1 --2- 172.21.15.93:0/3505350738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f4100d20 0x7fc6f4103100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009990 tx=0x7fc6e8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.390+0000 7fc6fafff700 1 -- 172.21.15.93:0/3505350738 shutdown_connections 2023-11-16T16:46:45.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.390+0000 7fc6fafff700 1 --2- 172.21.15.93:0/3505350738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f4100d20 0x7fc6f4103100 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.390+0000 7fc6fafff700 1 --2- 172.21.15.93:0/3505350738 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6f4105f60 0x7fc6f41083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.390+0000 7fc6fafff700 1 --2- 172.21.15.93:0/3505350738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6f4103640 0x7fc6f4105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.390+0000 7fc6fafff700 1 -- 172.21.15.93:0/3505350738 >> 172.21.15.93:0/3505350738 conn(0x7fc6f40fa7a0 msgr2=0x7fc6f40fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:45.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.391+0000 7fc6fafff700 1 -- 172.21.15.93:0/3505350738 shutdown_connections 2023-11-16T16:46:45.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.391+0000 7fc6fafff700 1 -- 172.21.15.93:0/3505350738 wait complete. 2023-11-16T16:46:45.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6fafff700 1 Processor -- start 2023-11-16T16:46:45.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6fafff700 1 -- start start 2023-11-16T16:46:45.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6fafff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f4100d20 0x7fc6f4100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:45.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6fafff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6f4103640 0x7fc6f40fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:45.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6fafff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6f4105f60 0x7fc6f40ff210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:45.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6fafff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40ff9b0 con 0x7fc6f4100d20 2023-11-16T16:46:45.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6fafff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40ffb20 con 0x7fc6f4103640 2023-11-16T16:46:45.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6fafff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f41a0090 con 0x7fc6f4105f60 2023-11-16T16:46:45.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6f8d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f4100d20 0x7fc6f4100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:45.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6f8d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f4100d20 0x7fc6f4100690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58938/0 (socket says 172.21.15.93:58938) 2023-11-16T16:46:45.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6f8d9b700 1 -- 172.21.15.93:0/3802476747 learned_addr learned my addr 172.21.15.93:0/3802476747 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:45.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6f959c700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6f4105f60 0x7fc6f40ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:45.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.392+0000 7fc6f3fff700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6f4103640 0x7fc6f40fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:45.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.393+0000 7fc6f8d9b700 1 -- 172.21.15.93:0/3802476747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6f4105f60 msgr2=0x7fc6f40ff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:45.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.393+0000 7fc6f8d9b700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6f4105f60 0x7fc6f40ff210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.393+0000 7fc6f8d9b700 1 -- 172.21.15.93:0/3802476747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6f4103640 msgr2=0x7fc6f40fecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:45.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.393+0000 7fc6f8d9b700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6f4103640 0x7fc6f40fecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.393+0000 7fc6f8d9b700 1 -- 172.21.15.93:0/3802476747 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e80095f0 con 0x7fc6f4100d20 2023-11-16T16:46:45.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.393+0000 7fc6f8d9b700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f4100d20 0x7fc6f4100690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80036c0 tx=0x7fc6e800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:45.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.393+0000 7fc6f1ffb700 1 -- 172.21.15.93:0/3802476747 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e801d070 con 0x7fc6f4100d20 2023-11-16T16:46:45.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.393+0000 7fc6fafff700 1 -- 172.21.15.93:0/3802476747 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f410bc50 con 0x7fc6f4100d20 2023-11-16T16:46:45.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.393+0000 7fc6f1ffb700 1 -- 172.21.15.93:0/3802476747 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6e800fc30 con 0x7fc6f4100d20 2023-11-16T16:46:45.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.393+0000 7fc6fafff700 1 -- 172.21.15.93:0/3802476747 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f410bac0 con 0x7fc6f4100d20 2023-11-16T16:46:45.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.393+0000 7fc6f1ffb700 1 -- 172.21.15.93:0/3802476747 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80175f0 con 0x7fc6f4100d20 2023-11-16T16:46:45.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.394+0000 7fc6fafff700 1 -- 172.21.15.93:0/3802476747 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f40fc310 con 0x7fc6f4100d20 2023-11-16T16:46:45.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.396+0000 7fc6f1ffb700 1 -- 172.21.15.93:0/3802476747 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc6e8022470 con 0x7fc6f4100d20 2023-11-16T16:46:45.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.397+0000 7fc6f1ffb700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc6e40a1150 0x7fc6e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:45.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.397+0000 7fc6f1ffb700 1 -- 172.21.15.93:0/3802476747 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc6e80cded0 con 0x7fc6f4100d20 2023-11-16T16:46:45.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.398+0000 7fc6f3fff700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc6e40a1150 0x7fc6e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:45.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.398+0000 7fc6f3fff700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc6e40a1150 0x7fc6e40a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000bbd0 tx=0x7fc6e000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:45.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.401+0000 7fc6f1ffb700 1 -- 172.21.15.93:0/3802476747 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc6e8097460 con 0x7fc6f4100d20 2023-11-16T16:46:45.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:45 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/90784778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:45.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:45 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1545445614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:45.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:45 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/90784778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:45.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:45 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1545445614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:45.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.646+0000 7fc6fafff700 1 -- 172.21.15.93:0/3802476747 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6f402cfe0 con 0x7fc6f4100d20 2023-11-16T16:46:45.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.647+0000 7fc6f1ffb700 1 -- 172.21.15.93:0/3802476747 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10750 (secure 0 0 0) 0x7fc6e809aa80 con 0x7fc6f4100d20 2023-11-16T16:46:45.649 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:45.649 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3614592201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3614592201},{"type":"v1","addr":"172.21.15.93:6839","nonce":3614592201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4068789362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4068789362},{"type":"v1","addr":"172.21.15.93:6841","nonce":4068789362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/778982589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":778982589},{"type":"v1","addr":"172.21.15.159:6837","nonce":778982589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/149300707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":149300707},{"type":"v1","addr":"172.21.15.159:6839","nonce":149300707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-16T16:46:37.124338+0000","modified":"2023-11-16T16:46:41.651539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/3590027328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":3590027328},{"type":"v1","addr":"172.21.15.125:6839","nonce":3590027328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-16T16:46:45.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:45 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/90784778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:45.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:45 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1545445614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:45.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.649+0000 7fc6fafff700 1 -- 172.21.15.93:0/3802476747 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc6e40a1150 msgr2=0x7fc6e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.649+0000 7fc6fafff700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc6e40a1150 0x7fc6e40a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000bbd0 tx=0x7fc6e000b5b0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.649+0000 7fc6fafff700 1 -- 172.21.15.93:0/3802476747 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f4100d20 msgr2=0x7fc6f4100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.649+0000 7fc6fafff700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f4100d20 0x7fc6f4100690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80036c0 tx=0x7fc6e800f740 comp rx=0 tx=0).stop 2023-11-16T16:46:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.649+0000 7fc6fafff700 1 -- 172.21.15.93:0/3802476747 shutdown_connections 2023-11-16T16:46:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.649+0000 7fc6fafff700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc6e40a1150 0x7fc6e40a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.649+0000 7fc6fafff700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f4100d20 0x7fc6f4100690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.649+0000 7fc6fafff700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6f4105f60 0x7fc6f40ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.649+0000 7fc6fafff700 1 --2- 172.21.15.93:0/3802476747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6f4103640 0x7fc6f40fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.649+0000 7fc6fafff700 1 -- 172.21.15.93:0/3802476747 >> 172.21.15.93:0/3802476747 conn(0x7fc6f40fa7a0 msgr2=0x7fc6f4101980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:45.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.649+0000 7fc6fafff700 1 -- 172.21.15.93:0/3802476747 shutdown_connections 2023-11-16T16:46:45.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.649+0000 7fc6fafff700 1 -- 172.21.15.93:0/3802476747 wait complete. 2023-11-16T16:46:45.653 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 17 2023-11-16T16:46:45.671 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:46:45.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.867+0000 7fef371ff700 1 -- 172.21.15.93:0/2548341578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef30101480 msgr2=0x7fef301018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:45.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.867+0000 7fef371ff700 1 --2- 172.21.15.93:0/2548341578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef30101480 0x7fef301018d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009990 tx=0x7fef1c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.867+0000 7fef371ff700 1 -- 172.21.15.93:0/2548341578 shutdown_connections 2023-11-16T16:46:45.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.867+0000 7fef371ff700 1 --2- 172.21.15.93:0/2548341578 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef30101e10 0x7fef301086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.867+0000 7fef371ff700 1 --2- 172.21.15.93:0/2548341578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef30100290 0x7fef301006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.867+0000 7fef371ff700 1 --2- 172.21.15.93:0/2548341578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef30101480 0x7fef301018d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.867+0000 7fef371ff700 1 -- 172.21.15.93:0/2548341578 >> 172.21.15.93:0/2548341578 conn(0x7fef300fb870 msgr2=0x7fef300fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:45.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.867+0000 7fef371ff700 1 -- 172.21.15.93:0/2548341578 shutdown_connections 2023-11-16T16:46:45.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.867+0000 7fef371ff700 1 -- 172.21.15.93:0/2548341578 wait complete. 2023-11-16T16:46:45.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.868+0000 7fef371ff700 1 Processor -- start 2023-11-16T16:46:45.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.868+0000 7fef371ff700 1 -- start start 2023-11-16T16:46:45.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef371ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef30100290 0x7fef30198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:45.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef371ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef30101480 0x7fef30198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:45.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef371ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef30101e10 0x7fef3019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:45.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef371ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3019fdc0 con 0x7fef30101e10 2023-11-16T16:46:45.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef371ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3019ff00 con 0x7fef30101480 2023-11-16T16:46:45.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef371ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef301a0090 con 0x7fef30100290 2023-11-16T16:46:45.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef34f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef30100290 0x7fef30198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:45.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef34f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef30100290 0x7fef30198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:50682/0 (socket says 172.21.15.93:50682) 2023-11-16T16:46:45.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef34f9b700 1 -- 172.21.15.93:0/2503870543 learned_addr learned my addr 172.21.15.93:0/2503870543 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:45.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef3579c700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef30101e10 0x7fef3019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:45.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef2ffff700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef30101480 0x7fef30198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:45.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef34f9b700 1 -- 172.21.15.93:0/2503870543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef30101480 msgr2=0x7fef30198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:45.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef34f9b700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef30101480 0x7fef30198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef34f9b700 1 -- 172.21.15.93:0/2503870543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef30101e10 msgr2=0x7fef3019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:45.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef34f9b700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef30101e10 0x7fef3019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:45.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef34f9b700 1 -- 172.21.15.93:0/2503870543 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef28009c40 con 0x7fef30100290 2023-11-16T16:46:45.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.869+0000 7fef2ffff700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef30101480 0x7fef30198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:45.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.870+0000 7fef34f9b700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef30100290 0x7fef30198240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fef2400e970 tx=0x7fef2400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:45.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.870+0000 7fef2dffb700 1 -- 172.21.15.93:0/2503870543 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24009790 con 0x7fef30100290 2023-11-16T16:46:45.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.870+0000 7fef371ff700 1 -- 172.21.15.93:0/2503870543 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c0095f0 con 0x7fef30100290 2023-11-16T16:46:45.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.870+0000 7fef2dffb700 1 -- 172.21.15.93:0/2503870543 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef2400eeb0 con 0x7fef30100290 2023-11-16T16:46:45.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.870+0000 7fef2dffb700 1 -- 172.21.15.93:0/2503870543 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24003d20 con 0x7fef30100290 2023-11-16T16:46:45.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.870+0000 7fef371ff700 1 -- 172.21.15.93:0/2503870543 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3010bc50 con 0x7fef30100290 2023-11-16T16:46:45.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.871+0000 7fef371ff700 1 -- 172.21.15.93:0/2503870543 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef301928e0 con 0x7fef30100290 2023-11-16T16:46:45.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.873+0000 7fef2dffb700 1 -- 172.21.15.93:0/2503870543 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fef24003e80 con 0x7fef30100290 2023-11-16T16:46:45.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.874+0000 7fef2dffb700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fef200a1150 0x7fef200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:45.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.875+0000 7fef2dffb700 1 -- 172.21.15.93:0/2503870543 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fef24014070 con 0x7fef30100290 2023-11-16T16:46:45.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.875+0000 7fef2ffff700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fef200a1150 0x7fef200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:45.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.875+0000 7fef2ffff700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fef200a1150 0x7fef200a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef1c000c00 tx=0x7fef1c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:45.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:45.876+0000 7fef2dffb700 1 -- 172.21.15.93:0/2503870543 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fef240963f0 con 0x7fef30100290 2023-11-16T16:46:46.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.077+0000 7fef371ff700 1 -- 172.21.15.93:0/2503870543 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef30067a60 con 0x7fef30100290 2023-11-16T16:46:46.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.078+0000 7fef2dffb700 1 -- 172.21.15.93:0/2503870543 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23970 (secure 0 0 0) 0x7fef24099a10 con 0x7fef30100290 2023-11-16T16:46:46.081 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:46.081 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":88,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:46:41.651293+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:46:46.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.081+0000 7fef371ff700 1 -- 172.21.15.93:0/2503870543 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fef200a1150 msgr2=0x7fef200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.081+0000 7fef371ff700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fef200a1150 0x7fef200a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef1c000c00 tx=0x7fef1c005fb0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.081+0000 7fef371ff700 1 -- 172.21.15.93:0/2503870543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef30100290 msgr2=0x7fef30198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.081+0000 7fef371ff700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef30100290 0x7fef30198240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fef2400e970 tx=0x7fef2400ed30 comp rx=0 tx=0).stop 2023-11-16T16:46:46.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.081+0000 7fef371ff700 1 -- 172.21.15.93:0/2503870543 shutdown_connections 2023-11-16T16:46:46.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.081+0000 7fef371ff700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fef200a1150 0x7fef200a3600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.081+0000 7fef371ff700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef30101e10 0x7fef3019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.081+0000 7fef371ff700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef30100290 0x7fef30198240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.081+0000 7fef371ff700 1 --2- 172.21.15.93:0/2503870543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef30101480 0x7fef30198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.081+0000 7fef371ff700 1 -- 172.21.15.93:0/2503870543 >> 172.21.15.93:0/2503870543 conn(0x7fef300fb870 msgr2=0x7fef30106de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.081+0000 7fef371ff700 1 -- 172.21.15.93:0/2503870543 shutdown_connections 2023-11-16T16:46:46.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.081+0000 7fef371ff700 1 -- 172.21.15.93:0/2503870543 wait complete. 2023-11-16T16:46:46.103 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-16T16:46:37.124338+0000', 'modified': '2023-11-16T16:46:41.651539+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24566}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6839/3590027328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6838', 'nonce': 3590027328}, {'type': 'v1', 'addr': '172.21.15.125:6839', 'nonce': 3590027328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-16T16:46:46.103 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-16T16:46:46.103 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-16T16:46:46.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.301+0000 7fe66cdff700 1 -- 172.21.15.93:0/817687586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe668074b30 msgr2=0x7fe668072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.301+0000 7fe66cdff700 1 --2- 172.21.15.93:0/817687586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe668074b30 0x7fe668072f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe658009990 tx=0x7fe658009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.301+0000 7fe66cdff700 1 -- 172.21.15.93:0/817687586 shutdown_connections 2023-11-16T16:46:46.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.301+0000 7fe66cdff700 1 --2- 172.21.15.93:0/817687586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe668074b30 0x7fe668072f80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.301+0000 7fe66cdff700 1 --2- 172.21.15.93:0/817687586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6681080f0 0x7fe66810a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.301+0000 7fe66cdff700 1 --2- 172.21.15.93:0/817687586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe668073550 0x7fe6680739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.301+0000 7fe66cdff700 1 -- 172.21.15.93:0/817687586 >> 172.21.15.93:0/817687586 conn(0x7fe6680fb810 msgr2=0x7fe6680fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.301+0000 7fe66cdff700 1 -- 172.21.15.93:0/817687586 shutdown_connections 2023-11-16T16:46:46.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.301+0000 7fe66cdff700 1 -- 172.21.15.93:0/817687586 wait complete. 2023-11-16T16:46:46.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.302+0000 7fe66cdff700 1 Processor -- start 2023-11-16T16:46:46.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.302+0000 7fe66cdff700 1 -- start start 2023-11-16T16:46:46.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe66cdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe668073550 0x7fe668198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe66cdff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe668074b30 0x7fe668198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe66cdff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6681080f0 0x7fe66819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe66cdff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66819fd80 con 0x7fe668074b30 2023-11-16T16:46:46.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe66cdff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66819fec0 con 0x7fe6681080f0 2023-11-16T16:46:46.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe66cdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6681a0050 con 0x7fe668073550 2023-11-16T16:46:46.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe65ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe668074b30 0x7fe668198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe65ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe668074b30 0x7fe668198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58980/0 (socket says 172.21.15.93:58980) 2023-11-16T16:46:46.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe66659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe668073550 0x7fe668198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe65ffff700 1 -- 172.21.15.93:0/4148451284 learned_addr learned my addr 172.21.15.93:0/4148451284 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:46.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe666d9d700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6681080f0 0x7fe66819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe66659c700 1 -- 172.21.15.93:0/4148451284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6681080f0 msgr2=0x7fe66819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe66659c700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6681080f0 0x7fe66819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe66659c700 1 -- 172.21.15.93:0/4148451284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe668074b30 msgr2=0x7fe668198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe66659c700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe668074b30 0x7fe668198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe66659c700 1 -- 172.21.15.93:0/4148451284 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6580095f0 con 0x7fe668073550 2023-11-16T16:46:46.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.303+0000 7fe65ffff700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe668074b30 0x7fe668198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:46.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.304+0000 7fe66659c700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe668073550 0x7fe668198200 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe65800b680 tx=0x7fe658009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.304+0000 7fe65f7fe700 1 -- 172.21.15.93:0/4148451284 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65801d070 con 0x7fe668073550 2023-11-16T16:46:46.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.304+0000 7fe65f7fe700 1 -- 172.21.15.93:0/4148451284 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe658021410 con 0x7fe668073550 2023-11-16T16:46:46.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.304+0000 7fe66cdff700 1 -- 172.21.15.93:0/4148451284 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66810de00 con 0x7fe668073550 2023-11-16T16:46:46.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.304+0000 7fe65f7fe700 1 -- 172.21.15.93:0/4148451284 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658017650 con 0x7fe668073550 2023-11-16T16:46:46.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.304+0000 7fe66cdff700 1 -- 172.21.15.93:0/4148451284 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66810dc70 con 0x7fe668073550 2023-11-16T16:46:46.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.307+0000 7fe65f7fe700 1 -- 172.21.15.93:0/4148451284 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe658021810 con 0x7fe668073550 2023-11-16T16:46:46.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.308+0000 7fe65f7fe700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe6540a11d0 0x7fe6540a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.308+0000 7fe6806d4b80 1 Processor -- start 2023-11-16T16:46:46.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.308+0000 7fe65f7fe700 1 -- 172.21.15.93:0/4148451284 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe6580cde90 con 0x7fe668073550 2023-11-16T16:46:46.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.308+0000 7fe6806d4b80 1 -- start start 2023-11-16T16:46:46.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.308+0000 7fe65ffff700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe6540a11d0 0x7fe6540a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe6806d4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c06982bb0 0x564c06982fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe6806d4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 0x564c06983ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe6806d4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c069840c0 0x564c06984530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe6806d4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c06888010 con 0x564c06983500 2023-11-16T16:46:46.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe6806d4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c068da850 con 0x564c069840c0 2023-11-16T16:46:46.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe6806d4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c068d3ce0 con 0x564c06982bb0 2023-11-16T16:46:46.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe65ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 0x564c06983ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe65ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 0x564c06983ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58996/0 (socket says 172.21.15.93:58996) 2023-11-16T16:46:46.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe65ffff700 1 -- 172.21.15.93:0/1894085129 learned_addr learned my addr 172.21.15.93:0/1894085129 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:46.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe66659c700 1 --2- 172.21.15.93:0/1894085129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c06982bb0 0x564c06982fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe666d9d700 1 --2- 172.21.15.93:0/1894085129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c069840c0 0x564c06984530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe65ffff700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe6540a11d0 0x7fe6540a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe650005950 tx=0x7fe650011950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe65ffff700 1 -- 172.21.15.93:0/1894085129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c06982bb0 msgr2=0x564c06982fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe65ffff700 1 --2- 172.21.15.93:0/1894085129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c06982bb0 0x564c06982fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe65ffff700 1 -- 172.21.15.93:0/1894085129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c069840c0 msgr2=0x564c06984530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe65ffff700 1 --2- 172.21.15.93:0/1894085129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c069840c0 0x564c06984530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.309+0000 7fe65ffff700 1 -- 172.21.15.93:0/1894085129 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c067555e0 con 0x564c06983500 2023-11-16T16:46:46.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.310+0000 7fe65ffff700 1 --2- 172.21.15.93:0/1894085129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 0x564c06983ae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe65001d810 tx=0x7fe65001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d793c488ef02ee server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.310+0000 7fe65cff9700 1 -- 172.21.15.93:0/1894085129 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650004830 con 0x564c06983500 2023-11-16T16:46:46.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.310+0000 7fe65cff9700 1 -- 172.21.15.93:0/1894085129 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe650004e50 con 0x564c06983500 2023-11-16T16:46:46.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.310+0000 7fe65cff9700 1 -- 172.21.15.93:0/1894085129 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65000f720 con 0x564c06983500 2023-11-16T16:46:46.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.310+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1894085129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 msgr2=0x564c06983ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.310+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1894085129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 0x564c06983ae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe65001d810 tx=0x7fe65001c000 comp rx=0 tx=0).stop 2023-11-16T16:46:46.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.311+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1894085129 shutdown_connections 2023-11-16T16:46:46.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.311+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1894085129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c069840c0 0x564c06984530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.311+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1894085129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c06982bb0 0x564c06982fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.311+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1894085129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 0x564c06983ae0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.311+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1894085129 >> 172.21.15.93:0/1894085129 conn(0x564c06978310 msgr2=0x564c0697d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.311+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1894085129 shutdown_connections 2023-11-16T16:46:46.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.311+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1894085129 wait complete. 2023-11-16T16:46:46.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.311+0000 7fe6806d4b80 1 -- start start 2023-11-16T16:46:46.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.311+0000 7fe6806d4b80 1 Processor -- start 2023-11-16T16:46:46.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe6806d4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06982bb0 0x564c0698c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe6806d4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 0x564c0698cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe6806d4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c069840c0 0x564c06993b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe6806d4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c06875890 con 0x564c06983500 2023-11-16T16:46:46.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe6806d4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c06875530 con 0x564c06982bb0 2023-11-16T16:46:46.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe6806d4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c067578d0 con 0x564c069840c0 2023-11-16T16:46:46.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe6806d4b80 10 client.0 _ll_register_callbacks cb 0x564c06893730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:46.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe65ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 0x564c0698cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe65ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 0x564c0698cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59002/0 (socket says 172.21.15.93:59002) 2023-11-16T16:46:46.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe65ffff700 1 -- 172.21.15.93:0/1254971330 learned_addr learned my addr 172.21.15.93:0/1254971330 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:46.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe66659c700 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06982bb0 0x564c0698c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe666d9d700 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c069840c0 0x564c06993b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe65ffff700 1 -- 172.21.15.93:0/1254971330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c069840c0 msgr2=0x564c06993b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe65ffff700 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c069840c0 0x564c06993b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe65ffff700 1 -- 172.21.15.93:0/1254971330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06982bb0 msgr2=0x564c0698c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe65ffff700 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06982bb0 0x564c0698c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe65ffff700 1 -- 172.21.15.93:0/1254971330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65001d550 con 0x564c06983500 2023-11-16T16:46:46.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.312+0000 7fe666d9d700 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c069840c0 0x564c06993b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:46.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.313+0000 7fe65ffff700 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 0x564c0698cae0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe65001d7e0 tx=0x7fe650009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618ce294caad28bb server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.313+0000 7fe64dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:46:46.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.313+0000 7fe64dffb700 1 -- 172.21.15.93:0/1254971330 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65000f9a0 con 0x564c06983500 2023-11-16T16:46:46.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.313+0000 7fe64dffb700 1 -- 172.21.15.93:0/1254971330 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe65000fb00 con 0x564c06983500 2023-11-16T16:46:46.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.313+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1254971330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c06975ad0 con 0x564c06983500 2023-11-16T16:46:46.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.313+0000 7fe64dffb700 1 -- 172.21.15.93:0/1254971330 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe65000fcc0 con 0x564c06983500 2023-11-16T16:46:46.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.313+0000 7fe6806d4b80 10 client.14637 fetch_fsmap learned FSMap version 17 2023-11-16T16:46:46.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.313+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1254971330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x564c0689a140 con 0x564c06983500 2023-11-16T16:46:46.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.313+0000 7fe64dffb700 1 -- 172.21.15.93:0/1254971330 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c06975ad0 con 0x564c06983500 2023-11-16T16:46:46.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.314+0000 7fe64dffb700 1 -- 172.21.15.93:0/1254971330 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe6500268b0 con 0x564c06983500 2023-11-16T16:46:46.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.314+0000 7fe6806d4b80 10 client.14637 fetch_fsmap finished waiting for FSMap version 17 2023-11-16T16:46:46.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.314+0000 7fe6806d4b80 10 client.14637 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-16T16:46:46.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.314+0000 7fe6806d4b80 20 client.14637 populate_metadata read hostname 'smithi093' 2023-11-16T16:46:46.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.314+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564c06997760 0x564c06996bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.314+0000 7fe6806d4b80 4 client.14637 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:46.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.314+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1254971330 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c06896050 con 0x564c06997760 2023-11-16T16:46:46.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.314+0000 7fe66659c700 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564c06997760 0x564c06996bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.315+0000 7fe66659c700 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564c06997760 0x564c06996bc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6ea9f97af8056e1 server_cookie=e33eea7be230b56e in_seq=0 out_seq=0 2023-11-16T16:46:46.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.315+0000 7fe64dffb700 10 client.14637 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:46:46.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe64dffb700 1 -- 172.21.15.93:0/1254971330 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564c06896050 con 0x564c06997760 2023-11-16T16:46:46.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe64dffb700 10 client.14637 handle_command_reply: tid=0 2023-11-16T16:46:46.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe6806d4b80 1 client.14637 shutdown 2023-11-16T16:46:46.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1254971330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 msgr2=0x564c0698cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 0x564c0698cae0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe65001d7e0 tx=0x7fe650009650 comp rx=0 tx=0).stop 2023-11-16T16:46:46.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1254971330 shutdown_connections 2023-11-16T16:46:46.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564c06997760 0x564c06996bc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c069840c0 0x564c06993b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06982bb0 0x564c0698c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1254971330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06983500 0x564c0698cae0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe64dffb700 0 client.14637 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:46:46.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1254971330 >> 172.21.15.93:0/1254971330 conn(0x564c06978310 msgr2=0x564c0697cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1254971330 shutdown_connections 2023-11-16T16:46:46.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1254971330 wait complete. 2023-11-16T16:46:46.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.316+0000 7fe6806d4b80 20 client.14637 trim_cache size 0 max 16384 2023-11-16T16:46:46.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.334+0000 7fe6806d4b80 1 Processor -- start 2023-11-16T16:46:46.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.335+0000 7fe6806d4b80 1 -- start start 2023-11-16T16:46:46.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.335+0000 7fe6806d4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06982bb0 0x564c06996bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.335+0000 7fe6806d4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c06983500 0x564c06993a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.335+0000 7fe6806d4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c069840c0 0x564c06986fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.335+0000 7fe6806d4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c067dfed0 con 0x564c069840c0 2023-11-16T16:46:46.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.335+0000 7fe6806d4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0686fe10 con 0x564c06982bb0 2023-11-16T16:46:46.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.335+0000 7fe6806d4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c06891fa0 con 0x564c06983500 2023-11-16T16:46:46.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.335+0000 7fe666d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c069840c0 0x564c06986fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.335+0000 7fe666d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c069840c0 0x564c06986fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59016/0 (socket says 172.21.15.93:59016) 2023-11-16T16:46:46.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.335+0000 7fe666d9d700 1 -- 172.21.15.93:0/1324025541 learned_addr learned my addr 172.21.15.93:0/1324025541 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:46.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.335+0000 7fe66659c700 1 --2- 172.21.15.93:0/1324025541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06982bb0 0x564c06996bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.335+0000 7fe65ffff700 1 --2- 172.21.15.93:0/1324025541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c06983500 0x564c06993a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.336+0000 7fe666d9d700 1 -- 172.21.15.93:0/1324025541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c06983500 msgr2=0x564c06993a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.336+0000 7fe666d9d700 1 --2- 172.21.15.93:0/1324025541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c06983500 0x564c06993a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.336+0000 7fe666d9d700 1 -- 172.21.15.93:0/1324025541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06982bb0 msgr2=0x564c06996bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.336+0000 7fe666d9d700 1 --2- 172.21.15.93:0/1324025541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06982bb0 0x564c06996bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.336+0000 7fe666d9d700 1 -- 172.21.15.93:0/1324025541 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c06899e90 con 0x564c069840c0 2023-11-16T16:46:46.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.336+0000 7fe66659c700 1 --2- 172.21.15.93:0/1324025541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06982bb0 0x564c06996bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:46.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.336+0000 7fe666d9d700 1 --2- 172.21.15.93:0/1324025541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c069840c0 0x564c06986fc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe6600094f0 tx=0x7fe660009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c01d7fbca0f23b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.336+0000 7fe65d7fa700 1 -- 172.21.15.93:0/1324025541 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66000e040 con 0x564c069840c0 2023-11-16T16:46:46.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.336+0000 7fe65d7fa700 1 -- 172.21.15.93:0/1324025541 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe66001b070 con 0x564c069840c0 2023-11-16T16:46:46.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.336+0000 7fe65d7fa700 1 -- 172.21.15.93:0/1324025541 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6600206e0 con 0x564c069840c0 2023-11-16T16:46:46.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.337+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1324025541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c069840c0 msgr2=0x564c06986fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.337+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1324025541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c069840c0 0x564c06986fc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe6600094f0 tx=0x7fe660009920 comp rx=0 tx=0).stop 2023-11-16T16:46:46.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.337+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1324025541 shutdown_connections 2023-11-16T16:46:46.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.337+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1324025541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c069840c0 0x564c06986fc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.337+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1324025541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06982bb0 0x564c06996bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.337+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/1324025541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c06983500 0x564c06993a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.337+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1324025541 >> 172.21.15.93:0/1324025541 conn(0x564c06978310 msgr2=0x564c069a0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.337+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1324025541 shutdown_connections 2023-11-16T16:46:46.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.337+0000 7fe6806d4b80 1 -- 172.21.15.93:0/1324025541 wait complete. 2023-11-16T16:46:46.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.337+0000 7fe6806d4b80 1 -- start start 2023-11-16T16:46:46.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.337+0000 7fe6806d4b80 1 Processor -- start 2023-11-16T16:46:46.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe6806d4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06982bb0 0x564c0698e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe6806d4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06983500 0x564c0698eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe6806d4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c069840c0 0x564c069eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe6806d4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0698f510 con 0x564c06982bb0 2023-11-16T16:46:46.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe6806d4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c069ebb00 con 0x564c06983500 2023-11-16T16:46:46.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe6806d4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c069ebc40 con 0x564c069840c0 2023-11-16T16:46:46.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe6806d4b80 10 client.0 _ll_register_callbacks cb 0x564c06893730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:46.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe66659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06982bb0 0x564c0698e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe66659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06982bb0 0x564c0698e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59026/0 (socket says 172.21.15.93:59026) 2023-11-16T16:46:46.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe66659c700 1 -- 172.21.15.93:0/2079035278 learned_addr learned my addr 172.21.15.93:0/2079035278 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:46.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe65ffff700 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06983500 0x564c0698eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe666d9d700 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c069840c0 0x564c069eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe66659c700 1 -- 172.21.15.93:0/2079035278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c069840c0 msgr2=0x564c069eb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe66659c700 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c069840c0 0x564c069eb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe66659c700 1 -- 172.21.15.93:0/2079035278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06983500 msgr2=0x564c0698eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe66659c700 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06983500 0x564c0698eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe66659c700 1 -- 172.21.15.93:0/2079035278 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6600091a0 con 0x564c06982bb0 2023-11-16T16:46:46.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe666d9d700 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c069840c0 0x564c069eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:46.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe66659c700 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06982bb0 0x564c0698e9b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe6580cb740 tx=0x7fe65809f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf539e6f1854649 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.338+0000 7fe64dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:46:46.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe64dffb700 1 -- 172.21.15.93:0/2079035278 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658021640 con 0x564c06982bb0 2023-11-16T16:46:46.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe64dffb700 1 -- 172.21.15.93:0/2079035278 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe65809b340 con 0x564c06982bb0 2023-11-16T16:46:46.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe6806d4b80 1 -- 172.21.15.93:0/2079035278 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe650004e50 con 0x564c06982bb0 2023-11-16T16:46:46.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe64dffb700 1 -- 172.21.15.93:0/2079035278 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe65809b500 con 0x564c06982bb0 2023-11-16T16:46:46.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe6806d4b80 10 client.14643 fetch_fsmap learned FSMap version 17 2023-11-16T16:46:46.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe6806d4b80 1 -- 172.21.15.93:0/2079035278 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x564c066eac10 con 0x564c06982bb0 2023-11-16T16:46:46.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe64dffb700 1 -- 172.21.15.93:0/2079035278 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe650004e50 con 0x564c06982bb0 2023-11-16T16:46:46.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe64dffb700 1 -- 172.21.15.93:0/2079035278 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe6580a3010 con 0x564c06982bb0 2023-11-16T16:46:46.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe6806d4b80 10 client.14643 fetch_fsmap finished waiting for FSMap version 17 2023-11-16T16:46:46.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe6806d4b80 10 client.14643 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-16T16:46:46.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe6806d4b80 20 client.14643 populate_metadata read hostname 'smithi093' 2023-11-16T16:46:46.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564c069f0550 0x564c069ecec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe6806d4b80 4 client.14643 mds_command: new command op to 24566 tid=0[{"prefix": "status"}] 2023-11-16T16:46:46.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.339+0000 7fe6806d4b80 1 -- 172.21.15.93:0/2079035278 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564c068a2610 con 0x564c069f0550 2023-11-16T16:46:46.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.340+0000 7fe65ffff700 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564c069f0550 0x564c069ecec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.340+0000 7fe64dffb700 10 client.14643 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:46:46.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.340+0000 7fe65ffff700 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564c069f0550 0x564c069ecec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7120436a360ead2 server_cookie=93cfba07863fffd4 in_seq=0 out_seq=0 2023-11-16T16:46:46.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.340+0000 7fe64dffb700 1 -- 172.21.15.93:0/2079035278 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x564c068a2610 con 0x564c069f0550 2023-11-16T16:46:46.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.340+0000 7fe64dffb700 10 client.14643 handle_command_reply: tid=0 2023-11-16T16:46:46.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.340+0000 7fe6806d4b80 1 client.14643 shutdown 2023-11-16T16:46:46.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.340+0000 7fe6806d4b80 1 -- 172.21.15.93:0/2079035278 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06982bb0 msgr2=0x564c0698e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.340+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06982bb0 0x564c0698e9b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe6580cb740 tx=0x7fe65809f110 comp rx=0 tx=0).stop 2023-11-16T16:46:46.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.340+0000 7fe6806d4b80 1 -- 172.21.15.93:0/2079035278 shutdown_connections 2023-11-16T16:46:46.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564c069f0550 0x564c069ecec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564c06982bb0 0x564c0698e9b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c069840c0 0x564c069eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe64dffb700 0 client.14643 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:46:46.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe6806d4b80 1 --2- 172.21.15.93:0/2079035278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564c06983500 0x564c0698eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe6806d4b80 1 -- 172.21.15.93:0/2079035278 >> 172.21.15.93:0/2079035278 conn(0x564c06978310 msgr2=0x564c06996bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe6806d4b80 1 -- 172.21.15.93:0/2079035278 shutdown_connections 2023-11-16T16:46:46.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe6806d4b80 1 -- 172.21.15.93:0/2079035278 wait complete. 2023-11-16T16:46:46.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe6806d4b80 20 client.14643 trim_cache size 0 max 16384 2023-11-16T16:46:46.353 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:46:46.353 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "6a59c1a8-849e-11ee-8dbe-212e2dc638e7", 2023-11-16T16:46:46.353 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 0, 2023-11-16T16:46:46.353 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24566, 2023-11-16T16:46:46.353 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-11-16T16:46:46.353 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-11-16T16:46:46.353 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-11-16T16:46:46.353 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 9.2137766429999992, 2023-11-16T16:46:46.354 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 17, 2023-11-16T16:46:46.354 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 88, 2023-11-16T16:46:46.354 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 87, 2023-11-16T16:46:46.354 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 16.97687844 2023-11-16T16:46:46.354 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:46:46.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe66cdff700 1 -- 172.21.15.93:0/4148451284 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe6540a11d0 msgr2=0x7fe6540a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe66cdff700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe6540a11d0 0x7fe6540a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe650005950 tx=0x7fe650011950 comp rx=0 tx=0).stop 2023-11-16T16:46:46.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe66cdff700 1 -- 172.21.15.93:0/4148451284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe668073550 msgr2=0x7fe668198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe66cdff700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe668073550 0x7fe668198200 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe65800b680 tx=0x7fe658009580 comp rx=0 tx=0).stop 2023-11-16T16:46:46.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe66cdff700 1 -- 172.21.15.93:0/4148451284 shutdown_connections 2023-11-16T16:46:46.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe66cdff700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe6540a11d0 0x7fe6540a3680 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe66cdff700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6681080f0 0x7fe66819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe66cdff700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe668073550 0x7fe668198200 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe66cdff700 1 --2- 172.21.15.93:0/4148451284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe668074b30 0x7fe668198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.341+0000 7fe66cdff700 1 -- 172.21.15.93:0/4148451284 >> 172.21.15.93:0/4148451284 conn(0x7fe6680fb810 msgr2=0x7fe6680fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.342+0000 7fe66cdff700 1 -- 172.21.15.93:0/4148451284 shutdown_connections 2023-11-16T16:46:46.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:46.342+0000 7fe66cdff700 1 -- 172.21.15.93:0/4148451284 wait complete. 2023-11-16T16:46:46.359 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:46 smithi093 ceph-mon[97863]: pgmap v285: 129 pgs: 1 creating+activating, 4 creating+peering, 118 active+clean, 6 unknown; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 7 op/s 2023-11-16T16:46:46.359 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:46 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3802476747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:46.359 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:46 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2503870543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:46.361 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-16T16:46:46.362 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-16T16:46:46.362 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph fs ls 2023-11-16T16:46:46.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:46 smithi159 ceph-mon[131528]: pgmap v285: 129 pgs: 1 creating+activating, 4 creating+peering, 118 active+clean, 6 unknown; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 7 op/s 2023-11-16T16:46:46.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:46 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3802476747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:46.486 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:46 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2503870543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:46 smithi125 ceph-mon[131076]: pgmap v285: 129 pgs: 1 creating+activating, 4 creating+peering, 118 active+clean, 6 unknown; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 7 op/s 2023-11-16T16:46:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:46 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3802476747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:46.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:46 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2503870543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:47.203 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:46:48.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:48 smithi093 ceph-mon[97863]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 321 B/s wr, 4 op/s 2023-11-16T16:46:48.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:48 smithi159 ceph-mon[131528]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 321 B/s wr, 4 op/s 2023-11-16T16:46:48.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:48 smithi125 ceph-mon[131076]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 321 B/s wr, 4 op/s 2023-11-16T16:46:48.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.733+0000 7fb81f673700 1 -- 172.21.15.93:0/3075889775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb818100ee0 msgr2=0x7fb818104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.733+0000 7fb81f673700 1 --2- 172.21.15.93:0/3075889775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb818100ee0 0x7fb818104eb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb808009920 tx=0x7fb808009c30 comp rx=0 tx=0).stop 2023-11-16T16:46:48.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.733+0000 7fb81f673700 1 -- 172.21.15.93:0/3075889775 shutdown_connections 2023-11-16T16:46:48.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.733+0000 7fb81f673700 1 --2- 172.21.15.93:0/3075889775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8181055e0 0x7fb81810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.734+0000 7fb81f673700 1 --2- 172.21.15.93:0/3075889775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb818100540 0x7fb818100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.734+0000 7fb81f673700 1 --2- 172.21.15.93:0/3075889775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb818100ee0 0x7fb818104eb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.734+0000 7fb81f673700 1 -- 172.21.15.93:0/3075889775 >> 172.21.15.93:0/3075889775 conn(0x7fb8180fbdc0 msgr2=0x7fb8180fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:48.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.734+0000 7fb81f673700 1 -- 172.21.15.93:0/3075889775 shutdown_connections 2023-11-16T16:46:48.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.734+0000 7fb81f673700 1 -- 172.21.15.93:0/3075889775 wait complete. 2023-11-16T16:46:48.738 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.735+0000 7fb81f673700 1 Processor -- start 2023-11-16T16:46:48.738 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.735+0000 7fb81f673700 1 -- start start 2023-11-16T16:46:48.738 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.735+0000 7fb81f673700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb818100540 0x7fb818196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.738 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.735+0000 7fb81f673700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb818100ee0 0x7fb818196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.735+0000 7fb81f673700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8181055e0 0x7fb81819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.735+0000 7fb81f673700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81819b400 con 0x7fb8181055e0 2023-11-16T16:46:48.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.735+0000 7fb81f673700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81819b570 con 0x7fb818100540 2023-11-16T16:46:48.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.735+0000 7fb81f673700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81819b730 con 0x7fb818100ee0 2023-11-16T16:46:48.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.735+0000 7fb81dc10700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8181055e0 0x7fb81819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.735+0000 7fb81dc10700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8181055e0 0x7fb81819adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59730/0 (socket says 172.21.15.93:59730) 2023-11-16T16:46:48.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.735+0000 7fb81dc10700 1 -- 172.21.15.93:0/3333640761 learned_addr learned my addr 172.21.15.93:0/3333640761 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:48.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.735+0000 7fb81d40f700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb818100540 0x7fb818196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.736+0000 7fb81cc0e700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb818100ee0 0x7fb818196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.736+0000 7fb81dc10700 1 -- 172.21.15.93:0/3333640761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb818100ee0 msgr2=0x7fb818196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.736+0000 7fb81dc10700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb818100ee0 0x7fb818196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.736+0000 7fb81dc10700 1 -- 172.21.15.93:0/3333640761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb818100540 msgr2=0x7fb818196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.736+0000 7fb81dc10700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb818100540 0x7fb818196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.736+0000 7fb81dc10700 1 -- 172.21.15.93:0/3333640761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb808009520 con 0x7fb8181055e0 2023-11-16T16:46:48.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.736+0000 7fb81d40f700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb818100540 0x7fb818196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:48.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.736+0000 7fb81dc10700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8181055e0 0x7fb81819adf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb810007ee0 tx=0x7fb81000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:48.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.736+0000 7fb80e7fc700 1 -- 172.21.15.93:0/3333640761 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810017070 con 0x7fb8181055e0 2023-11-16T16:46:48.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.736+0000 7fb80e7fc700 1 -- 172.21.15.93:0/3333640761 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81000f040 con 0x7fb8181055e0 2023-11-16T16:46:48.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.736+0000 7fb81f673700 1 -- 172.21.15.93:0/3333640761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb818110360 con 0x7fb8181055e0 2023-11-16T16:46:48.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.736+0000 7fb80e7fc700 1 -- 172.21.15.93:0/3333640761 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810005740 con 0x7fb8181055e0 2023-11-16T16:46:48.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.737+0000 7fb81f673700 1 -- 172.21.15.93:0/3333640761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8181101d0 con 0x7fb8181055e0 2023-11-16T16:46:48.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.738+0000 7fb81f673700 1 -- 172.21.15.93:0/3333640761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8181104f0 con 0x7fb8181055e0 2023-11-16T16:46:48.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.739+0000 7fb80e7fc700 1 -- 172.21.15.93:0/3333640761 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb81000dbd0 con 0x7fb8181055e0 2023-11-16T16:46:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.741+0000 7fb80e7fc700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb8040a1150 0x7fb8040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.741+0000 7fb80e7fc700 1 -- 172.21.15.93:0/3333640761 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb8100cc1e0 con 0x7fb8181055e0 2023-11-16T16:46:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.741+0000 7fb81d40f700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb8040a1150 0x7fb8040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.741+0000 7fb81d40f700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb8040a1150 0x7fb8040a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb814005fd0 tx=0x7fb814009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:48.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.745+0000 7fb80e7fc700 1 -- 172.21.15.93:0/3333640761 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb810095740 con 0x7fb8181055e0 2023-11-16T16:46:48.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.984+0000 7fb81f673700 1 -- 172.21.15.93:0/3333640761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb818066ea0 con 0x7fb8181055e0 2023-11-16T16:46:48.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.984+0000 7fb80e7fc700 1 -- 172.21.15.93:0/3333640761 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb810098d60 con 0x7fb8181055e0 2023-11-16T16:46:48.986 INFO:teuthology.orchestra.run.smithi093.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-16T16:46:48.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.987+0000 7fb81f673700 1 -- 172.21.15.93:0/3333640761 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb8040a1150 msgr2=0x7fb8040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.987+0000 7fb81f673700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb8040a1150 0x7fb8040a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb814005fd0 tx=0x7fb814009310 comp rx=0 tx=0).stop 2023-11-16T16:46:48.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.987+0000 7fb81f673700 1 -- 172.21.15.93:0/3333640761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8181055e0 msgr2=0x7fb81819adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.987+0000 7fb81f673700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8181055e0 0x7fb81819adf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb810007ee0 tx=0x7fb81000d3b0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.987+0000 7fb81f673700 1 -- 172.21.15.93:0/3333640761 shutdown_connections 2023-11-16T16:46:48.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.987+0000 7fb81f673700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb8040a1150 0x7fb8040a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.987+0000 7fb81f673700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8181055e0 0x7fb81819adf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.987+0000 7fb81f673700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb818100540 0x7fb818196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.987+0000 7fb81f673700 1 --2- 172.21.15.93:0/3333640761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb818100ee0 0x7fb818196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.987+0000 7fb81f673700 1 -- 172.21.15.93:0/3333640761 >> 172.21.15.93:0/3333640761 conn(0x7fb8180fbdc0 msgr2=0x7fb8180fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:48.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.987+0000 7fb81f673700 1 -- 172.21.15.93:0/3333640761 shutdown_connections 2023-11-16T16:46:48.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:48.987+0000 7fb81f673700 1 -- 172.21.15.93:0/3333640761 wait complete. 2023-11-16T16:46:49.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:49 smithi093 ceph-mon[97863]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 1 op/s 2023-11-16T16:46:49.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:49 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3333640761' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-16T16:46:49.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:49 smithi159 ceph-mon[131528]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 1 op/s 2023-11-16T16:46:49.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:49 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3333640761' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-16T16:46:49.486 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-16T16:46:49.487 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-16T16:46:49.487 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi093.front.sepia.ceph.com 2023-11-16T16:46:49.487 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-16T16:46:49.487 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-16T16:46:49.487 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-16T16:46:49.487 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-16T16:46:49.487 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-16T16:46:49.487 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-16T16:46:49.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:49.488 DEBUG:teuthology.orchestra.run.smithi093:> ip addr 2023-11-16T16:46:49.533 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-16T16:46:49.533 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-16T16:46:49.533 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1c brd ff:ff:ff:ff:ff:ff 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f0 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1d brd ff:ff:ff:ff:ff:ff 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f1 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:34 brd ff:ff:ff:ff:ff:ff 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-16T16:46:49.534 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:35 brd ff:ff:ff:ff:ff:ff 2023-11-16T16:46:49.535 INFO:teuthology.orchestra.run.smithi093.stdout: inet 172.21.15.93/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-16T16:46:49.535 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft 41585sec preferred_lft 41585sec 2023-11-16T16:46:49.535 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::2b1e:922d:25d9:39b2/64 scope link noprefixroute 2023-11-16T16:46:49.535 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-16T16:46:49.535 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-16T16:46:49.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:49.536 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-16T16:46:49.536 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-16T16:46:49.536 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add name ceph-brx type bridge 2023-11-16T16:46:49.536 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr flush dev ceph-brx 2023-11-16T16:46:49.536 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set ceph-brx up 2023-11-16T16:46:49.536 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-16T16:46:49.536 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-16T16:46:49.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:49 smithi125 ceph-mon[131076]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 1 op/s 2023-11-16T16:46:49.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:49 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3333640761' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-16T16:46:49.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:49.998 DEBUG:teuthology.orchestra.run.smithi093:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-16T16:46:50.069 INFO:teuthology.orchestra.run.smithi093.stdout:1 2023-11-16T16:46:50.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:50.073 DEBUG:teuthology.orchestra.run.smithi093:> route 2023-11-16T16:46:50.130 INFO:teuthology.orchestra.run.smithi093.stdout:Kernel IP routing table 2023-11-16T16:46:50.130 INFO:teuthology.orchestra.run.smithi093.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-16T16:46:50.130 INFO:teuthology.orchestra.run.smithi093.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-16T16:46:50.130 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-16T16:46:50.130 INFO:teuthology.orchestra.run.smithi093.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-16T16:46:50.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:50.132 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-16T16:46:50.132 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-16T16:46:50.132 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-16T16:46:50.132 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-16T16:46:50.132 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-16T16:46:50.132 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-16T16:46:50.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:50.817 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-11-16T16:46:50.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:50.861 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list-id 2023-11-16T16:46:50.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:50.945 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-16T16:46:50.945 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-16T16:46:50.946 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-16T16:46:50.946 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-16T16:46:50.946 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-16T16:46:51.129 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-16T16:46:51.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:51.130 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-16T16:46:51.130 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-16T16:46:51.130 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-16T16:46:51.130 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-16T16:46:51.130 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-16T16:46:51.130 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-16T16:46:51.130 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-16T16:46:51.130 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-16T16:46:51.344 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-11-16T16:46:51.344 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-11-16T16:46:51.344 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-11-16T16:46:51.344 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-11-16T16:46:51.344 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]: dispatch 2023-11-16T16:46:51.344 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-11-16T16:46:51.345 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2023-11-16T16:46:51.345 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2023-11-16T16:46:51.345 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-11-16T16:46:51.345 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:51 smithi093 ceph-mon[97863]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 109 B/s wr, 0 op/s 2023-11-16T16:46:51.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:51.420 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-16T16:46:51.420 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-16T16:46:51.420 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set brx.0 up 2023-11-16T16:46:51.420 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set dev brx.0 master ceph-brx 2023-11-16T16:46:51.420 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-16T16:46:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-11-16T16:46:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-11-16T16:46:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-11-16T16:46:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-11-16T16:46:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]: dispatch 2023-11-16T16:46:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-11-16T16:46:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2023-11-16T16:46:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2023-11-16T16:46:51.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-11-16T16:46:51.579 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:51 smithi125 ceph-mon[131076]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 109 B/s wr, 0 op/s 2023-11-16T16:46:51.593 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-16T16:46:51.593 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-16T16:46:51.594 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-16T16:46:51.637 INFO:teuthology.orchestra.run.smithi093.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-16T16:46:51.637 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-16T16:46:51.638 DEBUG:teuthology.orchestra.run.smithi093:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-16T16:46:51.722 DEBUG:teuthology.orchestra.run.smithi093:> sudo modprobe fuse 2023-11-16T16:46:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-11-16T16:46:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-11-16T16:46:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-11-16T16:46:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-11-16T16:46:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]: dispatch 2023-11-16T16:46:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-11-16T16:46:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2023-11-16T16:46:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2023-11-16T16:46:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-11-16T16:46:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:51 smithi159 ceph-mon[131528]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 109 B/s wr, 0 op/s 2023-11-16T16:46:51.834 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-16T16:46:51.834 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-16T16:46:51.901 INFO:teuthology.orchestra.run.smithi093.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-16T16:46:51.904 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-16T16:46:51.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:51.905 DEBUG:teuthology.orchestra.run.smithi093:> ls /sys/fs/fuse/connections 2023-11-16T16:46:51.949 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-16T16:46:51.949 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-11-16T16:46:51.993 DEBUG:teuthology.orchestra.run.smithi093:> sudo modprobe fuse 2023-11-16T16:46:52.063 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-16T16:46:52.063 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-16T16:46:52.131 INFO:teuthology.orchestra.run.smithi093.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-16T16:46:52.134 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-16T16:46:52.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:52.135 DEBUG:teuthology.orchestra.run.smithi093:> ls /sys/fs/fuse/connections 2023-11-16T16:46:52.195 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi093.stderr:2023-11-16T16:46:52.193+0000 7fe32b9985c0 -1 init, newargv = 0x55f4362b2370 newargc=15 2023-11-16T16:46:52.195 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi093.stderr:ceph-fuse[134685]: starting ceph client 2023-11-16T16:46:52.204 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi093.stderr:ceph-fuse[134685]: starting fuse 2023-11-16T16:46:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-11-16T16:46:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-11-16T16:46:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-11-16T16:46:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-11-16T16:46:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]': finished 2023-11-16T16:46:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-11-16T16:46:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2023-11-16T16:46:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2023-11-16T16:46:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-11-16T16:46:52.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:52 smithi125 ceph-mon[131076]: osdmap e89: 12 total, 12 up, 12 in 2023-11-16T16:46:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-11-16T16:46:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-11-16T16:46:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-11-16T16:46:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-11-16T16:46:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]': finished 2023-11-16T16:46:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-11-16T16:46:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2023-11-16T16:46:52.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2023-11-16T16:46:52.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-11-16T16:46:52.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:52 smithi093 ceph-mon[97863]: osdmap e89: 12 total, 12 up, 12 in 2023-11-16T16:46:52.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-11-16T16:46:52.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-11-16T16:46:52.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-11-16T16:46:52.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-11-16T16:46:52.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]': finished 2023-11-16T16:46:52.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-11-16T16:46:52.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2023-11-16T16:46:52.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2023-11-16T16:46:52.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-11-16T16:46:52.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:52 smithi159 ceph-mon[131528]: osdmap e89: 12 total, 12 up, 12 in 2023-11-16T16:46:53.280 DEBUG:teuthology.orchestra.run.smithi093:> sudo modprobe fuse 2023-11-16T16:46:53.361 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-16T16:46:53.361 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-16T16:46:53.431 INFO:teuthology.orchestra.run.smithi093.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-16T16:46:53.434 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-16T16:46:53.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:53.435 DEBUG:teuthology.orchestra.run.smithi093:> ls /sys/fs/fuse/connections 2023-11-16T16:46:53.479 INFO:teuthology.orchestra.run.smithi093.stdout:121 2023-11-16T16:46:53.479 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-11-16T16:46:53.479 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-16T16:46:53.479 DEBUG:teuthology.orchestra.run.smithi093:> import glob 2023-11-16T16:46:53.479 DEBUG:teuthology.orchestra.run.smithi093:> import re 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> import os 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> import subprocess 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> def _find_admin_socket(client_name): 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> files = glob.glob(asok_path) 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> # Given a non-glob path, it better be there 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> if "*" not in asok_path: 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> assert(len(files) == 1) 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> return files[0] 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> 2023-11-16T16:46:53.480 DEBUG:teuthology.orchestra.run.smithi093:> for f in files: 2023-11-16T16:46:53.481 DEBUG:teuthology.orchestra.run.smithi093:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-16T16:46:53.481 DEBUG:teuthology.orchestra.run.smithi093:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-16T16:46:53.481 DEBUG:teuthology.orchestra.run.smithi093:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-16T16:46:53.481 DEBUG:teuthology.orchestra.run.smithi093:> contents = proc_f.read() 2023-11-16T16:46:53.481 DEBUG:teuthology.orchestra.run.smithi093:> if mountpoint in contents: 2023-11-16T16:46:53.481 DEBUG:teuthology.orchestra.run.smithi093:> return f 2023-11-16T16:46:53.481 DEBUG:teuthology.orchestra.run.smithi093:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-16T16:46:53.481 DEBUG:teuthology.orchestra.run.smithi093:> 2023-11-16T16:46:53.481 DEBUG:teuthology.orchestra.run.smithi093:> print(_find_admin_socket("client.admin")) 2023-11-16T16:46:53.481 DEBUG:teuthology.orchestra.run.smithi093:> ' 2023-11-16T16:46:53.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:53 smithi125 ceph-mon[131076]: osdmap e90: 12 total, 12 up, 12 in 2023-11-16T16:46:53.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:53 smithi125 ceph-mon[131076]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:53.578 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:53 smithi093 ceph-mon[97863]: osdmap e90: 12 total, 12 up, 12 in 2023-11-16T16:46:53.579 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:53 smithi093 ceph-mon[97863]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:53.662 INFO:teuthology.orchestra.run.smithi093.stdout:/var/run/ceph/ceph-client.admin.134685.asok 2023-11-16T16:46:53.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:53 smithi159 ceph-mon[131528]: osdmap e90: 12 total, 12 up, 12 in 2023-11-16T16:46:53.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:53 smithi159 ceph-mon[131528]: pgmap v291: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:46:53.840 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.134685.asok 2023-11-16T16:46:53.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:53.840 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.134685.asok status 2023-11-16T16:46:54.065 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:46:54.065 INFO:teuthology.orchestra.run.smithi093.stdout: "metadata": { 2023-11-16T16:46:54.065 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_sha1": "5e4a743ad7fd1422ebd6c4dc950a21a303279207", 2023-11-16T16:46:54.066 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_version": "ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev)", 2023-11-16T16:46:54.066 INFO:teuthology.orchestra.run.smithi093.stdout: "entity_id": "admin", 2023-11-16T16:46:54.066 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": "smithi093", 2023-11-16T16:46:54.066 INFO:teuthology.orchestra.run.smithi093.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-16T16:46:54.066 INFO:teuthology.orchestra.run.smithi093.stdout: "pid": "134685", 2023-11-16T16:46:54.066 INFO:teuthology.orchestra.run.smithi093.stdout: "root": "/" 2023-11-16T16:46:54.066 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:46:54.066 INFO:teuthology.orchestra.run.smithi093.stdout: "dentry_count": 0, 2023-11-16T16:46:54.066 INFO:teuthology.orchestra.run.smithi093.stdout: "dentry_pinned_count": 0, 2023-11-16T16:46:54.066 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 14655, 2023-11-16T16:46:54.067 INFO:teuthology.orchestra.run.smithi093.stdout: "inst": { 2023-11-16T16:46:54.067 INFO:teuthology.orchestra.run.smithi093.stdout: "name": { 2023-11-16T16:46:54.067 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "client", 2023-11-16T16:46:54.067 INFO:teuthology.orchestra.run.smithi093.stdout: "num": 14655 2023-11-16T16:46:54.067 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:46:54.067 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": { 2023-11-16T16:46:54.067 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v1", 2023-11-16T16:46:54.067 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "192.168.0.1:0", 2023-11-16T16:46:54.067 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 3908601814 2023-11-16T16:46:54.067 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:46:54.067 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:46:54.067 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": { 2023-11-16T16:46:54.068 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v1", 2023-11-16T16:46:54.068 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "192.168.0.1:0", 2023-11-16T16:46:54.068 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 3908601814 2023-11-16T16:46:54.068 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:46:54.068 INFO:teuthology.orchestra.run.smithi093.stdout: "inst_str": "client.14655 192.168.0.1:0/3908601814", 2023-11-16T16:46:54.068 INFO:teuthology.orchestra.run.smithi093.stdout: "addr_str": "192.168.0.1:0/3908601814", 2023-11-16T16:46:54.068 INFO:teuthology.orchestra.run.smithi093.stdout: "inode_count": 1, 2023-11-16T16:46:54.068 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_epoch": 17, 2023-11-16T16:46:54.068 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_epoch": 89, 2023-11-16T16:46:54.068 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_epoch_barrier": 0, 2023-11-16T16:46:54.068 INFO:teuthology.orchestra.run.smithi093.stdout: "blocklisted": false, 2023-11-16T16:46:54.068 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs" 2023-11-16T16:46:54.069 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:46:54.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:54.080 DEBUG:teuthology.orchestra.run.smithi093:> stat --file-system '--printf=%T 2023-11-16T16:46:54.080 DEBUG:teuthology.orchestra.run.smithi093:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-16T16:46:54.124 INFO:teuthology.orchestra.run.smithi093.stdout:fuseblk 2023-11-16T16:46:54.125 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-16T16:46:54.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:54.125 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-16T16:46:54.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:54.235 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-16T16:46:54.377 INFO:teuthology.orchestra.run.smithi093.stdout:# file: . 2023-11-16T16:46:54.377 INFO:teuthology.orchestra.run.smithi093.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-16T16:46:54.377 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:54.378 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:46:54.378 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-16T16:46:54.422 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:46:54.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.660+0000 7f74497ff700 1 -- 172.21.15.93:0/176801964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74441002b0 msgr2=0x7f74441006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:54.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.660+0000 7f74497ff700 1 --2- 172.21.15.93:0/176801964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74441002b0 0x7f74441006c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7434009990 tx=0x7f7434009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:54.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.660+0000 7f74497ff700 1 -- 172.21.15.93:0/176801964 shutdown_connections 2023-11-16T16:46:54.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.660+0000 7f74497ff700 1 --2- 172.21.15.93:0/176801964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7444101e30 0x7f74441086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:54.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.660+0000 7f74497ff700 1 --2- 172.21.15.93:0/176801964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74441002b0 0x7f74441006c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:54.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.660+0000 7f74497ff700 1 --2- 172.21.15.93:0/176801964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74441014a0 0x7f74441018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:54.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.660+0000 7f74497ff700 1 -- 172.21.15.93:0/176801964 >> 172.21.15.93:0/176801964 conn(0x7f74440fb890 msgr2=0x7f74440fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:54.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.660+0000 7f74497ff700 1 -- 172.21.15.93:0/176801964 shutdown_connections 2023-11-16T16:46:54.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.660+0000 7f74497ff700 1 -- 172.21.15.93:0/176801964 wait complete. 2023-11-16T16:46:54.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.661+0000 7f74497ff700 1 Processor -- start 2023-11-16T16:46:54.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.661+0000 7f74497ff700 1 -- start start 2023-11-16T16:46:54.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.661+0000 7f74497ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74441002b0 0x7f74440748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:54.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.661+0000 7f74497ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74441014a0 0x7f7444072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:54.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.661+0000 7f74497ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7444101e30 0x7f7444073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:54.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.661+0000 7f74497ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7444073bc0 con 0x7f74441014a0 2023-11-16T16:46:54.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f74497ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7444073d30 con 0x7f74441002b0 2023-11-16T16:46:54.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f74497ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74441a0080 con 0x7f7444101e30 2023-11-16T16:46:54.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f74427fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74441014a0 0x7f7444072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:54.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f74437fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7444101e30 0x7f7444073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:54.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f74427fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74441014a0 0x7f7444072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59752/0 (socket says 172.21.15.93:59752) 2023-11-16T16:46:54.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f74427fc700 1 -- 172.21.15.93:0/2469746820 learned_addr learned my addr 172.21.15.93:0/2469746820 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:54.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f7442ffd700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74441002b0 0x7f74440748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:54.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f74427fc700 1 -- 172.21.15.93:0/2469746820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7444101e30 msgr2=0x7f7444073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:54.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f74427fc700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7444101e30 0x7f7444073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:54.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f74427fc700 1 -- 172.21.15.93:0/2469746820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74441002b0 msgr2=0x7f74440748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:54.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f74427fc700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74441002b0 0x7f74440748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:54.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f74427fc700 1 -- 172.21.15.93:0/2469746820 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74340095f0 con 0x7f74441014a0 2023-11-16T16:46:54.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.662+0000 7f74427fc700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74441014a0 0x7f7444072ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f743800e9a0 tx=0x7f743800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:54.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.663+0000 7f742bfff700 1 -- 172.21.15.93:0/2469746820 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438009790 con 0x7f74441014a0 2023-11-16T16:46:54.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.663+0000 7f74497ff700 1 -- 172.21.15.93:0/2469746820 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744410bc10 con 0x7f74441014a0 2023-11-16T16:46:54.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.663+0000 7f742bfff700 1 -- 172.21.15.93:0/2469746820 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7438010430 con 0x7f74441014a0 2023-11-16T16:46:54.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.663+0000 7f742bfff700 1 -- 172.21.15.93:0/2469746820 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438018620 con 0x7f74441014a0 2023-11-16T16:46:54.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.663+0000 7f74497ff700 1 -- 172.21.15.93:0/2469746820 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744410ba80 con 0x7f74441014a0 2023-11-16T16:46:54.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.664+0000 7f74497ff700 1 -- 172.21.15.93:0/2469746820 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74441078c0 con 0x7f74441014a0 2023-11-16T16:46:54.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.666+0000 7f742bfff700 1 -- 172.21.15.93:0/2469746820 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f74380105a0 con 0x7f74441014a0 2023-11-16T16:46:54.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.667+0000 7f742bfff700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f742c0a1150 0x7f742c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:54.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.667+0000 7f742bfff700 1 -- 172.21.15.93:0/2469746820 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f7438014070 con 0x7f74441014a0 2023-11-16T16:46:54.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.667+0000 7f7442ffd700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f742c0a1150 0x7f742c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:54.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.668+0000 7f7442ffd700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f742c0a1150 0x7f742c0a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f743400b2e0 tx=0x7f7434005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:54.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.669+0000 7f742bfff700 1 -- 172.21.15.93:0/2469746820 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f74380963e0 con 0x7f74441014a0 2023-11-16T16:46:54.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.871+0000 7f74497ff700 1 -- 172.21.15.93:0/2469746820 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f744402cfe0 con 0x7f74441014a0 2023-11-16T16:46:54.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.873+0000 7f742bfff700 1 -- 172.21.15.93:0/2469746820 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24468 (secure 0 0 0) 0x7f7438026090 con 0x7f74441014a0 2023-11-16T16:46:54.876 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:54.876 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":90,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:46:52.315313+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:46:54.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.875+0000 7f74497ff700 1 -- 172.21.15.93:0/2469746820 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f742c0a1150 msgr2=0x7f742c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:54.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.875+0000 7f74497ff700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f742c0a1150 0x7f742c0a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f743400b2e0 tx=0x7f7434005960 comp rx=0 tx=0).stop 2023-11-16T16:46:54.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.875+0000 7f74497ff700 1 -- 172.21.15.93:0/2469746820 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74441014a0 msgr2=0x7f7444072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:54.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.875+0000 7f74497ff700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74441014a0 0x7f7444072ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f743800e9a0 tx=0x7f743800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:46:54.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.875+0000 7f74497ff700 1 -- 172.21.15.93:0/2469746820 shutdown_connections 2023-11-16T16:46:54.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.875+0000 7f74497ff700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f742c0a1150 0x7f742c0a3600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:54.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.875+0000 7f74497ff700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7444101e30 0x7f7444073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:54.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.875+0000 7f74497ff700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74441002b0 0x7f74440748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:54.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.875+0000 7f74497ff700 1 --2- 172.21.15.93:0/2469746820 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74441014a0 0x7f7444072ee0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:54.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.875+0000 7f74497ff700 1 -- 172.21.15.93:0/2469746820 >> 172.21.15.93:0/2469746820 conn(0x7f74440fb890 msgr2=0x7f7444104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:54.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.876+0000 7f74497ff700 1 -- 172.21.15.93:0/2469746820 shutdown_connections 2023-11-16T16:46:54.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:54.876+0000 7f74497ff700 1 -- 172.21.15.93:0/2469746820 wait complete. 2023-11-16T16:46:54.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:42:13.029135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:46:35.301321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:46:36.552545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-16T16:46:39.628380+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-16T16:46:54.897 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:46:55.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.095+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2268351456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9941002c0 msgr2=0x7fc9941006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.095+0000 7fc99a7ff700 1 --2- 172.21.15.93:0/2268351456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9941002c0 0x7fc9941006d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc984009990 tx=0x7fc984009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.096+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2268351456 shutdown_connections 2023-11-16T16:46:55.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.096+0000 7fc99a7ff700 1 --2- 172.21.15.93:0/2268351456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9941002c0 0x7fc9941006d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.096+0000 7fc99a7ff700 1 --2- 172.21.15.93:0/2268351456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc994101e40 0x7fc994108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.096+0000 7fc99a7ff700 1 --2- 172.21.15.93:0/2268351456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9941014b0 0x7fc994101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.096+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2268351456 >> 172.21.15.93:0/2268351456 conn(0x7fc9940fb8a0 msgr2=0x7fc9940fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:55.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.096+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2268351456 shutdown_connections 2023-11-16T16:46:55.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.096+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2268351456 wait complete. 2023-11-16T16:46:55.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.097+0000 7fc99a7ff700 1 Processor -- start 2023-11-16T16:46:55.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.097+0000 7fc99a7ff700 1 -- start start 2023-11-16T16:46:55.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.097+0000 7fc99a7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9941002c0 0x7fc9940748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:55.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.097+0000 7fc99a7ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9941014b0 0x7fc994072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:55.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.097+0000 7fc99a7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc994101e40 0x7fc994073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:55.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.097+0000 7fc99a7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994073bc0 con 0x7fc9941002c0 2023-11-16T16:46:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.097+0000 7fc99a7ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994073d30 con 0x7fc9941014b0 2023-11-16T16:46:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.097+0000 7fc99a7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941a00f0 con 0x7fc994101e40 2023-11-16T16:46:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.097+0000 7fc993fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9941002c0 0x7fc9940748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.097+0000 7fc993fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9941002c0 0x7fc9940748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59770/0 (socket says 172.21.15.93:59770) 2023-11-16T16:46:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.097+0000 7fc993fff700 1 -- 172.21.15.93:0/2638872592 learned_addr learned my addr 172.21.15.93:0/2638872592 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc9937fe700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9941014b0 0x7fc994072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc998d9c700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc994101e40 0x7fc994073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc993fff700 1 -- 172.21.15.93:0/2638872592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc994101e40 msgr2=0x7fc994073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc993fff700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc994101e40 0x7fc994073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc993fff700 1 -- 172.21.15.93:0/2638872592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9941014b0 msgr2=0x7fc994072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc993fff700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9941014b0 0x7fc994072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc993fff700 1 -- 172.21.15.93:0/2638872592 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c009c40 con 0x7fc9941002c0 2023-11-16T16:46:55.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc998d9c700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc994101e40 0x7fc994073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:55.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc993fff700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9941002c0 0x7fc9940748a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc984005740 tx=0x7fc984005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:55.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc9917fa700 1 -- 172.21.15.93:0/2638872592 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98401d070 con 0x7fc9941002c0 2023-11-16T16:46:55.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc9917fa700 1 -- 172.21.15.93:0/2638872592 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc984021410 con 0x7fc9941002c0 2023-11-16T16:46:55.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc9917fa700 1 -- 172.21.15.93:0/2638872592 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984017650 con 0x7fc9941002c0 2023-11-16T16:46:55.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2638872592 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9840095f0 con 0x7fc9941002c0 2023-11-16T16:46:55.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.098+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2638872592 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99410bc90 con 0x7fc9941002c0 2023-11-16T16:46:55.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.099+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2638872592 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc994107850 con 0x7fc9941002c0 2023-11-16T16:46:55.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.101+0000 7fc9917fa700 1 -- 172.21.15.93:0/2638872592 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc984021580 con 0x7fc9941002c0 2023-11-16T16:46:55.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.103+0000 7fc9917fa700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc97c0a1150 0x7fc97c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:55.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.103+0000 7fc9917fa700 1 -- 172.21.15.93:0/2638872592 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc9840cdff0 con 0x7fc9941002c0 2023-11-16T16:46:55.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.103+0000 7fc9937fe700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc97c0a1150 0x7fc97c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:55.106 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.103+0000 7fc9937fe700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc97c0a1150 0x7fc97c0a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc988009c60 tx=0x7fc988009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:55.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.108+0000 7fc9917fa700 1 -- 172.21.15.93:0/2638872592 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc984097520 con 0x7fc9941002c0 2023-11-16T16:46:55.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.306+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2638872592 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc994060f30 con 0x7fc9941002c0 2023-11-16T16:46:55.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.307+0000 7fc9917fa700 1 -- 172.21.15.93:0/2638872592 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc98409ab40 con 0x7fc9941002c0 2023-11-16T16:46:55.309 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-11-16T16:46:55.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.309+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2638872592 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc97c0a1150 msgr2=0x7fc97c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.309+0000 7fc99a7ff700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc97c0a1150 0x7fc97c0a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc988009c60 tx=0x7fc988009260 comp rx=0 tx=0).stop 2023-11-16T16:46:55.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.309+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2638872592 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9941002c0 msgr2=0x7fc9940748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.309+0000 7fc99a7ff700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9941002c0 0x7fc9940748a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc984005740 tx=0x7fc984005770 comp rx=0 tx=0).stop 2023-11-16T16:46:55.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.309+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2638872592 shutdown_connections 2023-11-16T16:46:55.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.309+0000 7fc99a7ff700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc97c0a1150 0x7fc97c0a3600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.309+0000 7fc99a7ff700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9941002c0 0x7fc9940748a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.309+0000 7fc99a7ff700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc994101e40 0x7fc994073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.309+0000 7fc99a7ff700 1 --2- 172.21.15.93:0/2638872592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9941014b0 0x7fc994072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.309+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2638872592 >> 172.21.15.93:0/2638872592 conn(0x7fc9940fb8a0 msgr2=0x7fc994104990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.310+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2638872592 shutdown_connections 2023-11-16T16:46:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.310+0000 7fc99a7ff700 1 -- 172.21.15.93:0/2638872592 wait complete. 2023-11-16T16:46:55.330 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:46:55.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.529+0000 7f10743ff700 1 -- 172.21.15.93:0/158921628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f106c1002c0 msgr2=0x7f106c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.529+0000 7f10743ff700 1 --2- 172.21.15.93:0/158921628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f106c1002c0 0x7f106c1006d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1068009990 tx=0x7f1068009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.529+0000 7f10743ff700 1 -- 172.21.15.93:0/158921628 shutdown_connections 2023-11-16T16:46:55.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.529+0000 7f10743ff700 1 --2- 172.21.15.93:0/158921628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f106c1002c0 0x7f106c1006d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.529+0000 7f10743ff700 1 --2- 172.21.15.93:0/158921628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106c101e40 0x7f106c108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.529+0000 7f10743ff700 1 --2- 172.21.15.93:0/158921628 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f106c1014b0 0x7f106c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.529+0000 7f10743ff700 1 -- 172.21.15.93:0/158921628 >> 172.21.15.93:0/158921628 conn(0x7f106c0fb8a0 msgr2=0x7f106c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:55.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.529+0000 7f10743ff700 1 -- 172.21.15.93:0/158921628 shutdown_connections 2023-11-16T16:46:55.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.529+0000 7f10743ff700 1 -- 172.21.15.93:0/158921628 wait complete. 2023-11-16T16:46:55.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.530+0000 7f10743ff700 1 Processor -- start 2023-11-16T16:46:55.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.531+0000 7f10743ff700 1 -- start start 2023-11-16T16:46:55.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.531+0000 7f10743ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f106c1002c0 0x7f106c196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:55.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.531+0000 7f10743ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f106c1014b0 0x7f106c1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:55.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.531+0000 7f10743ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106c101e40 0x7f106c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:55.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.531+0000 7f10743ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c19dc10 con 0x7f106c1002c0 2023-11-16T16:46:55.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.531+0000 7f10743ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c19dd50 con 0x7f106c1014b0 2023-11-16T16:46:55.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.531+0000 7f10743ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c19dee0 con 0x7f106c101e40 2023-11-16T16:46:55.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.531+0000 7f107219b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f106c1002c0 0x7f106c196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:55.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.531+0000 7f107219b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f106c1002c0 0x7f106c196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59784/0 (socket says 172.21.15.93:59784) 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.531+0000 7f107219b700 1 -- 172.21.15.93:0/371716311 learned_addr learned my addr 172.21.15.93:0/371716311 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.531+0000 7f107299c700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106c101e40 0x7f106c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.531+0000 7f107199a700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f106c1014b0 0x7f106c1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.532+0000 7f107219b700 1 -- 172.21.15.93:0/371716311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106c101e40 msgr2=0x7f106c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.532+0000 7f107219b700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106c101e40 0x7f106c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.532+0000 7f107219b700 1 -- 172.21.15.93:0/371716311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f106c1014b0 msgr2=0x7f106c1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.532+0000 7f107219b700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f106c1014b0 0x7f106c1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.532+0000 7f107219b700 1 -- 172.21.15.93:0/371716311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10680095f0 con 0x7f106c1002c0 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.532+0000 7f107219b700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f106c1002c0 0x7f106c196090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f106800b620 tx=0x7f1068005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.532+0000 7f10637fe700 1 -- 172.21.15.93:0/371716311 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106801d070 con 0x7f106c1002c0 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.532+0000 7f10637fe700 1 -- 172.21.15.93:0/371716311 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1068021410 con 0x7f106c1002c0 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.532+0000 7f10637fe700 1 -- 172.21.15.93:0/371716311 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068017650 con 0x7f106c1002c0 2023-11-16T16:46:55.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.532+0000 7f10743ff700 1 -- 172.21.15.93:0/371716311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c10bc90 con 0x7f106c1002c0 2023-11-16T16:46:55.537 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.532+0000 7f10743ff700 1 -- 172.21.15.93:0/371716311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c10bb00 con 0x7f106c1002c0 2023-11-16T16:46:55.537 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.533+0000 7f10743ff700 1 -- 172.21.15.93:0/371716311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106c1071f0 con 0x7f106c1002c0 2023-11-16T16:46:55.538 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.535+0000 7f10637fe700 1 -- 172.21.15.93:0/371716311 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1068021580 con 0x7f106c1002c0 2023-11-16T16:46:55.539 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.537+0000 7f10637fe700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f10580a1150 0x7f10580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:55.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.537+0000 7f10637fe700 1 -- 172.21.15.93:0/371716311 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f10680ce140 con 0x7f106c1002c0 2023-11-16T16:46:55.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.537+0000 7f107199a700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f10580a1150 0x7f10580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:55.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.537+0000 7f107199a700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f10580a1150 0x7f10580a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f105c009c60 tx=0x7f105c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:55.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.541+0000 7f10637fe700 1 -- 172.21.15.93:0/371716311 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f10680975c0 con 0x7f106c1002c0 2023-11-16T16:46:55.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:55 smithi093 ceph-mon[97863]: pgmap v292: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-11-16T16:46:55.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:55 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2469746820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:55.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:55 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2638872592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:46:55.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.742+0000 7f10743ff700 1 -- 172.21.15.93:0/371716311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f106c061c40 con 0x7f106c1002c0 2023-11-16T16:46:55.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.742+0000 7f10637fe700 1 -- 172.21.15.93:0/371716311 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f106809abe0 con 0x7f106c1002c0 2023-11-16T16:46:55.744 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:46:55.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.744+0000 7f10743ff700 1 -- 172.21.15.93:0/371716311 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f10580a1150 msgr2=0x7f10580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.744+0000 7f10743ff700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f10580a1150 0x7f10580a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f105c009c60 tx=0x7f105c009260 comp rx=0 tx=0).stop 2023-11-16T16:46:55.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.744+0000 7f10743ff700 1 -- 172.21.15.93:0/371716311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f106c1002c0 msgr2=0x7f106c196090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.744+0000 7f10743ff700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f106c1002c0 0x7f106c196090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f106800b620 tx=0x7f1068005740 comp rx=0 tx=0).stop 2023-11-16T16:46:55.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.744+0000 7f10743ff700 1 -- 172.21.15.93:0/371716311 shutdown_connections 2023-11-16T16:46:55.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.744+0000 7f10743ff700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f10580a1150 0x7f10580a3600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.744+0000 7f10743ff700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f106c1002c0 0x7f106c196090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.744+0000 7f10743ff700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106c101e40 0x7f106c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.744+0000 7f10743ff700 1 --2- 172.21.15.93:0/371716311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f106c1014b0 0x7f106c1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.745+0000 7f10743ff700 1 -- 172.21.15.93:0/371716311 >> 172.21.15.93:0/371716311 conn(0x7f106c0fb8a0 msgr2=0x7f106c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:55.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.745+0000 7f10743ff700 1 -- 172.21.15.93:0/371716311 shutdown_connections 2023-11-16T16:46:55.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.745+0000 7f10743ff700 1 -- 172.21.15.93:0/371716311 wait complete. 2023-11-16T16:46:55.764 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:46:55.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:55 smithi125 ceph-mon[131076]: pgmap v292: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-11-16T16:46:55.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:55 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2469746820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:55.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:55 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2638872592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:46:55.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.963+0000 7efff55ff700 1 -- 172.21.15.93:0/4071153419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff0108180 msgr2=0x7efff010a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.963+0000 7efff55ff700 1 --2- 172.21.15.93:0/4071153419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff0108180 0x7efff010a600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7effe800b0a0 tx=0x7effe800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.963+0000 7efff55ff700 1 -- 172.21.15.93:0/4071153419 shutdown_connections 2023-11-16T16:46:55.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.963+0000 7efff55ff700 1 --2- 172.21.15.93:0/4071153419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff0108180 0x7efff010a600 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.963+0000 7efff55ff700 1 --2- 172.21.15.93:0/4071153419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efff0074b30 0x7efff0072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.963+0000 7efff55ff700 1 --2- 172.21.15.93:0/4071153419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff0073550 0x7efff00739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.963+0000 7efff55ff700 1 -- 172.21.15.93:0/4071153419 >> 172.21.15.93:0/4071153419 conn(0x7efff00fb8a0 msgr2=0x7efff00fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:55.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.963+0000 7efff55ff700 1 -- 172.21.15.93:0/4071153419 shutdown_connections 2023-11-16T16:46:55.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.963+0000 7efff55ff700 1 -- 172.21.15.93:0/4071153419 wait complete. 2023-11-16T16:46:55.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.964+0000 7efff55ff700 1 Processor -- start 2023-11-16T16:46:55.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.964+0000 7efff55ff700 1 -- start start 2023-11-16T16:46:55.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.964+0000 7efff55ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff0073550 0x7efff0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:55.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.964+0000 7efff55ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efff0074b30 0x7efff01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:55.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.964+0000 7efff55ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff0108180 0x7efff019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:55.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.964+0000 7efff55ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff019fee0 con 0x7efff0073550 2023-11-16T16:46:55.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.964+0000 7efff55ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff01a0020 con 0x7efff0108180 2023-11-16T16:46:55.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.964+0000 7efff55ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff01a01b0 con 0x7efff0074b30 2023-11-16T16:46:55.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effeeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff0073550 0x7efff0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:55.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effeeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff0073550 0x7efff0198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59800/0 (socket says 172.21.15.93:59800) 2023-11-16T16:46:55.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effeeffd700 1 -- 172.21.15.93:0/3344163685 learned_addr learned my addr 172.21.15.93:0/3344163685 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:55.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effef7fe700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff0108180 0x7efff019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:55.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effe7fff700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efff0074b30 0x7efff01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:55.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effeeffd700 1 -- 172.21.15.93:0/3344163685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efff0074b30 msgr2=0x7efff01988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effeeffd700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efff0074b30 0x7efff01988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effeeffd700 1 -- 172.21.15.93:0/3344163685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff0108180 msgr2=0x7efff019f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:55.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effeeffd700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff0108180 0x7efff019f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:55.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effeeffd700 1 -- 172.21.15.93:0/3344163685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8009520 con 0x7efff0073550 2023-11-16T16:46:55.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effef7fe700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff0108180 0x7efff019f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:55.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effeeffd700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff0073550 0x7efff0198360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7effe000e970 tx=0x7effe000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:55.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effecff9700 1 -- 172.21.15.93:0/3344163685 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0009790 con 0x7efff0073550 2023-11-16T16:46:55.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.965+0000 7effecff9700 1 -- 172.21.15.93:0/3344163685 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effe000c7e0 con 0x7efff0073550 2023-11-16T16:46:55.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.966+0000 7efff55ff700 1 -- 172.21.15.93:0/3344163685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe8009d00 con 0x7efff0073550 2023-11-16T16:46:55.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.966+0000 7effecff9700 1 -- 172.21.15.93:0/3344163685 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe00054e0 con 0x7efff0073550 2023-11-16T16:46:55.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.966+0000 7efff55ff700 1 -- 172.21.15.93:0/3344163685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff010dea0 con 0x7efff0073550 2023-11-16T16:46:55.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.967+0000 7efff55ff700 1 -- 172.21.15.93:0/3344163685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff0106830 con 0x7efff0073550 2023-11-16T16:46:55.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.969+0000 7effecff9700 1 -- 172.21.15.93:0/3344163685 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7effe000c950 con 0x7efff0073550 2023-11-16T16:46:55.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.970+0000 7effecff9700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7effdc0a11d0 0x7effdc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:55.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.970+0000 7effecff9700 1 -- 172.21.15.93:0/3344163685 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7effe0014070 con 0x7efff0073550 2023-11-16T16:46:55.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.970+0000 7effe7fff700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7effdc0a11d0 0x7effdc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:55.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.971+0000 7effe7fff700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7effdc0a11d0 0x7effdc0a3680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efff0074850 tx=0x7effd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:55.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:55.972+0000 7effecff9700 1 -- 172.21.15.93:0/3344163685 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7effe0096460 con 0x7efff0073550 2023-11-16T16:46:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:55 smithi159 ceph-mon[131528]: pgmap v292: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-11-16T16:46:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:55 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2469746820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:55 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2638872592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:46:56.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.171+0000 7efff55ff700 1 -- 172.21.15.93:0/3344163685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efff0067b00 con 0x7efff0073550 2023-11-16T16:46:56.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.172+0000 7effecff9700 1 -- 172.21.15.93:0/3344163685 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7effe0099a80 con 0x7efff0073550 2023-11-16T16:46:56.173 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-11-16T16:46:56.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.174+0000 7efff55ff700 1 -- 172.21.15.93:0/3344163685 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7effdc0a11d0 msgr2=0x7effdc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:56.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.174+0000 7efff55ff700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7effdc0a11d0 0x7effdc0a3680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efff0074850 tx=0x7effd8009260 comp rx=0 tx=0).stop 2023-11-16T16:46:56.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.174+0000 7efff55ff700 1 -- 172.21.15.93:0/3344163685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff0073550 msgr2=0x7efff0198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:56.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.174+0000 7efff55ff700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff0073550 0x7efff0198360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7effe000e970 tx=0x7effe000ec80 comp rx=0 tx=0).stop 2023-11-16T16:46:56.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.174+0000 7efff55ff700 1 -- 172.21.15.93:0/3344163685 shutdown_connections 2023-11-16T16:46:56.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.174+0000 7efff55ff700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7effdc0a11d0 0x7effdc0a3680 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.174+0000 7efff55ff700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff0073550 0x7efff0198360 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.174+0000 7efff55ff700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff0108180 0x7efff019f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.174+0000 7efff55ff700 1 --2- 172.21.15.93:0/3344163685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efff0074b30 0x7efff01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.174+0000 7efff55ff700 1 -- 172.21.15.93:0/3344163685 >> 172.21.15.93:0/3344163685 conn(0x7efff00fb8a0 msgr2=0x7efff010a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:56.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.174+0000 7efff55ff700 1 -- 172.21.15.93:0/3344163685 shutdown_connections 2023-11-16T16:46:56.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.175+0000 7efff55ff700 1 -- 172.21.15.93:0/3344163685 wait complete. 2023-11-16T16:46:56.195 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-16T16:46:56.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.393+0000 7f25e2fff700 1 -- 172.21.15.93:0/1988055951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25dc1014b0 msgr2=0x7f25dc101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:56.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.393+0000 7f25e2fff700 1 --2- 172.21.15.93:0/1988055951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25dc1014b0 0x7f25dc101900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f25c80098f0 tx=0x7f25c8009c00 comp rx=0 tx=0).stop 2023-11-16T16:46:56.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.393+0000 7f25e2fff700 1 -- 172.21.15.93:0/1988055951 shutdown_connections 2023-11-16T16:46:56.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.393+0000 7f25e2fff700 1 --2- 172.21.15.93:0/1988055951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25dc1002c0 0x7f25dc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.393+0000 7f25e2fff700 1 --2- 172.21.15.93:0/1988055951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25dc101e40 0x7f25dc108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.393+0000 7f25e2fff700 1 --2- 172.21.15.93:0/1988055951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25dc1014b0 0x7f25dc101900 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.393+0000 7f25e2fff700 1 -- 172.21.15.93:0/1988055951 >> 172.21.15.93:0/1988055951 conn(0x7f25dc0fb8a0 msgr2=0x7f25dc0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:56.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.393+0000 7f25e2fff700 1 -- 172.21.15.93:0/1988055951 shutdown_connections 2023-11-16T16:46:56.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.393+0000 7f25e2fff700 1 -- 172.21.15.93:0/1988055951 wait complete. 2023-11-16T16:46:56.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.394+0000 7f25e2fff700 1 Processor -- start 2023-11-16T16:46:56.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.394+0000 7f25e2fff700 1 -- start start 2023-11-16T16:46:56.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25e2fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25dc1002c0 0x7f25dc1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:56.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25e2fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25dc1014b0 0x7f25dc198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:56.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25e2fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25dc101e40 0x7f25dc19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:56.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25e2fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc19fd60 con 0x7f25dc1014b0 2023-11-16T16:46:56.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25e2fff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc19fea0 con 0x7f25dc1002c0 2023-11-16T16:46:56.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25e2fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc1a0030 con 0x7f25dc101e40 2023-11-16T16:46:56.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25dbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25dc1014b0 0x7f25dc198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:56.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25e0d9b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25dc1002c0 0x7f25dc1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:56.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25dbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25dc1014b0 0x7f25dc198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59822/0 (socket says 172.21.15.93:59822) 2023-11-16T16:46:56.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25dbfff700 1 -- 172.21.15.93:0/3672689798 learned_addr learned my addr 172.21.15.93:0/3672689798 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:56.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25e159c700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25dc101e40 0x7f25dc19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:56.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25dbfff700 1 -- 172.21.15.93:0/3672689798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25dc101e40 msgr2=0x7f25dc19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:56.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25dbfff700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25dc101e40 0x7f25dc19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25dbfff700 1 -- 172.21.15.93:0/3672689798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25dc1002c0 msgr2=0x7f25dc1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:56.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25dbfff700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25dc1002c0 0x7f25dc1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25dbfff700 1 -- 172.21.15.93:0/3672689798 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c8009520 con 0x7f25dc1014b0 2023-11-16T16:46:56.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.395+0000 7f25e0d9b700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25dc1002c0 0x7f25dc1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:56.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.396+0000 7f25dbfff700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25dc1014b0 0x7f25dc198720 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009fd0 tx=0x7f25c800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:56.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.396+0000 7f25d9ffb700 1 -- 172.21.15.93:0/3672689798 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c801c070 con 0x7f25dc1014b0 2023-11-16T16:46:56.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.396+0000 7f25e2fff700 1 -- 172.21.15.93:0/3672689798 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25dc10bc20 con 0x7f25dc1014b0 2023-11-16T16:46:56.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.396+0000 7f25d9ffb700 1 -- 172.21.15.93:0/3672689798 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25c800f7d0 con 0x7f25dc1014b0 2023-11-16T16:46:56.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.396+0000 7f25d9ffb700 1 -- 172.21.15.93:0/3672689798 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8004b10 con 0x7f25dc1014b0 2023-11-16T16:46:56.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.396+0000 7f25e2fff700 1 -- 172.21.15.93:0/3672689798 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25dc10ba90 con 0x7f25dc1014b0 2023-11-16T16:46:56.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.397+0000 7f25e2fff700 1 -- 172.21.15.93:0/3672689798 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25dc192980 con 0x7f25dc1014b0 2023-11-16T16:46:56.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.399+0000 7f25d9ffb700 1 -- 172.21.15.93:0/3672689798 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f25c8020070 con 0x7f25dc1014b0 2023-11-16T16:46:56.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.400+0000 7f25d9ffb700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f25cc0a1150 0x7f25cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:56.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.400+0000 7f25d9ffb700 1 -- 172.21.15.93:0/3672689798 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f25c80cd920 con 0x7f25dc1014b0 2023-11-16T16:46:56.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.401+0000 7f25e0d9b700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f25cc0a1150 0x7f25cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:56.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.401+0000 7f25e0d9b700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f25cc0a1150 0x7f25cc0a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f25dc101310 tx=0x7f25d0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:56.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.404+0000 7f25d9ffb700 1 -- 172.21.15.93:0/3672689798 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f25c805c730 con 0x7f25dc1014b0 2023-11-16T16:46:56.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.605+0000 7f25e2fff700 1 -- 172.21.15.93:0/3672689798 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f25dc061980 con 0x7f25dc1014b0 2023-11-16T16:46:56.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.606+0000 7f25d9ffb700 1 -- 172.21.15.93:0/3672689798 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f25c8096da0 con 0x7f25dc1014b0 2023-11-16T16:46:56.607 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:46:56.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.608+0000 7f25e2fff700 1 -- 172.21.15.93:0/3672689798 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f25cc0a1150 msgr2=0x7f25cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:56.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.608+0000 7f25e2fff700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f25cc0a1150 0x7f25cc0a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f25dc101310 tx=0x7f25d0009310 comp rx=0 tx=0).stop 2023-11-16T16:46:56.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.608+0000 7f25e2fff700 1 -- 172.21.15.93:0/3672689798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25dc1014b0 msgr2=0x7f25dc198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:56.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.608+0000 7f25e2fff700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25dc1014b0 0x7f25dc198720 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009fd0 tx=0x7f25c800b840 comp rx=0 tx=0).stop 2023-11-16T16:46:56.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.608+0000 7f25e2fff700 1 -- 172.21.15.93:0/3672689798 shutdown_connections 2023-11-16T16:46:56.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.608+0000 7f25e2fff700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f25cc0a1150 0x7f25cc0a3600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.608+0000 7f25e2fff700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25dc101e40 0x7f25dc19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.608+0000 7f25e2fff700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25dc1002c0 0x7f25dc1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.608+0000 7f25e2fff700 1 --2- 172.21.15.93:0/3672689798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25dc1014b0 0x7f25dc198720 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.608+0000 7f25e2fff700 1 -- 172.21.15.93:0/3672689798 >> 172.21.15.93:0/3672689798 conn(0x7f25dc0fb8a0 msgr2=0x7f25dc106d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:56.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.608+0000 7f25e2fff700 1 -- 172.21.15.93:0/3672689798 shutdown_connections 2023-11-16T16:46:56.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.608+0000 7f25e2fff700 1 -- 172.21.15.93:0/3672689798 wait complete. 2023-11-16T16:46:56.628 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-16T16:46:56.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.825+0000 7f6f89dff700 1 -- 172.21.15.93:0/4284924338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f84101db0 msgr2=0x7f6f84108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.825+0000 7f6f89dff700 1 --2- 172.21.15.93:0/4284924338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f84101db0 0x7f6f84108670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00b0a0 tx=0x7f6f7c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.825+0000 7f6f89dff700 1 -- 172.21.15.93:0/4284924338 shutdown_connections 2023-11-16T16:46:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.825+0000 7f6f89dff700 1 --2- 172.21.15.93:0/4284924338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f84101db0 0x7f6f84108670 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.825+0000 7f6f89dff700 1 --2- 172.21.15.93:0/4284924338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84100230 0x7f6f84100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.825+0000 7f6f89dff700 1 --2- 172.21.15.93:0/4284924338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f84101420 0x7f6f84101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.825+0000 7f6f89dff700 1 -- 172.21.15.93:0/4284924338 >> 172.21.15.93:0/4284924338 conn(0x7f6f840fb810 msgr2=0x7f6f840fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.825+0000 7f6f89dff700 1 -- 172.21.15.93:0/4284924338 shutdown_connections 2023-11-16T16:46:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.825+0000 7f6f89dff700 1 -- 172.21.15.93:0/4284924338 wait complete. 2023-11-16T16:46:56.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.826+0000 7f6f89dff700 1 Processor -- start 2023-11-16T16:46:56.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.826+0000 7f6f89dff700 1 -- start start 2023-11-16T16:46:56.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.826+0000 7f6f89dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84100230 0x7f6f841982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:56.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.826+0000 7f6f89dff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f84101420 0x7f6f841987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:56.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.826+0000 7f6f89dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f84101db0 0x7f6f8419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.826+0000 7f6f89dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8419fe30 con 0x7f6f84101db0 2023-11-16T16:46:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.826+0000 7f6f89dff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8419ff70 con 0x7f6f84101420 2023-11-16T16:46:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.826+0000 7f6f89dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841a0100 con 0x7f6f84100230 2023-11-16T16:46:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f83fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f84101db0 0x7f6f8419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f83fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f84101db0 0x7f6f8419f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59842/0 (socket says 172.21.15.93:59842) 2023-11-16T16:46:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f83fff700 1 -- 172.21.15.93:0/2090378131 learned_addr learned my addr 172.21.15.93:0/2090378131 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f82ffd700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f84101420 0x7f6f841987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f837fe700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84100230 0x7f6f841982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f83fff700 1 -- 172.21.15.93:0/2090378131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84100230 msgr2=0x7f6f841982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f83fff700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84100230 0x7f6f841982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f83fff700 1 -- 172.21.15.93:0/2090378131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f84101420 msgr2=0x7f6f841987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f83fff700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f84101420 0x7f6f841987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f83fff700 1 -- 172.21.15.93:0/2090378131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c009d00 con 0x7f6f84101db0 2023-11-16T16:46:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f82ffd700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f84101420 0x7f6f841987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f83fff700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f84101db0 0x7f6f8419f860 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00d010 tx=0x7f6f7c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f80ff9700 1 -- 172.21.15.93:0/2090378131 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00e040 con 0x7f6f84101db0 2023-11-16T16:46:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f80ff9700 1 -- 172.21.15.93:0/2090378131 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f7c003e10 con 0x7f6f84101db0 2023-11-16T16:46:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f89dff700 1 -- 172.21.15.93:0/2090378131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8410bc00 con 0x7f6f84101db0 2023-11-16T16:46:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f80ff9700 1 -- 172.21.15.93:0/2090378131 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0053c0 con 0x7f6f84101db0 2023-11-16T16:46:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.827+0000 7f6f89dff700 1 -- 172.21.15.93:0/2090378131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8410ba70 con 0x7f6f84101db0 2023-11-16T16:46:56.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.828+0000 7f6f89dff700 1 -- 172.21.15.93:0/2090378131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f841a06a0 con 0x7f6f84101db0 2023-11-16T16:46:56.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.830+0000 7f6f80ff9700 1 -- 172.21.15.93:0/2090378131 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6f7c01da40 con 0x7f6f84101db0 2023-11-16T16:46:56.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.832+0000 7f6f80ff9700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6f6c0a1150 0x7f6f6c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:56.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.833+0000 7f6f80ff9700 1 -- 172.21.15.93:0/2090378131 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6f7c0cdf50 con 0x7f6f84101db0 2023-11-16T16:46:56.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.833+0000 7f6f837fe700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6f6c0a1150 0x7f6f6c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:56.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.833+0000 7f6f837fe700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6f6c0a1150 0x7f6f6c0a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6f84101280 tx=0x7f6f74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:56.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:56.837+0000 7f6f80ff9700 1 -- 172.21.15.93:0/2090378131 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6f7c0973d0 con 0x7f6f84101db0 2023-11-16T16:46:56.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:56 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/371716311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:46:56.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:56 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3344163685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:46:56.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:56 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/371716311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:46:56.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:56 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3344163685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:46:57.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.049+0000 7f6f89dff700 1 -- 172.21.15.93:0/2090378131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6f841a0c50 con 0x7f6f84101db0 2023-11-16T16:46:57.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.050+0000 7f6f80ff9700 1 -- 172.21.15.93:0/2090378131 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+588 (secure 0 0 0) 0x7f6f7c09a9f0 con 0x7f6f84101db0 2023-11-16T16:46:57.051 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/1465890024 2023-11-17T16:39:50.432397+0000 2023-11-16T16:46:57.051 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/3409666551 2023-11-17T16:39:50.432397+0000 2023-11-16T16:46:57.051 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/2198359920 2023-11-17T16:39:50.432397+0000 2023-11-16T16:46:57.051 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/772471105 2023-11-17T16:39:24.569132+0000 2023-11-16T16:46:57.052 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/772471105 2023-11-17T16:39:24.569132+0000 2023-11-16T16:46:57.052 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/2198359920 2023-11-17T16:39:50.432397+0000 2023-11-16T16:46:57.052 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/438537209 2023-11-17T16:39:24.569132+0000 2023-11-16T16:46:57.052 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2518663109 2023-11-17T16:39:24.569132+0000 2023-11-16T16:46:57.052 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/696516505 2023-11-17T16:39:24.569132+0000 2023-11-16T16:46:57.052 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/3700969994 2023-11-17T16:39:50.432397+0000 2023-11-16T16:46:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.052+0000 7f6f89dff700 1 -- 172.21.15.93:0/2090378131 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6f6c0a1150 msgr2=0x7f6f6c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.052+0000 7f6f89dff700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6f6c0a1150 0x7f6f6c0a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6f84101280 tx=0x7f6f74009260 comp rx=0 tx=0).stop 2023-11-16T16:46:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.052+0000 7f6f89dff700 1 -- 172.21.15.93:0/2090378131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f84101db0 msgr2=0x7f6f8419f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.052+0000 7f6f89dff700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f84101db0 0x7f6f8419f860 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00d010 tx=0x7f6f7c00bd30 comp rx=0 tx=0).stop 2023-11-16T16:46:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.052+0000 7f6f89dff700 1 -- 172.21.15.93:0/2090378131 shutdown_connections 2023-11-16T16:46:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.052+0000 7f6f89dff700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6f6c0a1150 0x7f6f6c0a3600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.052+0000 7f6f89dff700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f84101db0 0x7f6f8419f860 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.052+0000 7f6f89dff700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84100230 0x7f6f841982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.052+0000 7f6f89dff700 1 --2- 172.21.15.93:0/2090378131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f84101420 0x7f6f841987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.052+0000 7f6f89dff700 1 -- 172.21.15.93:0/2090378131 >> 172.21.15.93:0/2090378131 conn(0x7f6f840fb810 msgr2=0x7f6f84106c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.052+0000 7f6f89dff700 1 -- 172.21.15.93:0/2090378131 shutdown_connections 2023-11-16T16:46:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.052+0000 7f6f89dff700 1 -- 172.21.15.93:0/2090378131 wait complete. 2023-11-16T16:46:57.055 INFO:teuthology.orchestra.run.smithi093.stderr:listed 10 entries 2023-11-16T16:46:57.072 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:46:57.072 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-16T16:46:57.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:56 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/371716311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:46:57.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:56 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3344163685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:46:57.116 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:46:57.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.354+0000 7f1a9b59e700 1 -- 172.21.15.93:0/3024670715 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a9c074a30 msgr2=0x7f1a9c072f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:57.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.354+0000 7f1a9b59e700 1 --2- 172.21.15.93:0/3024670715 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a9c074a30 0x7f1a9c072f20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009990 tx=0x7f1a8c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.354+0000 7f1a9b59e700 1 -- 172.21.15.93:0/3024670715 shutdown_connections 2023-11-16T16:46:57.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.354+0000 7f1a9b59e700 1 --2- 172.21.15.93:0/3024670715 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a9c074a30 0x7f1a9c072f20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.354+0000 7f1a9b59e700 1 --2- 172.21.15.93:0/3024670715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a9c107f90 0x7f1a9c10a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.354+0000 7f1a9b59e700 1 --2- 172.21.15.93:0/3024670715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a9c073460 0x7f1a9c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.354+0000 7f1a9b59e700 1 -- 172.21.15.93:0/3024670715 >> 172.21.15.93:0/3024670715 conn(0x7f1a9c0fb6d0 msgr2=0x7f1a9c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:57.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.354+0000 7f1a9b59e700 1 -- 172.21.15.93:0/3024670715 shutdown_connections 2023-11-16T16:46:57.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.354+0000 7f1a9b59e700 1 -- 172.21.15.93:0/3024670715 wait complete. 2023-11-16T16:46:57.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.355+0000 7f1a9b59e700 1 Processor -- start 2023-11-16T16:46:57.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.355+0000 7f1a9b59e700 1 -- start start 2023-11-16T16:46:57.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.355+0000 7f1a9b59e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a9c073460 0x7f1a9c197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:57.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.355+0000 7f1a9b59e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a9c074a30 0x7f1a9c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:57.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.355+0000 7f1a9b59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a9c107f90 0x7f1a9c19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:57.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.355+0000 7f1a9b59e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c19fb30 con 0x7f1a9c107f90 2023-11-16T16:46:57.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.355+0000 7f1a9b59e700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c19fc70 con 0x7f1a9c074a30 2023-11-16T16:46:57.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.355+0000 7f1a9b59e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c19fe00 con 0x7f1a9c073460 2023-11-16T16:46:57.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.356+0000 7f1a9ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a9c107f90 0x7f1a9c19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:57.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.356+0000 7f1a9ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a9c107f90 0x7f1a9c19f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38530/0 (socket says 172.21.15.93:38530) 2023-11-16T16:46:57.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.356+0000 7f1a9ad9d700 1 -- 172.21.15.93:0/1305293365 learned_addr learned my addr 172.21.15.93:0/1305293365 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:57.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.356+0000 7f1a93fff700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a9c074a30 0x7f1a9c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:57.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.356+0000 7f1a9a59c700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a9c073460 0x7f1a9c197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:57.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.356+0000 7f1a9ad9d700 1 -- 172.21.15.93:0/1305293365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a9c073460 msgr2=0x7f1a9c197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:57.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.356+0000 7f1a9ad9d700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a9c073460 0x7f1a9c197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.356+0000 7f1a9ad9d700 1 -- 172.21.15.93:0/1305293365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a9c074a30 msgr2=0x7f1a9c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:57.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.356+0000 7f1a9ad9d700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a9c074a30 0x7f1a9c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.356+0000 7f1a9ad9d700 1 -- 172.21.15.93:0/1305293365 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c0095f0 con 0x7f1a9c107f90 2023-11-16T16:46:57.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.356+0000 7f1a9ad9d700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a9c107f90 0x7f1a9c19f560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400ef30 tx=0x7f1a9400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:57.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.356+0000 7f1a937fe700 1 -- 172.21.15.93:0/1305293365 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94018070 con 0x7f1a9c107f90 2023-11-16T16:46:57.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.357+0000 7f1a937fe700 1 -- 172.21.15.93:0/1305293365 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a9400f040 con 0x7f1a9c107f90 2023-11-16T16:46:57.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.357+0000 7f1a9b59e700 1 -- 172.21.15.93:0/1305293365 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c10dc40 con 0x7f1a9c107f90 2023-11-16T16:46:57.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.357+0000 7f1a937fe700 1 -- 172.21.15.93:0/1305293365 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400cd00 con 0x7f1a9c107f90 2023-11-16T16:46:57.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.357+0000 7f1a9b59e700 1 -- 172.21.15.93:0/1305293365 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9c10dab0 con 0x7f1a9c107f90 2023-11-16T16:46:57.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.358+0000 7f1a9b59e700 1 -- 172.21.15.93:0/1305293365 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9c106720 con 0x7f1a9c107f90 2023-11-16T16:46:57.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.359+0000 7f1a937fe700 1 -- 172.21.15.93:0/1305293365 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1a94003a00 con 0x7f1a9c107f90 2023-11-16T16:46:57.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.362+0000 7f1a937fe700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1a880a1150 0x7f1a880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:57.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.362+0000 7f1a937fe700 1 -- 172.21.15.93:0/1305293365 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1a940cc790 con 0x7f1a9c107f90 2023-11-16T16:46:57.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.362+0000 7f1a9a59c700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1a880a1150 0x7f1a880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:57.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.362+0000 7f1a9a59c700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1a880a1150 0x7f1a880a3600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00b2e0 tx=0x7f1a8c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:57.367 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.366+0000 7f1a937fe700 1 -- 172.21.15.93:0/1305293365 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1a94095c10 con 0x7f1a9c107f90 2023-11-16T16:46:57.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.566+0000 7f1a9b59e700 1 -- 172.21.15.93:0/1305293365 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a9c02cfe0 con 0x7f1a9c107f90 2023-11-16T16:46:57.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.568+0000 7f1a937fe700 1 -- 172.21.15.93:0/1305293365 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24468 (secure 0 0 0) 0x7f1a94099230 con 0x7f1a9c107f90 2023-11-16T16:46:57.570 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:46:57.570 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":90,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:46:52.315313+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:46:57.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.570+0000 7f1a9b59e700 1 -- 172.21.15.93:0/1305293365 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1a880a1150 msgr2=0x7f1a880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:57.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.570+0000 7f1a9b59e700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1a880a1150 0x7f1a880a3600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00b2e0 tx=0x7f1a8c005fb0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.570+0000 7f1a9b59e700 1 -- 172.21.15.93:0/1305293365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a9c107f90 msgr2=0x7f1a9c19f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:57.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.570+0000 7f1a9b59e700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a9c107f90 0x7f1a9c19f560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400ef30 tx=0x7f1a9400c3a0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.570+0000 7f1a9b59e700 1 -- 172.21.15.93:0/1305293365 shutdown_connections 2023-11-16T16:46:57.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.570+0000 7f1a9b59e700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1a880a1150 0x7f1a880a3600 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.570+0000 7f1a9b59e700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a9c107f90 0x7f1a9c19f560 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.570+0000 7f1a9b59e700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a9c073460 0x7f1a9c197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.570+0000 7f1a9b59e700 1 --2- 172.21.15.93:0/1305293365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a9c074a30 0x7f1a9c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.570+0000 7f1a9b59e700 1 -- 172.21.15.93:0/1305293365 >> 172.21.15.93:0/1305293365 conn(0x7f1a9c0fb6d0 msgr2=0x7f1a9c0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:57.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.571+0000 7f1a9b59e700 1 -- 172.21.15.93:0/1305293365 shutdown_connections 2023-11-16T16:46:57.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.571+0000 7f1a9b59e700 1 -- 172.21.15.93:0/1305293365 wait complete. 2023-11-16T16:46:57.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:42:13.029135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:46:35.301321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:46:36.552545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-16T16:46:39.628380+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-16T16:46:57.592 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:46:57.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.790+0000 7f75a43ff700 1 -- 172.21.15.93:0/3786156331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f759c1014c0 msgr2=0x7f759c101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:57.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.790+0000 7f75a43ff700 1 --2- 172.21.15.93:0/3786156331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f759c1014c0 0x7f759c101910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f758c009990 tx=0x7f758c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.790+0000 7f75a43ff700 1 -- 172.21.15.93:0/3786156331 shutdown_connections 2023-11-16T16:46:57.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.790+0000 7f75a43ff700 1 --2- 172.21.15.93:0/3786156331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f759c101e50 0x7f759c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.790+0000 7f75a43ff700 1 --2- 172.21.15.93:0/3786156331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f759c1002d0 0x7f759c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.790+0000 7f75a43ff700 1 --2- 172.21.15.93:0/3786156331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f759c1014c0 0x7f759c101910 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.790+0000 7f75a43ff700 1 -- 172.21.15.93:0/3786156331 >> 172.21.15.93:0/3786156331 conn(0x7f759c0fb8b0 msgr2=0x7f759c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:57.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.790+0000 7f75a43ff700 1 -- 172.21.15.93:0/3786156331 shutdown_connections 2023-11-16T16:46:57.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.790+0000 7f75a43ff700 1 -- 172.21.15.93:0/3786156331 wait complete. 2023-11-16T16:46:57.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.791+0000 7f75a43ff700 1 Processor -- start 2023-11-16T16:46:57.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.791+0000 7f75a43ff700 1 -- start start 2023-11-16T16:46:57.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.791+0000 7f75a43ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f759c1002d0 0x7f759c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:57.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.791+0000 7f75a43ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f759c1014c0 0x7f759c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:57.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.791+0000 7f75a43ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f759c101e50 0x7f759c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:57.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.791+0000 7f75a43ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c073bc0 con 0x7f759c101e50 2023-11-16T16:46:57.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.791+0000 7f75a43ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c073d30 con 0x7f759c1014c0 2023-11-16T16:46:57.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.791+0000 7f75a43ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c1a00a0 con 0x7f759c1002d0 2023-11-16T16:46:57.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.791+0000 7f75a299c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f759c101e50 0x7f759c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:57.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.791+0000 7f75a219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f759c1002d0 0x7f759c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:57.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.791+0000 7f75a299c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f759c101e50 0x7f759c073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38552/0 (socket says 172.21.15.93:38552) 2023-11-16T16:46:57.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.791+0000 7f75a299c700 1 -- 172.21.15.93:0/36560789 learned_addr learned my addr 172.21.15.93:0/36560789 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:57.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.792+0000 7f75a199a700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f759c1014c0 0x7f759c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:57.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.792+0000 7f75a299c700 1 -- 172.21.15.93:0/36560789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f759c1002d0 msgr2=0x7f759c0748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:57.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.792+0000 7f75a299c700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f759c1002d0 0x7f759c0748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.792+0000 7f75a299c700 1 -- 172.21.15.93:0/36560789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f759c1014c0 msgr2=0x7f759c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:57.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.792+0000 7f75a299c700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f759c1014c0 0x7f759c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:57.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.792+0000 7f75a299c700 1 -- 172.21.15.93:0/36560789 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0095f0 con 0x7f759c101e50 2023-11-16T16:46:57.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.792+0000 7f75a299c700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f759c101e50 0x7f759c073420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f759400ef30 tx=0x7f759400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:57.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.792+0000 7f75937fe700 1 -- 172.21.15.93:0/36560789 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594018070 con 0x7f759c101e50 2023-11-16T16:46:57.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.792+0000 7f75937fe700 1 -- 172.21.15.93:0/36560789 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f759400f040 con 0x7f759c101e50 2023-11-16T16:46:57.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.792+0000 7f75a43ff700 1 -- 172.21.15.93:0/36560789 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c10bc40 con 0x7f759c101e50 2023-11-16T16:46:57.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.792+0000 7f75937fe700 1 -- 172.21.15.93:0/36560789 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759400cd00 con 0x7f759c101e50 2023-11-16T16:46:57.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.792+0000 7f75a43ff700 1 -- 172.21.15.93:0/36560789 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c10bab0 con 0x7f759c101e50 2023-11-16T16:46:57.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.794+0000 7f75a43ff700 1 -- 172.21.15.93:0/36560789 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759c1a0730 con 0x7f759c101e50 2023-11-16T16:46:57.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.796+0000 7f75937fe700 1 -- 172.21.15.93:0/36560789 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7594003a00 con 0x7f759c101e50 2023-11-16T16:46:57.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.797+0000 7f75937fe700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f75880a1150 0x7f75880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:57.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.797+0000 7f75937fe700 1 -- 172.21.15.93:0/36560789 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f75940cc6e0 con 0x7f759c101e50 2023-11-16T16:46:57.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.797+0000 7f75a219b700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f75880a1150 0x7f75880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:57.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.798+0000 7f75a219b700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f75880a1150 0x7f75880a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f759c101320 tx=0x7f7598007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:57.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.800+0000 7f75937fe700 1 -- 172.21.15.93:0/36560789 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7594095b60 con 0x7f759c101e50 2023-11-16T16:46:57.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:57 smithi093 ceph-mon[97863]: pgmap v293: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:46:57.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:57 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3672689798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:46:57.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:57 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2090378131' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-16T16:46:57.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:57 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1305293365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:57 smithi159 ceph-mon[131528]: pgmap v293: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:46:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:57 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3672689798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:46:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:57 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2090378131' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-16T16:46:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:57 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1305293365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:57.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.996+0000 7f75a43ff700 1 -- 172.21.15.93:0/36560789 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f759c107200 con 0x7f759c101e50 2023-11-16T16:46:57.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.997+0000 7f75937fe700 1 -- 172.21.15.93:0/36560789 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7594099180 con 0x7f759c101e50 2023-11-16T16:46:57.998 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-11-16T16:46:58.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.999+0000 7f75a43ff700 1 -- 172.21.15.93:0/36560789 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f75880a1150 msgr2=0x7f75880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:58.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.999+0000 7f75a43ff700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f75880a1150 0x7f75880a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f759c101320 tx=0x7f7598007400 comp rx=0 tx=0).stop 2023-11-16T16:46:58.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.999+0000 7f75a43ff700 1 -- 172.21.15.93:0/36560789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f759c101e50 msgr2=0x7f759c073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:58.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.999+0000 7f75a43ff700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f759c101e50 0x7f759c073420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f759400ef30 tx=0x7f759400c3a0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.999+0000 7f75a43ff700 1 -- 172.21.15.93:0/36560789 shutdown_connections 2023-11-16T16:46:58.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.999+0000 7f75a43ff700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f75880a1150 0x7f75880a3600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.999+0000 7f75a43ff700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f759c101e50 0x7f759c073420 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.999+0000 7f75a43ff700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f759c1002d0 0x7f759c0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.999+0000 7f75a43ff700 1 --2- 172.21.15.93:0/36560789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f759c1014c0 0x7f759c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.999+0000 7f75a43ff700 1 -- 172.21.15.93:0/36560789 >> 172.21.15.93:0/36560789 conn(0x7f759c0fb8b0 msgr2=0x7f759c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:58.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:57.999+0000 7f75a43ff700 1 -- 172.21.15.93:0/36560789 shutdown_connections 2023-11-16T16:46:58.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.000+0000 7f75a43ff700 1 -- 172.21.15.93:0/36560789 wait complete. 2023-11-16T16:46:58.019 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:46:58.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:57 smithi125 ceph-mon[131076]: pgmap v293: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:46:58.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:57 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3672689798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:46:58.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:57 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2090378131' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-16T16:46:58.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:57 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1305293365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:46:58.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.217+0000 7f4ac3dff700 1 -- 172.21.15.93:0/488262454 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4abc0734d0 msgr2=0x7f4abc073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:58.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.217+0000 7f4ac3dff700 1 --2- 172.21.15.93:0/488262454 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4abc0734d0 0x7f4abc073940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009990 tx=0x7f4aac009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.217+0000 7f4ac3dff700 1 -- 172.21.15.93:0/488262454 shutdown_connections 2023-11-16T16:46:58.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.217+0000 7f4ac3dff700 1 --2- 172.21.15.93:0/488262454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4abc108180 0x7f4abc10a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.217+0000 7f4ac3dff700 1 --2- 172.21.15.93:0/488262454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4abc074ab0 0x7f4abc072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.217+0000 7f4ac3dff700 1 --2- 172.21.15.93:0/488262454 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4abc0734d0 0x7f4abc073940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.217+0000 7f4ac3dff700 1 -- 172.21.15.93:0/488262454 >> 172.21.15.93:0/488262454 conn(0x7f4abc0fb8b0 msgr2=0x7f4abc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:58.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.217+0000 7f4ac3dff700 1 -- 172.21.15.93:0/488262454 shutdown_connections 2023-11-16T16:46:58.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.217+0000 7f4ac3dff700 1 -- 172.21.15.93:0/488262454 wait complete. 2023-11-16T16:46:58.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac3dff700 1 Processor -- start 2023-11-16T16:46:58.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac3dff700 1 -- start start 2023-11-16T16:46:58.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac3dff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4abc0734d0 0x7f4abc198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:58.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac3dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4abc074ab0 0x7f4abc198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:58.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac3dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4abc108180 0x7f4abc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:58.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac3dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc19fda0 con 0x7f4abc108180 2023-11-16T16:46:58.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac3dff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc19fee0 con 0x7f4abc0734d0 2023-11-16T16:46:58.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac3dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc1a0070 con 0x7f4abc074ab0 2023-11-16T16:46:58.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac239c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4abc108180 0x7f4abc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:58.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac139a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4abc074ab0 0x7f4abc198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:58.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac239c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4abc108180 0x7f4abc19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38574/0 (socket says 172.21.15.93:38574) 2023-11-16T16:46:58.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac239c700 1 -- 172.21.15.93:0/102087945 learned_addr learned my addr 172.21.15.93:0/102087945 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:58.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.218+0000 7f4ac139a700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4abc074ab0 0x7f4abc198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:38916/0 (socket says 172.21.15.93:38916) 2023-11-16T16:46:58.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.219+0000 7f4ac1b9b700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4abc0734d0 0x7f4abc198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:58.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.219+0000 7f4ac239c700 1 -- 172.21.15.93:0/102087945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4abc074ab0 msgr2=0x7f4abc198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:58.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.219+0000 7f4ac239c700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4abc074ab0 0x7f4abc198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.219+0000 7f4ac239c700 1 -- 172.21.15.93:0/102087945 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4abc0734d0 msgr2=0x7f4abc198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:58.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.219+0000 7f4ac239c700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4abc0734d0 0x7f4abc198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.219+0000 7f4ac239c700 1 -- 172.21.15.93:0/102087945 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac0095f0 con 0x7f4abc108180 2023-11-16T16:46:58.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.219+0000 7f4ac239c700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4abc108180 0x7f4abc19f7d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400ef30 tx=0x7f4ab400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:58.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.219+0000 7f4ab2ffd700 1 -- 172.21.15.93:0/102087945 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4018070 con 0x7f4abc108180 2023-11-16T16:46:58.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.219+0000 7f4ac3dff700 1 -- 172.21.15.93:0/102087945 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4abc10de20 con 0x7f4abc108180 2023-11-16T16:46:58.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.219+0000 7f4ab2ffd700 1 -- 172.21.15.93:0/102087945 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ab400f040 con 0x7f4abc108180 2023-11-16T16:46:58.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.219+0000 7f4ac3dff700 1 -- 172.21.15.93:0/102087945 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4abc10dc90 con 0x7f4abc108180 2023-11-16T16:46:58.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.219+0000 7f4ab2ffd700 1 -- 172.21.15.93:0/102087945 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab400cca0 con 0x7f4abc108180 2023-11-16T16:46:58.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.220+0000 7f4ac3dff700 1 -- 172.21.15.93:0/102087945 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4abc1068c0 con 0x7f4abc108180 2023-11-16T16:46:58.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.222+0000 7f4ab2ffd700 1 -- 172.21.15.93:0/102087945 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4ab4003a00 con 0x7f4abc108180 2023-11-16T16:46:58.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.224+0000 7f4ab2ffd700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4aa80a1150 0x7f4aa80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:58.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.224+0000 7f4ab2ffd700 1 -- 172.21.15.93:0/102087945 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4ab40cc720 con 0x7f4abc108180 2023-11-16T16:46:58.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.224+0000 7f4ac1b9b700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4aa80a1150 0x7f4aa80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:58.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.224+0000 7f4ac1b9b700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4aa80a1150 0x7f4aa80a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009c60 tx=0x7f4ab8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:58.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.227+0000 7f4ab2ffd700 1 -- 172.21.15.93:0/102087945 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4ab4095ba0 con 0x7f4abc108180 2023-11-16T16:46:58.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.427+0000 7f4ac3dff700 1 -- 172.21.15.93:0/102087945 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4abc061700 con 0x7f4abc108180 2023-11-16T16:46:58.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.427+0000 7f4ab2ffd700 1 -- 172.21.15.93:0/102087945 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ab40991c0 con 0x7f4abc108180 2023-11-16T16:46:58.429 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:46:58.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.430+0000 7f4ac3dff700 1 -- 172.21.15.93:0/102087945 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4aa80a1150 msgr2=0x7f4aa80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:58.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.430+0000 7f4ac3dff700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4aa80a1150 0x7f4aa80a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009c60 tx=0x7f4ab8009260 comp rx=0 tx=0).stop 2023-11-16T16:46:58.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.430+0000 7f4ac3dff700 1 -- 172.21.15.93:0/102087945 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4abc108180 msgr2=0x7f4abc19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:58.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.430+0000 7f4ac3dff700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4abc108180 0x7f4abc19f7d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400ef30 tx=0x7f4ab400c3a0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.430+0000 7f4ac3dff700 1 -- 172.21.15.93:0/102087945 shutdown_connections 2023-11-16T16:46:58.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.430+0000 7f4ac3dff700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4aa80a1150 0x7f4aa80a3600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.430+0000 7f4ac3dff700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4abc108180 0x7f4abc19f7d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.430+0000 7f4ac3dff700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4abc0734d0 0x7f4abc198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.430+0000 7f4ac3dff700 1 --2- 172.21.15.93:0/102087945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4abc074ab0 0x7f4abc198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.430+0000 7f4ac3dff700 1 -- 172.21.15.93:0/102087945 >> 172.21.15.93:0/102087945 conn(0x7f4abc0fb8b0 msgr2=0x7f4abc0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:58.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.430+0000 7f4ac3dff700 1 -- 172.21.15.93:0/102087945 shutdown_connections 2023-11-16T16:46:58.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.430+0000 7f4ac3dff700 1 -- 172.21.15.93:0/102087945 wait complete. 2023-11-16T16:46:58.450 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:46:58.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.654+0000 7f5529dff700 1 -- 172.21.15.93:0/4127574669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55241002c0 msgr2=0x7f55241006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:58.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.654+0000 7f5529dff700 1 --2- 172.21.15.93:0/4127574669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55241002c0 0x7f55241006d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5514009990 tx=0x7f5514009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.654+0000 7f5529dff700 1 -- 172.21.15.93:0/4127574669 shutdown_connections 2023-11-16T16:46:58.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.654+0000 7f5529dff700 1 --2- 172.21.15.93:0/4127574669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55241002c0 0x7f55241006d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.654+0000 7f5529dff700 1 --2- 172.21.15.93:0/4127574669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5524101e40 0x7f5524108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.654+0000 7f5529dff700 1 --2- 172.21.15.93:0/4127574669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55241014b0 0x7f5524101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.654+0000 7f5529dff700 1 -- 172.21.15.93:0/4127574669 >> 172.21.15.93:0/4127574669 conn(0x7f55240fb8a0 msgr2=0x7f55240fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:58.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.654+0000 7f5529dff700 1 -- 172.21.15.93:0/4127574669 shutdown_connections 2023-11-16T16:46:58.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.654+0000 7f5529dff700 1 -- 172.21.15.93:0/4127574669 wait complete. 2023-11-16T16:46:58.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.655+0000 7f5529dff700 1 Processor -- start 2023-11-16T16:46:58.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.655+0000 7f5529dff700 1 -- start start 2023-11-16T16:46:58.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f5529dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55241002c0 0x7f5524198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:58.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f5529dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55241014b0 0x7f55241987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:58.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f5529dff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5524101e40 0x7f552419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:58.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f5529dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552419fdf0 con 0x7f55241014b0 2023-11-16T16:46:58.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f5529dff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552419ff30 con 0x7f5524101e40 2023-11-16T16:46:58.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f5529dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55241a00c0 con 0x7f55241002c0 2023-11-16T16:46:58.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f55237fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55241002c0 0x7f5524198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:58.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f5522ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55241014b0 0x7f55241987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:58.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f55237fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55241002c0 0x7f5524198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:38930/0 (socket says 172.21.15.93:38930) 2023-11-16T16:46:58.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f55237fe700 1 -- 172.21.15.93:0/1211433058 learned_addr learned my addr 172.21.15.93:0/1211433058 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:58.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f5523fff700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5524101e40 0x7f552419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:58.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f55237fe700 1 -- 172.21.15.93:0/1211433058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5524101e40 msgr2=0x7f552419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:58.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f55237fe700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5524101e40 0x7f552419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f55237fe700 1 -- 172.21.15.93:0/1211433058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55241014b0 msgr2=0x7f55241987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:58.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f55237fe700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55241014b0 0x7f55241987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f55237fe700 1 -- 172.21.15.93:0/1211433058 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140095f0 con 0x7f55241002c0 2023-11-16T16:46:58.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.656+0000 7f5522ffd700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55241014b0 0x7f55241987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:58.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.657+0000 7f55237fe700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55241002c0 0x7f5524198270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5514004730 tx=0x7f5514004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:58.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.657+0000 7f5520ff9700 1 -- 172.21.15.93:0/1211433058 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551401d070 con 0x7f55241002c0 2023-11-16T16:46:58.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.657+0000 7f5520ff9700 1 -- 172.21.15.93:0/1211433058 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5514021410 con 0x7f55241002c0 2023-11-16T16:46:58.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.657+0000 7f5529dff700 1 -- 172.21.15.93:0/1211433058 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552410bc90 con 0x7f55241002c0 2023-11-16T16:46:58.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.657+0000 7f5520ff9700 1 -- 172.21.15.93:0/1211433058 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514017650 con 0x7f55241002c0 2023-11-16T16:46:58.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.657+0000 7f5529dff700 1 -- 172.21.15.93:0/1211433058 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552410bb00 con 0x7f55241002c0 2023-11-16T16:46:58.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.658+0000 7f5529dff700 1 -- 172.21.15.93:0/1211433058 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55241929f0 con 0x7f55241002c0 2023-11-16T16:46:58.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.660+0000 7f5520ff9700 1 -- 172.21.15.93:0/1211433058 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5514021580 con 0x7f55241002c0 2023-11-16T16:46:58.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.661+0000 7f5520ff9700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f550c0a1150 0x7f550c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:58.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.661+0000 7f5520ff9700 1 -- 172.21.15.93:0/1211433058 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f55140ce1e0 con 0x7f55241002c0 2023-11-16T16:46:58.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.661+0000 7f5522ffd700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f550c0a1150 0x7f550c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:58.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.662+0000 7f5522ffd700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f550c0a1150 0x7f550c0a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5518011490 tx=0x7f5518009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:58.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.663+0000 7f5520ff9700 1 -- 172.21.15.93:0/1211433058 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5514097660 con 0x7f55241002c0 2023-11-16T16:46:58.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.863+0000 7f5529dff700 1 -- 172.21.15.93:0/1211433058 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5524067b00 con 0x7f55241002c0 2023-11-16T16:46:58.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.864+0000 7f5520ff9700 1 -- 172.21.15.93:0/1211433058 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f551409ac80 con 0x7f55241002c0 2023-11-16T16:46:58.866 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-11-16T16:46:58.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.866+0000 7f5529dff700 1 -- 172.21.15.93:0/1211433058 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f550c0a1150 msgr2=0x7f550c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:58.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.866+0000 7f5529dff700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f550c0a1150 0x7f550c0a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5518011490 tx=0x7f5518009260 comp rx=0 tx=0).stop 2023-11-16T16:46:58.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.866+0000 7f5529dff700 1 -- 172.21.15.93:0/1211433058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55241002c0 msgr2=0x7f5524198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:58.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.866+0000 7f5529dff700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55241002c0 0x7f5524198270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5514004730 tx=0x7f5514004760 comp rx=0 tx=0).stop 2023-11-16T16:46:58.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.866+0000 7f5529dff700 1 -- 172.21.15.93:0/1211433058 shutdown_connections 2023-11-16T16:46:58.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.866+0000 7f5529dff700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f550c0a1150 0x7f550c0a3600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.866+0000 7f5529dff700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5524101e40 0x7f552419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.866+0000 7f5529dff700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55241002c0 0x7f5524198270 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.866+0000 7f5529dff700 1 --2- 172.21.15.93:0/1211433058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55241014b0 0x7f55241987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:58.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.866+0000 7f5529dff700 1 -- 172.21.15.93:0/1211433058 >> 172.21.15.93:0/1211433058 conn(0x7f55240fb8a0 msgr2=0x7f5524106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:58.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.866+0000 7f5529dff700 1 -- 172.21.15.93:0/1211433058 shutdown_connections 2023-11-16T16:46:58.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:58.866+0000 7f5529dff700 1 -- 172.21.15.93:0/1211433058 wait complete. 2023-11-16T16:46:58.884 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:58 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/36560789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:46:58.884 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:58 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/102087945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:46:58.887 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-16T16:46:58.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:58 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/36560789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:46:58.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:58 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/102087945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:46:59.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:58 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/36560789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:46:59.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:58 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/102087945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:46:59.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.086+0000 7fabacbff700 1 -- 172.21.15.93:0/2769534057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faba81014b0 msgr2=0x7faba8101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:59.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.086+0000 7fabacbff700 1 --2- 172.21.15.93:0/2769534057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faba81014b0 0x7faba8101900 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab90009990 tx=0x7fab90009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.086+0000 7fabacbff700 1 -- 172.21.15.93:0/2769534057 shutdown_connections 2023-11-16T16:46:59.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.086+0000 7fabacbff700 1 --2- 172.21.15.93:0/2769534057 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba8101e40 0x7faba8108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.086+0000 7fabacbff700 1 --2- 172.21.15.93:0/2769534057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faba81002c0 0x7faba81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.086+0000 7fabacbff700 1 --2- 172.21.15.93:0/2769534057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faba81014b0 0x7faba8101900 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.086+0000 7fabacbff700 1 -- 172.21.15.93:0/2769534057 >> 172.21.15.93:0/2769534057 conn(0x7faba80fb8a0 msgr2=0x7faba80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:59.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.086+0000 7fabacbff700 1 -- 172.21.15.93:0/2769534057 shutdown_connections 2023-11-16T16:46:59.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.086+0000 7fabacbff700 1 -- 172.21.15.93:0/2769534057 wait complete. 2023-11-16T16:46:59.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.087+0000 7fabacbff700 1 Processor -- start 2023-11-16T16:46:59.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.087+0000 7fabacbff700 1 -- start start 2023-11-16T16:46:59.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.087+0000 7fabacbff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba81002c0 0x7faba8196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:59.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.087+0000 7fabacbff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faba81014b0 0x7faba8196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:59.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.087+0000 7fabacbff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faba8101e40 0x7faba819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:59.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.087+0000 7fabacbff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba819dba0 con 0x7faba81002c0 2023-11-16T16:46:59.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.087+0000 7fabacbff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba819dce0 con 0x7faba8101e40 2023-11-16T16:46:59.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.087+0000 7fabacbff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba819de70 con 0x7faba81014b0 2023-11-16T16:46:59.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7faba659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba81002c0 0x7faba8196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:59.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7faba659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba81002c0 0x7faba8196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38602/0 (socket says 172.21.15.93:38602) 2023-11-16T16:46:59.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7faba659c700 1 -- 172.21.15.93:0/387331455 learned_addr learned my addr 172.21.15.93:0/387331455 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:59.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7fab9ffff700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faba81014b0 0x7faba8196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:59.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7faba6d9d700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faba8101e40 0x7faba819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:59.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7faba659c700 1 -- 172.21.15.93:0/387331455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faba81014b0 msgr2=0x7faba8196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:59.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7faba659c700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faba81014b0 0x7faba8196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7faba659c700 1 -- 172.21.15.93:0/387331455 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faba8101e40 msgr2=0x7faba819d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:59.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7faba659c700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faba8101e40 0x7faba819d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7faba659c700 1 -- 172.21.15.93:0/387331455 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab900095f0 con 0x7faba81002c0 2023-11-16T16:46:59.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7faba659c700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba81002c0 0x7faba8196020 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab9800e970 tx=0x7fab9800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:59.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7fab9f7fe700 1 -- 172.21.15.93:0/387331455 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98009790 con 0x7faba81002c0 2023-11-16T16:46:59.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7fab9f7fe700 1 -- 172.21.15.93:0/387331455 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab9800c7e0 con 0x7faba81002c0 2023-11-16T16:46:59.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.088+0000 7fabacbff700 1 -- 172.21.15.93:0/387331455 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba810bc20 con 0x7faba81002c0 2023-11-16T16:46:59.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.089+0000 7fab9f7fe700 1 -- 172.21.15.93:0/387331455 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98003c70 con 0x7faba81002c0 2023-11-16T16:46:59.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.089+0000 7fabacbff700 1 -- 172.21.15.93:0/387331455 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba810ba90 con 0x7faba81002c0 2023-11-16T16:46:59.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.090+0000 7fabacbff700 1 -- 172.21.15.93:0/387331455 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba8107860 con 0x7faba81002c0 2023-11-16T16:46:59.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.092+0000 7fab9f7fe700 1 -- 172.21.15.93:0/387331455 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fab9800cc50 con 0x7faba81002c0 2023-11-16T16:46:59.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.093+0000 7fab9f7fe700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fab940a1150 0x7fab940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:59.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.093+0000 7fab9f7fe700 1 -- 172.21.15.93:0/387331455 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fab98014070 con 0x7faba81002c0 2023-11-16T16:46:59.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.093+0000 7fab9ffff700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fab940a1150 0x7fab940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:59.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.094+0000 7fab9ffff700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fab940a1150 0x7fab940a3600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fab9000b2e0 tx=0x7fab90005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:59.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.095+0000 7fab9f7fe700 1 -- 172.21.15.93:0/387331455 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fab98096370 con 0x7faba81002c0 2023-11-16T16:46:59.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.294+0000 7fabacbff700 1 -- 172.21.15.93:0/387331455 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faba8067b00 con 0x7faba81002c0 2023-11-16T16:46:59.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.295+0000 7fab9f7fe700 1 -- 172.21.15.93:0/387331455 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fab98099990 con 0x7faba81002c0 2023-11-16T16:46:59.296 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:46:59.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.297+0000 7fabacbff700 1 -- 172.21.15.93:0/387331455 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fab940a1150 msgr2=0x7fab940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:59.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.297+0000 7fabacbff700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fab940a1150 0x7fab940a3600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fab9000b2e0 tx=0x7fab90005fb0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.297+0000 7fabacbff700 1 -- 172.21.15.93:0/387331455 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba81002c0 msgr2=0x7faba8196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:59.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.297+0000 7fabacbff700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba81002c0 0x7faba8196020 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab9800e970 tx=0x7fab9800ec80 comp rx=0 tx=0).stop 2023-11-16T16:46:59.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.297+0000 7fabacbff700 1 -- 172.21.15.93:0/387331455 shutdown_connections 2023-11-16T16:46:59.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.297+0000 7fabacbff700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fab940a1150 0x7fab940a3600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.297+0000 7fabacbff700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba81002c0 0x7faba8196020 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.297+0000 7fabacbff700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faba8101e40 0x7faba819d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.297+0000 7fabacbff700 1 --2- 172.21.15.93:0/387331455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faba81014b0 0x7faba8196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.297+0000 7fabacbff700 1 -- 172.21.15.93:0/387331455 >> 172.21.15.93:0/387331455 conn(0x7faba80fb8a0 msgr2=0x7faba81049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:59.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.298+0000 7fabacbff700 1 -- 172.21.15.93:0/387331455 shutdown_connections 2023-11-16T16:46:59.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.298+0000 7fabacbff700 1 -- 172.21.15.93:0/387331455 wait complete. 2023-11-16T16:46:59.318 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-16T16:46:59.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.513+0000 7f9400fff700 1 -- 172.21.15.93:0/2561228343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93fc101420 msgr2=0x7f93fc101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:59.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.513+0000 7f9400fff700 1 --2- 172.21.15.93:0/2561228343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93fc101420 0x7f93fc101870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f93f00098f0 tx=0x7f93f0009c00 comp rx=0 tx=0).stop 2023-11-16T16:46:59.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.513+0000 7f9400fff700 1 -- 172.21.15.93:0/2561228343 shutdown_connections 2023-11-16T16:46:59.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.513+0000 7f9400fff700 1 --2- 172.21.15.93:0/2561228343 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93fc101db0 0x7f93fc108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.513+0000 7f9400fff700 1 --2- 172.21.15.93:0/2561228343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc100230 0x7f93fc100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.513+0000 7f9400fff700 1 --2- 172.21.15.93:0/2561228343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93fc101420 0x7f93fc101870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.513+0000 7f9400fff700 1 -- 172.21.15.93:0/2561228343 >> 172.21.15.93:0/2561228343 conn(0x7f93fc0fb810 msgr2=0x7f93fc0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:59.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.513+0000 7f9400fff700 1 -- 172.21.15.93:0/2561228343 shutdown_connections 2023-11-16T16:46:59.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.513+0000 7f9400fff700 1 -- 172.21.15.93:0/2561228343 wait complete. 2023-11-16T16:46:59.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.514+0000 7f9400fff700 1 Processor -- start 2023-11-16T16:46:59.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.515+0000 7f9400fff700 1 -- start start 2023-11-16T16:46:59.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.515+0000 7f9400fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93fc100230 0x7f93fc198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:59.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.515+0000 7f9400fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc101420 0x7f93fc198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:59.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.515+0000 7f9400fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93fc101db0 0x7f93fc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:59.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.515+0000 7f9400fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc19fda0 con 0x7f93fc101db0 2023-11-16T16:46:59.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.515+0000 7f9400fff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc19fee0 con 0x7f93fc100230 2023-11-16T16:46:59.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.515+0000 7f9400fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc1a0070 con 0x7f93fc101420 2023-11-16T16:46:59.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.515+0000 7f93fad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93fc101db0 0x7f93fc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:59.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.515+0000 7f93fad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93fc101db0 0x7f93fc19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38620/0 (socket says 172.21.15.93:38620) 2023-11-16T16:46:59.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.515+0000 7f93fad9d700 1 -- 172.21.15.93:0/655293442 learned_addr learned my addr 172.21.15.93:0/655293442 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:46:59.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.515+0000 7f93fa59c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93fc100230 0x7f93fc198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:59.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.515+0000 7f93f9d9b700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc101420 0x7f93fc198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:59.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.516+0000 7f93fad9d700 1 -- 172.21.15.93:0/655293442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc101420 msgr2=0x7f93fc198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:59.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.516+0000 7f93fad9d700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc101420 0x7f93fc198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.516+0000 7f93fad9d700 1 -- 172.21.15.93:0/655293442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93fc100230 msgr2=0x7f93fc198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:59.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.516+0000 7f93fad9d700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93fc100230 0x7f93fc198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.516+0000 7f93fad9d700 1 -- 172.21.15.93:0/655293442 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f0009520 con 0x7f93fc101db0 2023-11-16T16:46:59.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.516+0000 7f93fad9d700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93fc101db0 0x7f93fc19f7d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c110 tx=0x7f93f400c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:59.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.516+0000 7f93eb7fe700 1 -- 172.21.15.93:0/655293442 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4017070 con 0x7f93fc101db0 2023-11-16T16:46:59.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.516+0000 7f93eb7fe700 1 -- 172.21.15.93:0/655293442 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93f400e050 con 0x7f93fc101db0 2023-11-16T16:46:59.520 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.516+0000 7f9400fff700 1 -- 172.21.15.93:0/655293442 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93fc10bbf0 con 0x7f93fc101db0 2023-11-16T16:46:59.520 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.516+0000 7f93eb7fe700 1 -- 172.21.15.93:0/655293442 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40078f0 con 0x7f93fc101db0 2023-11-16T16:46:59.520 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.516+0000 7f9400fff700 1 -- 172.21.15.93:0/655293442 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc10ba60 con 0x7f93fc101db0 2023-11-16T16:46:59.520 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.517+0000 7f9400fff700 1 -- 172.21.15.93:0/655293442 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc0679f0 con 0x7f93fc101db0 2023-11-16T16:46:59.522 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.520+0000 7f93eb7fe700 1 -- 172.21.15.93:0/655293442 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f93f4007a50 con 0x7f93fc101db0 2023-11-16T16:46:59.523 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.522+0000 7f93eb7fe700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f93e40a1150 0x7f93e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:59.523 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.522+0000 7f93eb7fe700 1 -- 172.21.15.93:0/655293442 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f93f40cd0d0 con 0x7f93fc101db0 2023-11-16T16:46:59.523 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.522+0000 7f93fa59c700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f93e40a1150 0x7f93e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:59.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.523+0000 7f93fa59c700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f93e40a1150 0x7f93e40a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f93fc101280 tx=0x7f93ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:59.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.524+0000 7f93eb7fe700 1 -- 172.21.15.93:0/655293442 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f93f40964c0 con 0x7f93fc101db0 2023-11-16T16:46:59.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.740+0000 7f9400fff700 1 -- 172.21.15.93:0/655293442 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f93fc1a0ee0 con 0x7f93fc101db0 2023-11-16T16:46:59.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.741+0000 7f93eb7fe700 1 -- 172.21.15.93:0/655293442 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+588 (secure 0 0 0) 0x7f93f4099ae0 con 0x7f93fc101db0 2023-11-16T16:46:59.743 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/1465890024 2023-11-17T16:39:50.432397+0000 2023-11-16T16:46:59.743 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/3409666551 2023-11-17T16:39:50.432397+0000 2023-11-16T16:46:59.743 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/2198359920 2023-11-17T16:39:50.432397+0000 2023-11-16T16:46:59.743 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/772471105 2023-11-17T16:39:24.569132+0000 2023-11-16T16:46:59.743 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/772471105 2023-11-17T16:39:24.569132+0000 2023-11-16T16:46:59.743 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/2198359920 2023-11-17T16:39:50.432397+0000 2023-11-16T16:46:59.743 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/438537209 2023-11-17T16:39:24.569132+0000 2023-11-16T16:46:59.744 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2518663109 2023-11-17T16:39:24.569132+0000 2023-11-16T16:46:59.744 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/696516505 2023-11-17T16:39:24.569132+0000 2023-11-16T16:46:59.744 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/3700969994 2023-11-17T16:39:50.432397+0000 2023-11-16T16:46:59.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.743+0000 7f9400fff700 1 -- 172.21.15.93:0/655293442 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f93e40a1150 msgr2=0x7f93e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:59.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.743+0000 7f9400fff700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f93e40a1150 0x7f93e40a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f93fc101280 tx=0x7f93ec009260 comp rx=0 tx=0).stop 2023-11-16T16:46:59.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.743+0000 7f9400fff700 1 -- 172.21.15.93:0/655293442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93fc101db0 msgr2=0x7f93fc19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:59.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.743+0000 7f9400fff700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93fc101db0 0x7f93fc19f7d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c110 tx=0x7f93f400c420 comp rx=0 tx=0).stop 2023-11-16T16:46:59.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.743+0000 7f9400fff700 1 -- 172.21.15.93:0/655293442 shutdown_connections 2023-11-16T16:46:59.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.743+0000 7f9400fff700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f93e40a1150 0x7f93e40a3600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.743+0000 7f9400fff700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93fc101db0 0x7f93fc19f7d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.743+0000 7f9400fff700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93fc100230 0x7f93fc198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.743+0000 7f9400fff700 1 --2- 172.21.15.93:0/655293442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc101420 0x7f93fc198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:59.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.743+0000 7f9400fff700 1 -- 172.21.15.93:0/655293442 >> 172.21.15.93:0/655293442 conn(0x7f93fc0fb810 msgr2=0x7f93fc0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:59.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.743+0000 7f9400fff700 1 -- 172.21.15.93:0/655293442 shutdown_connections 2023-11-16T16:46:59.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:46:59.744+0000 7f9400fff700 1 -- 172.21.15.93:0/655293442 wait complete. 2023-11-16T16:46:59.747 INFO:teuthology.orchestra.run.smithi093.stderr:listed 10 entries 2023-11-16T16:46:59.763 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi093.front.sepia.ceph.com... 2023-11-16T16:46:59.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:46:59.764 DEBUG:teuthology.orchestra.run.smithi093:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-16T16:46:59.830 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi093.stderr:ceph-fuse[134685]: fuse finished with error 0 and tester_r 0 2023-11-16T16:46:59.834 INFO:teuthology.orchestra.run:waiting for 300 2023-11-16T16:46:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:59 smithi093 ceph-mon[97863]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:46:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:59 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1211433058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:46:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:46:59 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/387331455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:46:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:59 smithi159 ceph-mon[131528]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:46:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:59 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1211433058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:46:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:46:59 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/387331455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:47:00.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:59 smithi125 ceph-mon[131076]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:00.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:59 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1211433058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:47:00.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:46:59 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/387331455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:47:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:00 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/655293442' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-16T16:47:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:00 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/655293442' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-16T16:47:01.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:00 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/655293442' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-16T16:47:01.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:01 smithi093 ceph-mon[97863]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 445 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:01.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:01 smithi159 ceph-mon[131528]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 445 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:02.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:01 smithi125 ceph-mon[131076]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 445 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:03.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:03 smithi093 ceph-mon[97863]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 401 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:03.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:03 smithi159 ceph-mon[131528]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 401 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:04.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:03 smithi125 ceph-mon[131076]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 401 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:05.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:05 smithi093 ceph-mon[97863]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:05.937 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-16T16:47:05.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:47:05.937 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-16T16:47:05.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:05 smithi159 ceph-mon[131528]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:06.006 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:47:06.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:05 smithi125 ceph-mon[131076]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:06.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.225+0000 7fd1f39ff700 1 -- 172.21.15.93:0/1043070081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1ec101e30 msgr2=0x7fd1ec1086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:06.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.225+0000 7fd1f39ff700 1 --2- 172.21.15.93:0/1043070081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1ec101e30 0x7fd1ec1086f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400b0a0 tx=0x7fd1e400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.225+0000 7fd1f39ff700 1 -- 172.21.15.93:0/1043070081 shutdown_connections 2023-11-16T16:47:06.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.225+0000 7fd1f39ff700 1 --2- 172.21.15.93:0/1043070081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1ec101e30 0x7fd1ec1086f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.225+0000 7fd1f39ff700 1 --2- 172.21.15.93:0/1043070081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1ec1002b0 0x7fd1ec1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.225+0000 7fd1f39ff700 1 --2- 172.21.15.93:0/1043070081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1ec1014a0 0x7fd1ec1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.225+0000 7fd1f39ff700 1 -- 172.21.15.93:0/1043070081 >> 172.21.15.93:0/1043070081 conn(0x7fd1ec0fb890 msgr2=0x7fd1ec0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:06.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.225+0000 7fd1f39ff700 1 -- 172.21.15.93:0/1043070081 shutdown_connections 2023-11-16T16:47:06.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.226+0000 7fd1f39ff700 1 -- 172.21.15.93:0/1043070081 wait complete. 2023-11-16T16:47:06.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.226+0000 7fd1f39ff700 1 Processor -- start 2023-11-16T16:47:06.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f39ff700 1 -- start start 2023-11-16T16:47:06.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f39ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1ec1002b0 0x7fd1ec1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:06.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f39ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1ec1014a0 0x7fd1ec1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:06.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f39ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1ec101e30 0x7fd1ec19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:06.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f39ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec19fe20 con 0x7fd1ec101e30 2023-11-16T16:47:06.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f39ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec19ff60 con 0x7fd1ec1014a0 2023-11-16T16:47:06.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f39ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1a00f0 con 0x7fd1ec1002b0 2023-11-16T16:47:06.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f0f9a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1ec1014a0 0x7fd1ec1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:06.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f1f9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1ec101e30 0x7fd1ec19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:06.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f1f9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1ec101e30 0x7fd1ec19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38644/0 (socket says 172.21.15.93:38644) 2023-11-16T16:47:06.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f1f9c700 1 -- 172.21.15.93:0/2918834954 learned_addr learned my addr 172.21.15.93:0/2918834954 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:06.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f0f9a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1ec1014a0 0x7fd1ec1987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:32910/0 (socket says 172.21.15.93:32910) 2023-11-16T16:47:06.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.227+0000 7fd1f179b700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1ec1002b0 0x7fd1ec1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:06.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.228+0000 7fd1f1f9c700 1 -- 172.21.15.93:0/2918834954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1ec1002b0 msgr2=0x7fd1ec1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:06.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.228+0000 7fd1f1f9c700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1ec1002b0 0x7fd1ec1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.228+0000 7fd1f1f9c700 1 -- 172.21.15.93:0/2918834954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1ec1014a0 msgr2=0x7fd1ec1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:06.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.228+0000 7fd1f1f9c700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1ec1014a0 0x7fd1ec1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.228+0000 7fd1f1f9c700 1 -- 172.21.15.93:0/2918834954 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e4009d00 con 0x7fd1ec101e30 2023-11-16T16:47:06.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.228+0000 7fd1f0f9a700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1ec1014a0 0x7fd1ec1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:06.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.228+0000 7fd1f1f9c700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1ec101e30 0x7fd1ec19f850 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400d010 tx=0x7fd1e400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:06.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.228+0000 7fd1e27fc700 1 -- 172.21.15.93:0/2918834954 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400e040 con 0x7fd1ec101e30 2023-11-16T16:47:06.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.228+0000 7fd1e27fc700 1 -- 172.21.15.93:0/2918834954 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e4003e10 con 0x7fd1ec101e30 2023-11-16T16:47:06.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.228+0000 7fd1e27fc700 1 -- 172.21.15.93:0/2918834954 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40052a0 con 0x7fd1ec101e30 2023-11-16T16:47:06.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.228+0000 7fd1f39ff700 1 -- 172.21.15.93:0/2918834954 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec10bc80 con 0x7fd1ec101e30 2023-11-16T16:47:06.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.228+0000 7fd1f39ff700 1 -- 172.21.15.93:0/2918834954 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec10baf0 con 0x7fd1ec101e30 2023-11-16T16:47:06.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.229+0000 7fd1f39ff700 1 -- 172.21.15.93:0/2918834954 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec1a0640 con 0x7fd1ec101e30 2023-11-16T16:47:06.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.231+0000 7fd1e27fc700 1 -- 172.21.15.93:0/2918834954 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd1e401d480 con 0x7fd1ec101e30 2023-11-16T16:47:06.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.232+0000 7fd1e27fc700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd1d80a11d0 0x7fd1d80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:06.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.233+0000 7fd1e27fc700 1 -- 172.21.15.93:0/2918834954 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd1e40cdf40 con 0x7fd1ec101e30 2023-11-16T16:47:06.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.233+0000 7fd1f179b700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd1d80a11d0 0x7fd1d80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:06.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.233+0000 7fd1f179b700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd1d80a11d0 0x7fd1d80a3680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec101300 tx=0x7fd1e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:06.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.237+0000 7fd1e27fc700 1 -- 172.21.15.93:0/2918834954 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd1e40973c0 con 0x7fd1ec101e30 2023-11-16T16:47:06.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.435+0000 7fd1f39ff700 1 -- 172.21.15.93:0/2918834954 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1ec1a0bf0 con 0x7fd1ec101e30 2023-11-16T16:47:06.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.437+0000 7fd1e27fc700 1 -- 172.21.15.93:0/2918834954 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24468 (secure 0 0 0) 0x7fd1e409a9e0 con 0x7fd1ec101e30 2023-11-16T16:47:06.439 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:47:06.440 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":90,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:46:52.315313+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:47:06.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.440+0000 7fd1f39ff700 1 -- 172.21.15.93:0/2918834954 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd1d80a11d0 msgr2=0x7fd1d80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:06.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.440+0000 7fd1f39ff700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd1d80a11d0 0x7fd1d80a3680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec101300 tx=0x7fd1e8009260 comp rx=0 tx=0).stop 2023-11-16T16:47:06.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.440+0000 7fd1f39ff700 1 -- 172.21.15.93:0/2918834954 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1ec101e30 msgr2=0x7fd1ec19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:06.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.440+0000 7fd1f39ff700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1ec101e30 0x7fd1ec19f850 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400d010 tx=0x7fd1e400bc80 comp rx=0 tx=0).stop 2023-11-16T16:47:06.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.440+0000 7fd1f39ff700 1 -- 172.21.15.93:0/2918834954 shutdown_connections 2023-11-16T16:47:06.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.440+0000 7fd1f39ff700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd1d80a11d0 0x7fd1d80a3680 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.440+0000 7fd1f39ff700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1ec101e30 0x7fd1ec19f850 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.440+0000 7fd1f39ff700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1ec1002b0 0x7fd1ec1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.440+0000 7fd1f39ff700 1 --2- 172.21.15.93:0/2918834954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1ec1014a0 0x7fd1ec1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.440+0000 7fd1f39ff700 1 -- 172.21.15.93:0/2918834954 >> 172.21.15.93:0/2918834954 conn(0x7fd1ec0fb890 msgr2=0x7fd1ec106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:06.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.440+0000 7fd1f39ff700 1 -- 172.21.15.93:0/2918834954 shutdown_connections 2023-11-16T16:47:06.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.440+0000 7fd1f39ff700 1 -- 172.21.15.93:0/2918834954 wait complete. 2023-11-16T16:47:06.461 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:47:06.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.660+0000 7fa684bff700 1 -- 172.21.15.93:0/3039049767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa680100290 msgr2=0x7fa6801006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.660+0000 7fa684bff700 1 --2- 172.21.15.93:0/3039049767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa680100290 0x7fa6801006a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa670009990 tx=0x7fa670009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.661+0000 7fa684bff700 1 -- 172.21.15.93:0/3039049767 shutdown_connections 2023-11-16T16:47:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.661+0000 7fa684bff700 1 --2- 172.21.15.93:0/3039049767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa680101e10 0x7fa6801086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.661+0000 7fa684bff700 1 --2- 172.21.15.93:0/3039049767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa680100290 0x7fa6801006a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.661+0000 7fa684bff700 1 --2- 172.21.15.93:0/3039049767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa680101480 0x7fa6801018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.661+0000 7fa684bff700 1 -- 172.21.15.93:0/3039049767 >> 172.21.15.93:0/3039049767 conn(0x7fa6800fb870 msgr2=0x7fa6800fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.661+0000 7fa684bff700 1 -- 172.21.15.93:0/3039049767 shutdown_connections 2023-11-16T16:47:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.661+0000 7fa684bff700 1 -- 172.21.15.93:0/3039049767 wait complete. 2023-11-16T16:47:06.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.662+0000 7fa684bff700 1 Processor -- start 2023-11-16T16:47:06.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.662+0000 7fa684bff700 1 -- start start 2023-11-16T16:47:06.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa684bff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa680100290 0x7fa680195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:06.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa684bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa680101480 0x7fa680196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:06.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa684bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa680101e10 0x7fa68019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:06.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa684bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68019db50 con 0x7fa680101e10 2023-11-16T16:47:06.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa684bff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68019dc90 con 0x7fa680100290 2023-11-16T16:47:06.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa684bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68019de20 con 0x7fa680101480 2023-11-16T16:47:06.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa67ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa680101e10 0x7fa68019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:06.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa67ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa680101e10 0x7fa68019d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38664/0 (socket says 172.21.15.93:38664) 2023-11-16T16:47:06.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa67ed9d700 1 -- 172.21.15.93:0/3875611933 learned_addr learned my addr 172.21.15.93:0/3875611933 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:06.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa67dd9b700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa680101480 0x7fa680196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:06.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa67e59c700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa680100290 0x7fa680195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:06.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa67ed9d700 1 -- 172.21.15.93:0/3875611933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa680101480 msgr2=0x7fa680196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:06.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa67ed9d700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa680101480 0x7fa680196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa67ed9d700 1 -- 172.21.15.93:0/3875611933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa680100290 msgr2=0x7fa680195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:06.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa67ed9d700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa680100290 0x7fa680195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.663+0000 7fa67ed9d700 1 -- 172.21.15.93:0/3875611933 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700095f0 con 0x7fa680101e10 2023-11-16T16:47:06.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.664+0000 7fa67ed9d700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa680101e10 0x7fa68019d580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa67800ef30 tx=0x7fa67800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:06.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.664+0000 7fa66f7fe700 1 -- 172.21.15.93:0/3875611933 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678018070 con 0x7fa680101e10 2023-11-16T16:47:06.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.664+0000 7fa66f7fe700 1 -- 172.21.15.93:0/3875611933 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa67800f040 con 0x7fa680101e10 2023-11-16T16:47:06.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.664+0000 7fa684bff700 1 -- 172.21.15.93:0/3875611933 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68010bc40 con 0x7fa680101e10 2023-11-16T16:47:06.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.664+0000 7fa66f7fe700 1 -- 172.21.15.93:0/3875611933 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67800cd00 con 0x7fa680101e10 2023-11-16T16:47:06.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.664+0000 7fa684bff700 1 -- 172.21.15.93:0/3875611933 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68010bab0 con 0x7fa680101e10 2023-11-16T16:47:06.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.665+0000 7fa684bff700 1 -- 172.21.15.93:0/3875611933 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa680067a60 con 0x7fa680101e10 2023-11-16T16:47:06.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.667+0000 7fa66f7fe700 1 -- 172.21.15.93:0/3875611933 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa678003a00 con 0x7fa680101e10 2023-11-16T16:47:06.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.669+0000 7fa66f7fe700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa6680a1150 0x7fa6680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:06.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.669+0000 7fa66f7fe700 1 -- 172.21.15.93:0/3875611933 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa6780cc790 con 0x7fa680101e10 2023-11-16T16:47:06.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.669+0000 7fa67e59c700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa6680a1150 0x7fa6680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:06.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.670+0000 7fa67e59c700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa6680a1150 0x7fa6680a3600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa67000b2e0 tx=0x7fa670005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:06.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.673+0000 7fa66f7fe700 1 -- 172.21.15.93:0/3875611933 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa678095c10 con 0x7fa680101e10 2023-11-16T16:47:06.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.870+0000 7fa684bff700 1 -- 172.21.15.93:0/3875611933 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa680107210 con 0x7fa680101e10 2023-11-16T16:47:06.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.872+0000 7fa66f7fe700 1 -- 172.21.15.93:0/3875611933 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24468 (secure 0 0 0) 0x7fa678099230 con 0x7fa680101e10 2023-11-16T16:47:06.874 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:47:06.874 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":90,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:46:52.315313+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:47:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.874+0000 7fa684bff700 1 -- 172.21.15.93:0/3875611933 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa6680a1150 msgr2=0x7fa6680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.874+0000 7fa684bff700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa6680a1150 0x7fa6680a3600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa67000b2e0 tx=0x7fa670005fb0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.874+0000 7fa684bff700 1 -- 172.21.15.93:0/3875611933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa680101e10 msgr2=0x7fa68019d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.874+0000 7fa684bff700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa680101e10 0x7fa68019d580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa67800ef30 tx=0x7fa67800c3a0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.874+0000 7fa684bff700 1 -- 172.21.15.93:0/3875611933 shutdown_connections 2023-11-16T16:47:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.874+0000 7fa684bff700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa6680a1150 0x7fa6680a3600 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.874+0000 7fa684bff700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa680101e10 0x7fa68019d580 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.874+0000 7fa684bff700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa680100290 0x7fa680195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.874+0000 7fa684bff700 1 --2- 172.21.15.93:0/3875611933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa680101480 0x7fa680196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:06.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.874+0000 7fa684bff700 1 -- 172.21.15.93:0/3875611933 >> 172.21.15.93:0/3875611933 conn(0x7fa6800fb870 msgr2=0x7fa6800fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:06.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.874+0000 7fa684bff700 1 -- 172.21.15.93:0/3875611933 shutdown_connections 2023-11-16T16:47:06.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:06.874+0000 7fa684bff700 1 -- 172.21.15.93:0/3875611933 wait complete. 2023-11-16T16:47:06.892 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:06 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2918834954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:06.895 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-16T16:47:06.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:06 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2918834954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:07.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:06 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2918834954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:07.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.094+0000 7f054afff700 1 -- 172.21.15.93:0/816889967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05441014c0 msgr2=0x7f0544101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:07.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.094+0000 7f054afff700 1 --2- 172.21.15.93:0/816889967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05441014c0 0x7f0544101910 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0530009990 tx=0x7f0530009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.094+0000 7f054afff700 1 -- 172.21.15.93:0/816889967 shutdown_connections 2023-11-16T16:47:07.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.094+0000 7f054afff700 1 --2- 172.21.15.93:0/816889967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0544101e50 0x7f0544108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.094+0000 7f054afff700 1 --2- 172.21.15.93:0/816889967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05441002d0 0x7f05441006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.094+0000 7f054afff700 1 --2- 172.21.15.93:0/816889967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05441014c0 0x7f0544101910 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.094+0000 7f054afff700 1 -- 172.21.15.93:0/816889967 >> 172.21.15.93:0/816889967 conn(0x7f05440fb8b0 msgr2=0x7f05440fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:07.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.094+0000 7f054afff700 1 -- 172.21.15.93:0/816889967 shutdown_connections 2023-11-16T16:47:07.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.094+0000 7f054afff700 1 -- 172.21.15.93:0/816889967 wait complete. 2023-11-16T16:47:07.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.095+0000 7f054afff700 1 Processor -- start 2023-11-16T16:47:07.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.095+0000 7f054afff700 1 -- start start 2023-11-16T16:47:07.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054afff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05441002d0 0x7f0544198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:07.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054afff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05441014c0 0x7f0544198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:07.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054afff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0544101e50 0x7f054419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:07.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054afff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054419fe90 con 0x7f0544101e50 2023-11-16T16:47:07.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054afff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054419ffd0 con 0x7f05441002d0 2023-11-16T16:47:07.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054afff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05441a0160 con 0x7f05441014c0 2023-11-16T16:47:07.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054959c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0544101e50 0x7f054419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:07.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054959c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0544101e50 0x7f054419f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42390/0 (socket says 172.21.15.93:42390) 2023-11-16T16:47:07.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054959c700 1 -- 172.21.15.93:0/2436214015 learned_addr learned my addr 172.21.15.93:0/2436214015 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:07.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f0548d9b700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05441002d0 0x7f0544198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:07.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f0543fff700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05441014c0 0x7f0544198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:07.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054959c700 1 -- 172.21.15.93:0/2436214015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05441014c0 msgr2=0x7f0544198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:07.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054959c700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05441014c0 0x7f0544198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054959c700 1 -- 172.21.15.93:0/2436214015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05441002d0 msgr2=0x7f0544198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:07.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054959c700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05441002d0 0x7f0544198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f054959c700 1 -- 172.21.15.93:0/2436214015 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300095f0 con 0x7f0544101e50 2023-11-16T16:47:07.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.096+0000 7f0543fff700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05441014c0 0x7f0544198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:07.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.097+0000 7f054959c700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0544101e50 0x7f054419f8c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f053c00ef30 tx=0x7f053c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:07.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.097+0000 7f0541ffb700 1 -- 172.21.15.93:0/2436214015 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c018070 con 0x7f0544101e50 2023-11-16T16:47:07.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.097+0000 7f054afff700 1 -- 172.21.15.93:0/2436214015 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054410bc80 con 0x7f0544101e50 2023-11-16T16:47:07.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.097+0000 7f0541ffb700 1 -- 172.21.15.93:0/2436214015 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f053c00f040 con 0x7f0544101e50 2023-11-16T16:47:07.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.097+0000 7f0541ffb700 1 -- 172.21.15.93:0/2436214015 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c00cdb0 con 0x7f0544101e50 2023-11-16T16:47:07.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.097+0000 7f054afff700 1 -- 172.21.15.93:0/2436214015 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054410baf0 con 0x7f0544101e50 2023-11-16T16:47:07.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.098+0000 7f054afff700 1 -- 172.21.15.93:0/2436214015 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05441929e0 con 0x7f0544101e50 2023-11-16T16:47:07.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.100+0000 7f0541ffb700 1 -- 172.21.15.93:0/2436214015 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f053c003a00 con 0x7f0544101e50 2023-11-16T16:47:07.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.101+0000 7f0541ffb700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f05340a11d0 0x7f05340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:07.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.101+0000 7f0541ffb700 1 -- 172.21.15.93:0/2436214015 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f053c0cc710 con 0x7f0544101e50 2023-11-16T16:47:07.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.101+0000 7f0548d9b700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f05340a11d0 0x7f05340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:07.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.102+0000 7f0548d9b700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f05340a11d0 0x7f05340a3680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0544101320 tx=0x7f0538009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:07.106 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.105+0000 7f0541ffb700 1 -- 172.21.15.93:0/2436214015 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f053c095bf0 con 0x7f0544101e50 2023-11-16T16:47:07.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.349+0000 7f054afff700 1 -- 172.21.15.93:0/2436214015 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f054402cfe0 con 0x7f0544101e50 2023-11-16T16:47:07.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.618+0000 7f0541ffb700 1 -- 172.21.15.93:0/2436214015 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f053c099210 con 0x7f0544101e50 2023-11-16T16:47:07.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.620+0000 7f054afff700 1 -- 172.21.15.93:0/2436214015 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f05340a11d0 msgr2=0x7f05340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:07.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.620+0000 7f054afff700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f05340a11d0 0x7f05340a3680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0544101320 tx=0x7f0538009260 comp rx=0 tx=0).stop 2023-11-16T16:47:07.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.620+0000 7f054afff700 1 -- 172.21.15.93:0/2436214015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0544101e50 msgr2=0x7f054419f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:07.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.620+0000 7f054afff700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0544101e50 0x7f054419f8c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f053c00ef30 tx=0x7f053c00c3a0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.620+0000 7f054afff700 1 -- 172.21.15.93:0/2436214015 shutdown_connections 2023-11-16T16:47:07.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.620+0000 7f054afff700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f05340a11d0 0x7f05340a3680 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.620+0000 7f054afff700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0544101e50 0x7f054419f8c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.620+0000 7f054afff700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05441002d0 0x7f0544198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.620+0000 7f054afff700 1 --2- 172.21.15.93:0/2436214015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05441014c0 0x7f0544198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.620+0000 7f054afff700 1 -- 172.21.15.93:0/2436214015 >> 172.21.15.93:0/2436214015 conn(0x7f05440fb8b0 msgr2=0x7f0544106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:07.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.620+0000 7f054afff700 1 -- 172.21.15.93:0/2436214015 shutdown_connections 2023-11-16T16:47:07.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.620+0000 7f054afff700 1 -- 172.21.15.93:0/2436214015 wait complete. 2023-11-16T16:47:07.642 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-11-16T16:47:07.643 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-16T16:47:07.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.841+0000 7feff1fff700 1 -- 172.21.15.93:0/2655330562 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fefec1014b0 msgr2=0x7fefec101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:07.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.841+0000 7feff1fff700 1 --2- 172.21.15.93:0/2655330562 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fefec1014b0 0x7fefec101900 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009990 tx=0x7fefe0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.841+0000 7feff1fff700 1 -- 172.21.15.93:0/2655330562 shutdown_connections 2023-11-16T16:47:07.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.841+0000 7feff1fff700 1 --2- 172.21.15.93:0/2655330562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefec101e40 0x7fefec108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.841+0000 7feff1fff700 1 --2- 172.21.15.93:0/2655330562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fefec1002c0 0x7fefec1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.841+0000 7feff1fff700 1 --2- 172.21.15.93:0/2655330562 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fefec1014b0 0x7fefec101900 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.841+0000 7feff1fff700 1 -- 172.21.15.93:0/2655330562 >> 172.21.15.93:0/2655330562 conn(0x7fefec0fb8a0 msgr2=0x7fefec0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:07.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.842+0000 7feff1fff700 1 -- 172.21.15.93:0/2655330562 shutdown_connections 2023-11-16T16:47:07.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.842+0000 7feff1fff700 1 -- 172.21.15.93:0/2655330562 wait complete. 2023-11-16T16:47:07.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.843+0000 7feff1fff700 1 Processor -- start 2023-11-16T16:47:07.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.843+0000 7feff1fff700 1 -- start start 2023-11-16T16:47:07.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.843+0000 7feff1fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefec1002c0 0x7fefec0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:07.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.843+0000 7feff1fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fefec1014b0 0x7fefec072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:07.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.843+0000 7feff1fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fefec101e40 0x7fefec073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:07.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.843+0000 7feff1fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec073bc0 con 0x7fefec1014b0 2023-11-16T16:47:07.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.843+0000 7feff1fff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec073d30 con 0x7fefec1002c0 2023-11-16T16:47:07.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.843+0000 7feff1fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec1a00e0 con 0x7fefec101e40 2023-11-16T16:47:07.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.843+0000 7fefeaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fefec1014b0 0x7fefec072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:07.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.843+0000 7fefeaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fefec1014b0 0x7fefec072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42400/0 (socket says 172.21.15.93:42400) 2023-11-16T16:47:07.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.843+0000 7fefeaffd700 1 -- 172.21.15.93:0/2696926243 learned_addr learned my addr 172.21.15.93:0/2696926243 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:07.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.843+0000 7fefeb7fe700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefec1002c0 0x7fefec0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:07.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.844+0000 7fefebfff700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fefec101e40 0x7fefec073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:07.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.844+0000 7fefeaffd700 1 -- 172.21.15.93:0/2696926243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fefec101e40 msgr2=0x7fefec073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:07.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.844+0000 7fefeaffd700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fefec101e40 0x7fefec073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.844+0000 7fefeaffd700 1 -- 172.21.15.93:0/2696926243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefec1002c0 msgr2=0x7fefec0748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:07.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.844+0000 7fefeaffd700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefec1002c0 0x7fefec0748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:07.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.844+0000 7fefeaffd700 1 -- 172.21.15.93:0/2696926243 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe00095f0 con 0x7fefec1014b0 2023-11-16T16:47:07.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.844+0000 7fefeaffd700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fefec1014b0 0x7fefec072ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fefe0000c00 tx=0x7fefe00047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:07.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.844+0000 7fefe8ff9700 1 -- 172.21.15.93:0/2696926243 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe001d070 con 0x7fefec1014b0 2023-11-16T16:47:07.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.844+0000 7fefe8ff9700 1 -- 172.21.15.93:0/2696926243 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefe0022470 con 0x7fefec1014b0 2023-11-16T16:47:07.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.844+0000 7fefe8ff9700 1 -- 172.21.15.93:0/2696926243 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0017650 con 0x7fefec1014b0 2023-11-16T16:47:07.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.844+0000 7feff1fff700 1 -- 172.21.15.93:0/2696926243 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefec10bc30 con 0x7fefec1014b0 2023-11-16T16:47:07.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.844+0000 7feff1fff700 1 -- 172.21.15.93:0/2696926243 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec10baa0 con 0x7fefec1014b0 2023-11-16T16:47:07.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.845+0000 7feff1fff700 1 -- 172.21.15.93:0/2696926243 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefec107860 con 0x7fefec1014b0 2023-11-16T16:47:07.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.847+0000 7fefe8ff9700 1 -- 172.21.15.93:0/2696926243 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fefe00225e0 con 0x7fefec1014b0 2023-11-16T16:47:07.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.848+0000 7fefe8ff9700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fefd40a1150 0x7fefd40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:07.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.849+0000 7fefe8ff9700 1 -- 172.21.15.93:0/2696926243 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fefe00ce2c0 con 0x7fefec1014b0 2023-11-16T16:47:07.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.849+0000 7fefeb7fe700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fefd40a1150 0x7fefd40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:07.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.849+0000 7fefeb7fe700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fefd40a1150 0x7fefd40a3600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fefec101310 tx=0x7fefdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:07.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:07.852+0000 7fefe8ff9700 1 -- 172.21.15.93:0/2696926243 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fefe0097740 con 0x7fefec1014b0 2023-11-16T16:47:07.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:07 smithi093 ceph-mon[97863]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:07.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:07 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3875611933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:07.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:07 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2436214015' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-16T16:47:07.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:07 smithi159 ceph-mon[131528]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:07.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:07 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3875611933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:07.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:07 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2436214015' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-16T16:47:08.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:07 smithi125 ceph-mon[131076]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:08.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:07 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3875611933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:08.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:07 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2436214015' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-16T16:47:08.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.095+0000 7feff1fff700 1 -- 172.21.15.93:0/2696926243 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fefec061a80 con 0x7fefec1014b0 2023-11-16T16:47:08.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.628+0000 7fefe8ff9700 1 -- 172.21.15.93:0/2696926243 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fefe009ad60 con 0x7fefec1014b0 2023-11-16T16:47:08.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.631+0000 7feff1fff700 1 -- 172.21.15.93:0/2696926243 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fefd40a1150 msgr2=0x7fefd40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:08.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.631+0000 7feff1fff700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fefd40a1150 0x7fefd40a3600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fefec101310 tx=0x7fefdc009260 comp rx=0 tx=0).stop 2023-11-16T16:47:08.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.631+0000 7feff1fff700 1 -- 172.21.15.93:0/2696926243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fefec1014b0 msgr2=0x7fefec072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:08.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.631+0000 7feff1fff700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fefec1014b0 0x7fefec072ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fefe0000c00 tx=0x7fefe00047a0 comp rx=0 tx=0).stop 2023-11-16T16:47:08.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.631+0000 7feff1fff700 1 -- 172.21.15.93:0/2696926243 shutdown_connections 2023-11-16T16:47:08.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.631+0000 7feff1fff700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fefd40a1150 0x7fefd40a3600 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:08.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.631+0000 7feff1fff700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fefec101e40 0x7fefec073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:08.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.631+0000 7feff1fff700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefec1002c0 0x7fefec0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:08.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.631+0000 7feff1fff700 1 --2- 172.21.15.93:0/2696926243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fefec1014b0 0x7fefec072ee0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:08.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.631+0000 7feff1fff700 1 -- 172.21.15.93:0/2696926243 >> 172.21.15.93:0/2696926243 conn(0x7fefec0fb8a0 msgr2=0x7fefec1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:08.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.631+0000 7feff1fff700 1 -- 172.21.15.93:0/2696926243 shutdown_connections 2023-11-16T16:47:08.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.631+0000 7feff1fff700 1 -- 172.21.15.93:0/2696926243 wait complete. 2023-11-16T16:47:08.656 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-11-16T16:47:08.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.856+0000 7fd8259ff700 1 -- 172.21.15.93:0/2873324854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820074b30 msgr2=0x7fd820072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:08.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.856+0000 7fd8259ff700 1 --2- 172.21.15.93:0/2873324854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820074b30 0x7fd820072f80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098f0 tx=0x7fd810009c00 comp rx=0 tx=0).stop 2023-11-16T16:47:08.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.856+0000 7fd8259ff700 1 -- 172.21.15.93:0/2873324854 shutdown_connections 2023-11-16T16:47:08.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.856+0000 7fd8259ff700 1 --2- 172.21.15.93:0/2873324854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd820108180 0x7fd82010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:08.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.856+0000 7fd8259ff700 1 --2- 172.21.15.93:0/2873324854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820074b30 0x7fd820072f80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:08.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.856+0000 7fd8259ff700 1 --2- 172.21.15.93:0/2873324854 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd820073550 0x7fd8200739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:08.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.856+0000 7fd8259ff700 1 -- 172.21.15.93:0/2873324854 >> 172.21.15.93:0/2873324854 conn(0x7fd8200fb8a0 msgr2=0x7fd8200fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:08.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.856+0000 7fd8259ff700 1 -- 172.21.15.93:0/2873324854 shutdown_connections 2023-11-16T16:47:08.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.856+0000 7fd8259ff700 1 -- 172.21.15.93:0/2873324854 wait complete. 2023-11-16T16:47:08.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.857+0000 7fd8259ff700 1 Processor -- start 2023-11-16T16:47:08.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.857+0000 7fd8259ff700 1 -- start start 2023-11-16T16:47:08.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd8259ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820073550 0x7fd8201982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:08.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd8259ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd820074b30 0x7fd820198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:08.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd8259ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd820108180 0x7fd82019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:08.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd8259ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82019fe60 con 0x7fd820074b30 2023-11-16T16:47:08.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd8259ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82019ffa0 con 0x7fd820073550 2023-11-16T16:47:08.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd8259ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201a0130 con 0x7fd820108180 2023-11-16T16:47:08.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd81e7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd820074b30 0x7fd820198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:08.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd81effd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820073550 0x7fd8201982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:08.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd81e7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd820074b30 0x7fd820198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42406/0 (socket says 172.21.15.93:42406) 2023-11-16T16:47:08.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd81effd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820073550 0x7fd8201982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:48700/0 (socket says 172.21.15.93:48700) 2023-11-16T16:47:08.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd81e7fc700 1 -- 172.21.15.93:0/3792473881 learned_addr learned my addr 172.21.15.93:0/3792473881 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:08.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd81f7fe700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd820108180 0x7fd82019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:08.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd81e7fc700 1 -- 172.21.15.93:0/3792473881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd820108180 msgr2=0x7fd82019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:08.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd81e7fc700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd820108180 0x7fd82019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:08.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd81e7fc700 1 -- 172.21.15.93:0/3792473881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820073550 msgr2=0x7fd8201982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:08.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd81e7fc700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820073550 0x7fd8201982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:08.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.858+0000 7fd81e7fc700 1 -- 172.21.15.93:0/3792473881 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8140095f0 con 0x7fd820074b30 2023-11-16T16:47:08.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.859+0000 7fd81e7fc700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd820074b30 0x7fd820198820 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd81400ebf0 tx=0x7fd81400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:08.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.859+0000 7fd8249fd700 1 -- 172.21.15.93:0/3792473881 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814009a10 con 0x7fd820074b30 2023-11-16T16:47:08.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.859+0000 7fd8249fd700 1 -- 172.21.15.93:0/3792473881 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd81400cc90 con 0x7fd820074b30 2023-11-16T16:47:08.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.859+0000 7fd8259ff700 1 -- 172.21.15.93:0/3792473881 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd810009520 con 0x7fd820074b30 2023-11-16T16:47:08.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.859+0000 7fd8249fd700 1 -- 172.21.15.93:0/3792473881 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814018750 con 0x7fd820074b30 2023-11-16T16:47:08.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.859+0000 7fd8259ff700 1 -- 172.21.15.93:0/3792473881 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82010de80 con 0x7fd820074b30 2023-11-16T16:47:08.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.860+0000 7fd8259ff700 1 -- 172.21.15.93:0/3792473881 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82010e010 con 0x7fd820074b30 2023-11-16T16:47:08.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.862+0000 7fd8249fd700 1 -- 172.21.15.93:0/3792473881 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd8140188b0 con 0x7fd820074b30 2023-11-16T16:47:08.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.863+0000 7fd8249fd700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8080a1150 0x7fd8080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:08.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.863+0000 7fd8249fd700 1 -- 172.21.15.93:0/3792473881 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd814014070 con 0x7fd820074b30 2023-11-16T16:47:08.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.863+0000 7fd81effd700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8080a1150 0x7fd8080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:08.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.864+0000 7fd81effd700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8080a1150 0x7fd8080a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd810009fd0 tx=0x7fd810019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:08.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:08.867+0000 7fd8249fd700 1 -- 172.21.15.93:0/3792473881 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd814096670 con 0x7fd820074b30 2023-11-16T16:47:08.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:08 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2436214015' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-16T16:47:08.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:08 smithi093 ceph-mon[97863]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:47:08.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:08 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2696926243' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-16T16:47:08.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:08 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2436214015' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-16T16:47:08.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:08 smithi159 ceph-mon[131528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:47:08.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:08 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2696926243' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-16T16:47:09.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:08 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2436214015' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-16T16:47:09.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:08 smithi125 ceph-mon[131076]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:47:09.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:08 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2696926243' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-16T16:47:09.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.114+0000 7fd8259ff700 1 -- 172.21.15.93:0/3792473881 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fd820106830 con 0x7fd820074b30 2023-11-16T16:47:09.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.636+0000 7fd8249fd700 1 -- 172.21.15.93:0/3792473881 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fd814099c90 con 0x7fd820074b30 2023-11-16T16:47:09.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.639+0000 7fd8259ff700 1 -- 172.21.15.93:0/3792473881 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8080a1150 msgr2=0x7fd8080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:09.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.639+0000 7fd8259ff700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8080a1150 0x7fd8080a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd810009fd0 tx=0x7fd810019040 comp rx=0 tx=0).stop 2023-11-16T16:47:09.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.639+0000 7fd8259ff700 1 -- 172.21.15.93:0/3792473881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd820074b30 msgr2=0x7fd820198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:09.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.639+0000 7fd8259ff700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd820074b30 0x7fd820198820 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd81400ebf0 tx=0x7fd81400ef00 comp rx=0 tx=0).stop 2023-11-16T16:47:09.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.639+0000 7fd8259ff700 1 -- 172.21.15.93:0/3792473881 shutdown_connections 2023-11-16T16:47:09.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.639+0000 7fd8259ff700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8080a1150 0x7fd8080a3600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:09.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.639+0000 7fd8259ff700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd820108180 0x7fd82019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:09.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.639+0000 7fd8259ff700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820073550 0x7fd8201982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:09.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.639+0000 7fd8259ff700 1 --2- 172.21.15.93:0/3792473881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd820074b30 0x7fd820198820 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:09.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.639+0000 7fd8259ff700 1 -- 172.21.15.93:0/3792473881 >> 172.21.15.93:0/3792473881 conn(0x7fd8200fb8a0 msgr2=0x7fd8200fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:09.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.639+0000 7fd8259ff700 1 -- 172.21.15.93:0/3792473881 shutdown_connections 2023-11-16T16:47:09.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.639+0000 7fd8259ff700 1 -- 172.21.15.93:0/3792473881 wait complete. 2023-11-16T16:47:09.667 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-11-16T16:47:09.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.869+0000 7f70a7bff700 1 -- 172.21.15.93:0/1763330809 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a01002d0 msgr2=0x7f70a01006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:09.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.869+0000 7f70a7bff700 1 --2- 172.21.15.93:0/1763330809 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a01002d0 0x7f70a01006e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f709c009990 tx=0x7f709c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:09.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.869+0000 7f70a7bff700 1 -- 172.21.15.93:0/1763330809 shutdown_connections 2023-11-16T16:47:09.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.869+0000 7f70a7bff700 1 --2- 172.21.15.93:0/1763330809 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a01002d0 0x7f70a01006e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:09.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.869+0000 7f70a7bff700 1 --2- 172.21.15.93:0/1763330809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70a0101e50 0x7f70a0108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:09.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.869+0000 7f70a7bff700 1 --2- 172.21.15.93:0/1763330809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70a01014c0 0x7f70a0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:09.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.869+0000 7f70a7bff700 1 -- 172.21.15.93:0/1763330809 >> 172.21.15.93:0/1763330809 conn(0x7f70a00fb8b0 msgr2=0x7f70a00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:09.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.870+0000 7f70a7bff700 1 -- 172.21.15.93:0/1763330809 shutdown_connections 2023-11-16T16:47:09.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.870+0000 7f70a7bff700 1 -- 172.21.15.93:0/1763330809 wait complete. 2023-11-16T16:47:09.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a7bff700 1 Processor -- start 2023-11-16T16:47:09.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a7bff700 1 -- start start 2023-11-16T16:47:09.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a7bff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70a01002d0 0x7f70a01960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:09.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a7bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a01014c0 0x7f70a0196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:09.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a7bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70a0101e50 0x7f70a019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:09.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a7bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a019dc40 con 0x7f70a01014c0 2023-11-16T16:47:09.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a7bff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a019dd80 con 0x7f70a01002d0 2023-11-16T16:47:09.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a7bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a019df10 con 0x7f70a0101e50 2023-11-16T16:47:09.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a519a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a01014c0 0x7f70a0196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:09.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a519a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a01014c0 0x7f70a0196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42414/0 (socket says 172.21.15.93:42414) 2023-11-16T16:47:09.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a519a700 1 -- 172.21.15.93:0/2776367765 learned_addr learned my addr 172.21.15.93:0/2776367765 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:09.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a599b700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70a01002d0 0x7f70a01960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:09.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.871+0000 7f70a619c700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70a0101e50 0x7f70a019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:09.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.872+0000 7f70a519a700 1 -- 172.21.15.93:0/2776367765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70a0101e50 msgr2=0x7f70a019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:09.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.872+0000 7f70a519a700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70a0101e50 0x7f70a019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:09.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.872+0000 7f70a519a700 1 -- 172.21.15.93:0/2776367765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70a01002d0 msgr2=0x7f70a01960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:09.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.872+0000 7f70a519a700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70a01002d0 0x7f70a01960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:09.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.872+0000 7f70a519a700 1 -- 172.21.15.93:0/2776367765 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0095f0 con 0x7f70a01014c0 2023-11-16T16:47:09.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.872+0000 7f70a599b700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70a01002d0 0x7f70a01960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:09.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.872+0000 7f70a519a700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a01014c0 0x7f70a0196600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f709000e9a0 tx=0x7f709000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:09.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.872+0000 7f7096ffd700 1 -- 172.21.15.93:0/2776367765 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090009790 con 0x7f70a01014c0 2023-11-16T16:47:09.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.872+0000 7f7096ffd700 1 -- 172.21.15.93:0/2776367765 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7090010430 con 0x7f70a01014c0 2023-11-16T16:47:09.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.872+0000 7f70a7bff700 1 -- 172.21.15.93:0/2776367765 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a010bca0 con 0x7f70a01014c0 2023-11-16T16:47:09.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.872+0000 7f7096ffd700 1 -- 172.21.15.93:0/2776367765 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090018680 con 0x7f70a01014c0 2023-11-16T16:47:09.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.872+0000 7f70a7bff700 1 -- 172.21.15.93:0/2776367765 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a010bb10 con 0x7f70a01014c0 2023-11-16T16:47:09.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.874+0000 7f70a7bff700 1 -- 172.21.15.93:0/2776367765 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a0107200 con 0x7f70a01014c0 2023-11-16T16:47:09.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.875+0000 7f7096ffd700 1 -- 172.21.15.93:0/2776367765 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f70900105a0 con 0x7f70a01014c0 2023-11-16T16:47:09.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.877+0000 7f7096ffd700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f708c0a11d0 0x7f708c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:09.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.877+0000 7f7096ffd700 1 -- 172.21.15.93:0/2776367765 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f7090014070 con 0x7f70a01014c0 2023-11-16T16:47:09.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.877+0000 7f70a599b700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f708c0a11d0 0x7f708c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:09.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.877+0000 7f70a599b700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f708c0a11d0 0x7f708c0a3680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b2e0 tx=0x7f709c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:09.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:09.880+0000 7f7096ffd700 1 -- 172.21.15.93:0/2776367765 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7090051730 con 0x7f70a01014c0 2023-11-16T16:47:09.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:09 smithi093 ceph-mon[97863]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:09.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:09 smithi093 ceph-mon[97863]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-16T16:47:09.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:09 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2696926243' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-16T16:47:09.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:09 smithi093 ceph-mon[97863]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-16T16:47:09.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:09 smithi093 ceph-mon[97863]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:47:09.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:09 smithi093 ceph-mon[97863]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-11-16T16:47:09.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:09 smithi093 ceph-mon[97863]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-11-16T16:47:09.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:09 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3792473881' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-16T16:47:09.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:09 smithi159 ceph-mon[131528]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:09.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:09 smithi159 ceph-mon[131528]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-16T16:47:09.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:09 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2696926243' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-16T16:47:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:09 smithi159 ceph-mon[131528]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-16T16:47:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:09 smithi159 ceph-mon[131528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:47:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:09 smithi159 ceph-mon[131528]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-11-16T16:47:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:09 smithi159 ceph-mon[131528]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-11-16T16:47:09.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:09 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3792473881' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-16T16:47:10.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:09 smithi125 ceph-mon[131076]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:47:10.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:09 smithi125 ceph-mon[131076]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-16T16:47:10.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:09 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2696926243' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-16T16:47:10.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:09 smithi125 ceph-mon[131076]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-16T16:47:10.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:09 smithi125 ceph-mon[131076]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:47:10.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:09 smithi125 ceph-mon[131076]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-11-16T16:47:10.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:09 smithi125 ceph-mon[131076]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-11-16T16:47:10.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:09 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3792473881' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-16T16:47:10.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.126+0000 7f70a7bff700 1 -- 172.21.15.93:0/2776367765 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f70a002cfe0 con 0x7f70a01014c0 2023-11-16T16:47:10.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.315+0000 7f7096ffd700 1 -- 172.21.15.93:0/2776367765 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f70900961d0 con 0x7f70a01014c0 2023-11-16T16:47:10.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.318+0000 7f70a7bff700 1 -- 172.21.15.93:0/2776367765 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f708c0a11d0 msgr2=0x7f708c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:10.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.318+0000 7f70a7bff700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f708c0a11d0 0x7f708c0a3680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b2e0 tx=0x7f709c005fb0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.318+0000 7f70a7bff700 1 -- 172.21.15.93:0/2776367765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a01014c0 msgr2=0x7f70a0196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:10.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.318+0000 7f70a7bff700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a01014c0 0x7f70a0196600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f709000e9a0 tx=0x7f709000ecb0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.318+0000 7f70a7bff700 1 -- 172.21.15.93:0/2776367765 shutdown_connections 2023-11-16T16:47:10.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.318+0000 7f70a7bff700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f708c0a11d0 0x7f708c0a3680 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.318+0000 7f70a7bff700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70a0101e50 0x7f70a019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.318+0000 7f70a7bff700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70a01002d0 0x7f70a01960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.318+0000 7f70a7bff700 1 --2- 172.21.15.93:0/2776367765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a01014c0 0x7f70a0196600 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.318+0000 7f70a7bff700 1 -- 172.21.15.93:0/2776367765 >> 172.21.15.93:0/2776367765 conn(0x7f70a00fb8b0 msgr2=0x7f70a00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:10.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.318+0000 7f70a7bff700 1 -- 172.21.15.93:0/2776367765 shutdown_connections 2023-11-16T16:47:10.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.318+0000 7f70a7bff700 1 -- 172.21.15.93:0/2776367765 wait complete. 2023-11-16T16:47:10.339 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2023-11-16T16:47:10.339 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-11-16T16:47:10.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.539+0000 7f1cced9d700 1 -- 172.21.15.93:0/2684180294 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cc8101cc0 msgr2=0x7f1cc8108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:10.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.539+0000 7f1cced9d700 1 --2- 172.21.15.93:0/2684180294 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cc8101cc0 0x7f1cc8108560 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009990 tx=0x7f1cc0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.539+0000 7f1cced9d700 1 -- 172.21.15.93:0/2684180294 shutdown_connections 2023-11-16T16:47:10.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.539+0000 7f1cced9d700 1 --2- 172.21.15.93:0/2684180294 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cc8101cc0 0x7f1cc8108560 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.539+0000 7f1cced9d700 1 --2- 172.21.15.93:0/2684180294 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1cc8100080 0x7f1cc8100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.539+0000 7f1cced9d700 1 --2- 172.21.15.93:0/2684180294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cc8101310 0x7f1cc8101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.539+0000 7f1cced9d700 1 -- 172.21.15.93:0/2684180294 >> 172.21.15.93:0/2684180294 conn(0x7f1cc80fb6a0 msgr2=0x7f1cc80fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:10.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.539+0000 7f1cced9d700 1 -- 172.21.15.93:0/2684180294 shutdown_connections 2023-11-16T16:47:10.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.539+0000 7f1cced9d700 1 -- 172.21.15.93:0/2684180294 wait complete. 2023-11-16T16:47:10.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.540+0000 7f1cced9d700 1 Processor -- start 2023-11-16T16:47:10.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.541+0000 7f1cced9d700 1 -- start start 2023-11-16T16:47:10.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.541+0000 7f1cced9d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1cc8100080 0x7f1cc8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:10.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.541+0000 7f1cced9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cc8101310 0x7f1cc81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:10.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.541+0000 7f1cced9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cc8101cc0 0x7f1cc819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:10.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.541+0000 7f1cced9d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc819fae0 con 0x7f1cc8101cc0 2023-11-16T16:47:10.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.541+0000 7f1cced9d700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc819fc20 con 0x7f1cc8100080 2023-11-16T16:47:10.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.541+0000 7f1cced9d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc819fdb0 con 0x7f1cc8101310 2023-11-16T16:47:10.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.541+0000 7f1cce59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cc8101cc0 0x7f1cc819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:10.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.541+0000 7f1cce59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cc8101cc0 0x7f1cc819f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42434/0 (socket says 172.21.15.93:42434) 2023-11-16T16:47:10.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.541+0000 7f1cce59c700 1 -- 172.21.15.93:0/2933931612 learned_addr learned my addr 172.21.15.93:0/2933931612 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:10.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.541+0000 7f1ccdd9b700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1cc8100080 0x7f1cc8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:10.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.541+0000 7f1ccd59a700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cc8101310 0x7f1cc81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:10.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.542+0000 7f1cce59c700 1 -- 172.21.15.93:0/2933931612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cc8101310 msgr2=0x7f1cc81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:10.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.542+0000 7f1cce59c700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cc8101310 0x7f1cc81984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.542+0000 7f1cce59c700 1 -- 172.21.15.93:0/2933931612 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1cc8100080 msgr2=0x7f1cc8197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:10.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.542+0000 7f1cce59c700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1cc8100080 0x7f1cc8197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.542+0000 7f1cce59c700 1 -- 172.21.15.93:0/2933931612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc00095f0 con 0x7f1cc8101cc0 2023-11-16T16:47:10.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.542+0000 7f1ccdd9b700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1cc8100080 0x7f1cc8197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:10.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.542+0000 7f1cce59c700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cc8101cc0 0x7f1cc819f510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0006010 tx=0x7f1cc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:10.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.542+0000 7f1cbeffd700 1 -- 172.21.15.93:0/2933931612 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc001d070 con 0x7f1cc8101cc0 2023-11-16T16:47:10.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.542+0000 7f1cbeffd700 1 -- 172.21.15.93:0/2933931612 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cc0022470 con 0x7f1cc8101cc0 2023-11-16T16:47:10.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.542+0000 7f1cced9d700 1 -- 172.21.15.93:0/2933931612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc810ba80 con 0x7f1cc8101cc0 2023-11-16T16:47:10.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.542+0000 7f1cbeffd700 1 -- 172.21.15.93:0/2933931612 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0017630 con 0x7f1cc8101cc0 2023-11-16T16:47:10.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.542+0000 7f1cced9d700 1 -- 172.21.15.93:0/2933931612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc810b8f0 con 0x7f1cc8101cc0 2023-11-16T16:47:10.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.543+0000 7f1cced9d700 1 -- 172.21.15.93:0/2933931612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc8067980 con 0x7f1cc8101cc0 2023-11-16T16:47:10.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.545+0000 7f1cbeffd700 1 -- 172.21.15.93:0/2933931612 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1cc00225e0 con 0x7f1cc8101cc0 2023-11-16T16:47:10.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.547+0000 7f1cbeffd700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1cb40a1150 0x7f1cb40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:10.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.547+0000 7f1cbeffd700 1 -- 172.21.15.93:0/2933931612 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1cc00cdff0 con 0x7f1cc8101cc0 2023-11-16T16:47:10.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.548+0000 7f1ccdd9b700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1cb40a1150 0x7f1cb40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:10.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.548+0000 7f1ccdd9b700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1cb40a1150 0x7f1cb40a3600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40093a0 tx=0x7f1cc400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:10.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.552+0000 7f1cbeffd700 1 -- 172.21.15.93:0/2933931612 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1cc005c730 con 0x7f1cc8101cc0 2023-11-16T16:47:10.577 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-b[156540]: 2023-11-16T16:47:10.156+0000 7fb7a87ca700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-16T16:47:10.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.783+0000 7f1cced9d700 1 -- 172.21.15.93:0/2933931612 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cc8107b70 con 0x7f1cb40a1150 2023-11-16T16:47:10.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.830+0000 7f1cbeffd700 1 -- 172.21.15.93:0/2933931612 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1cc8107b70 con 0x7f1cb40a1150 2023-11-16T16:47:10.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.833+0000 7f1cced9d700 1 -- 172.21.15.93:0/2933931612 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1cb40a1150 msgr2=0x7f1cb40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:10.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.833+0000 7f1cced9d700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1cb40a1150 0x7f1cb40a3600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40093a0 tx=0x7f1cc400b040 comp rx=0 tx=0).stop 2023-11-16T16:47:10.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.833+0000 7f1cced9d700 1 -- 172.21.15.93:0/2933931612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cc8101cc0 msgr2=0x7f1cc819f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:10.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.833+0000 7f1cced9d700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cc8101cc0 0x7f1cc819f510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0006010 tx=0x7f1cc0004730 comp rx=0 tx=0).stop 2023-11-16T16:47:10.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.833+0000 7f1cced9d700 1 -- 172.21.15.93:0/2933931612 shutdown_connections 2023-11-16T16:47:10.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.833+0000 7f1cced9d700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1cb40a1150 0x7f1cb40a3600 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.833+0000 7f1cced9d700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cc8101cc0 0x7f1cc819f510 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.833+0000 7f1cced9d700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1cc8100080 0x7f1cc8197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.833+0000 7f1cced9d700 1 --2- 172.21.15.93:0/2933931612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cc8101310 0x7f1cc81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:10.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.833+0000 7f1cced9d700 1 -- 172.21.15.93:0/2933931612 >> 172.21.15.93:0/2933931612 conn(0x7f1cc80fb6a0 msgr2=0x7f1cc80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:10.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.833+0000 7f1cced9d700 1 -- 172.21.15.93:0/2933931612 shutdown_connections 2023-11-16T16:47:10.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:10.833+0000 7f1cced9d700 1 -- 172.21.15.93:0/2933931612 wait complete. 2023-11-16T16:47:10.853 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-11-16T16:47:10.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:47:10.790+0000 7f18740c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:47:10.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:47:10.790+0000 7f18740c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:47:10.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:47:10.790+0000 7f18740c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:47:10.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:47:10.790+0000 7f18740c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:47:10.900 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:47:10.790+0000 7f18740c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:47:10.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] up:active 2023-11-16T16:47:10.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3792473881' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-16T16:47:10.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-mon[97863]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-16T16:47:10.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-mon[97863]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-16T16:47:10.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-mon[97863]: Cluster is now healthy 2023-11-16T16:47:10.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-mon[97863]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-11-16T16:47:10.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-mon[97863]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-11-16T16:47:10.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-mon[97863]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-11-16T16:47:10.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2776367765' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-16T16:47:10.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] up:active 2023-11-16T16:47:10.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2776367765' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-16T16:47:10.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:10 smithi093 ceph-mon[97863]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:47:10.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:10 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] up:active 2023-11-16T16:47:10.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:10 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3792473881' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-16T16:47:10.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:10 smithi159 ceph-mon[131528]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-16T16:47:10.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:10 smithi159 ceph-mon[131528]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-16T16:47:10.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:10 smithi159 ceph-mon[131528]: Cluster is now healthy 2023-11-16T16:47:10.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:10 smithi159 ceph-mon[131528]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-11-16T16:47:10.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:10 smithi159 ceph-mon[131528]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-11-16T16:47:10.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:10 smithi159 ceph-mon[131528]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-11-16T16:47:10.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:10 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2776367765' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-16T16:47:10.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:10 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] up:active 2023-11-16T16:47:10.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:10 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2776367765' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-16T16:47:10.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:10 smithi159 ceph-mon[131528]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:47:11.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.052+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/2580229028 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7b4100d40 msgr2=0x7fd7b4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:11.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.052+0000 7fd7ba7ff700 1 --2- 172.21.15.93:0/2580229028 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7b4100d40 0x7fd7b4103120 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009990 tx=0x7fd7a4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.052+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/2580229028 shutdown_connections 2023-11-16T16:47:11.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.052+0000 7fd7ba7ff700 1 --2- 172.21.15.93:0/2580229028 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7b4100d40 0x7fd7b4103120 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.053+0000 7fd7ba7ff700 1 --2- 172.21.15.93:0/2580229028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7b4105f80 0x7fd7b41083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.053+0000 7fd7ba7ff700 1 --2- 172.21.15.93:0/2580229028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7b4103660 0x7fd7b4105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.053+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/2580229028 >> 172.21.15.93:0/2580229028 conn(0x7fd7b40fa7c0 msgr2=0x7fd7b40fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:11.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.053+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/2580229028 shutdown_connections 2023-11-16T16:47:11.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.053+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/2580229028 wait complete. 2023-11-16T16:47:11.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.054+0000 7fd7ba7ff700 1 Processor -- start 2023-11-16T16:47:11.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.054+0000 7fd7ba7ff700 1 -- start start 2023-11-16T16:47:11.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7ba7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7b4100d40 0x7fd7b4196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:11.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7ba7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7b4103660 0x7fd7b4196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7ba7ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7b4105f80 0x7fd7b419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7ba7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b419db90 con 0x7fd7b4100d40 2023-11-16T16:47:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7ba7ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b419dcd0 con 0x7fd7b4105f80 2023-11-16T16:47:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7ba7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b419de60 con 0x7fd7b4103660 2023-11-16T16:47:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7b3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7b4100d40 0x7fd7b4196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7b3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7b4100d40 0x7fd7b4196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42468/0 (socket says 172.21.15.93:42468) 2023-11-16T16:47:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7b3fff700 1 -- 172.21.15.93:0/1658075227 learned_addr learned my addr 172.21.15.93:0/1658075227 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7b37fe700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7b4103660 0x7fd7b4196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7b8d9c700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7b4105f80 0x7fd7b419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7b3fff700 1 -- 172.21.15.93:0/1658075227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7b4103660 msgr2=0x7fd7b4196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7b3fff700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7b4103660 0x7fd7b4196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7b3fff700 1 -- 172.21.15.93:0/1658075227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7b4105f80 msgr2=0x7fd7b419d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7b3fff700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7b4105f80 0x7fd7b419d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7b3fff700 1 -- 172.21.15.93:0/1658075227 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a40095f0 con 0x7fd7b4100d40 2023-11-16T16:47:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.055+0000 7fd7b8d9c700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7b4105f80 0x7fd7b419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.056+0000 7fd7b3fff700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7b4100d40 0x7fd7b4196010 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400b680 tx=0x7fd7a4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.056+0000 7fd7b17fa700 1 -- 172.21.15.93:0/1658075227 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a401d070 con 0x7fd7b4100d40 2023-11-16T16:47:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.056+0000 7fd7b17fa700 1 -- 172.21.15.93:0/1658075227 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7a4022470 con 0x7fd7b4100d40 2023-11-16T16:47:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.056+0000 7fd7b17fa700 1 -- 172.21.15.93:0/1658075227 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4017650 con 0x7fd7b4100d40 2023-11-16T16:47:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.056+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/1658075227 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b410bc70 con 0x7fd7b4100d40 2023-11-16T16:47:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.056+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/1658075227 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b410bae0 con 0x7fd7b4100d40 2023-11-16T16:47:11.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.057+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/1658075227 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b40fc330 con 0x7fd7b4100d40 2023-11-16T16:47:11.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.059+0000 7fd7b17fa700 1 -- 172.21.15.93:0/1658075227 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd7a40225e0 con 0x7fd7b4100d40 2023-11-16T16:47:11.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.060+0000 7fd7b17fa700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd79c0a1150 0x7fd79c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:11.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.060+0000 7fd7b17fa700 1 -- 172.21.15.93:0/1658075227 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd7a40ce190 con 0x7fd7b4100d40 2023-11-16T16:47:11.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.060+0000 7fd7b37fe700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd79c0a1150 0x7fd79c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:11.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.061+0000 7fd7b37fe700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd79c0a1150 0x7fd79c0a3600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009c60 tx=0x7fd7a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:11.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.064+0000 7fd7b17fa700 1 -- 172.21.15.93:0/1658075227 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd7a4097610 con 0x7fd7b4100d40 2023-11-16T16:47:11.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] up:active 2023-11-16T16:47:11.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3792473881' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-16T16:47:11.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-mon[131076]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-16T16:47:11.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-mon[131076]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-16T16:47:11.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-mon[131076]: Cluster is now healthy 2023-11-16T16:47:11.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-mon[131076]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-11-16T16:47:11.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-mon[131076]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-11-16T16:47:11.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-mon[131076]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-11-16T16:47:11.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2776367765' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-16T16:47:11.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] up:active 2023-11-16T16:47:11.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2776367765' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-16T16:47:11.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:10 smithi125 ceph-mon[131076]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:47:11.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.294+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/1658075227 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7b4061a80 con 0x7fd79c0a1150 2023-11-16T16:47:11.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.298+0000 7fd7b17fa700 1 -- 172.21.15.93:0/1658075227 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fd7b4061a80 con 0x7fd79c0a1150 2023-11-16T16:47:11.300 INFO:teuthology.orchestra.run.smithi093.stdout:/volumes/_nogroup/sv_0/95948205-761c-4d9c-b074-12c79d3e9e15 2023-11-16T16:47:11.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.301+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/1658075227 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd79c0a1150 msgr2=0x7fd79c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:11.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.301+0000 7fd7ba7ff700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd79c0a1150 0x7fd79c0a3600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009c60 tx=0x7fd7a8009260 comp rx=0 tx=0).stop 2023-11-16T16:47:11.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.301+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/1658075227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7b4100d40 msgr2=0x7fd7b4196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:11.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.301+0000 7fd7ba7ff700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7b4100d40 0x7fd7b4196010 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400b680 tx=0x7fd7a4005740 comp rx=0 tx=0).stop 2023-11-16T16:47:11.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.301+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/1658075227 shutdown_connections 2023-11-16T16:47:11.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.301+0000 7fd7ba7ff700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd79c0a1150 0x7fd79c0a3600 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.301+0000 7fd7ba7ff700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7b4100d40 0x7fd7b4196010 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.301+0000 7fd7ba7ff700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7b4105f80 0x7fd7b419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.301+0000 7fd7ba7ff700 1 --2- 172.21.15.93:0/1658075227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7b4103660 0x7fd7b4196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.301+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/1658075227 >> 172.21.15.93:0/1658075227 conn(0x7fd7b40fa7c0 msgr2=0x7fd7b4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:11.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.301+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/1658075227 shutdown_connections 2023-11-16T16:47:11.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.301+0000 7fd7ba7ff700 1 -- 172.21.15.93:0/1658075227 wait complete. 2023-11-16T16:47:11.321 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-11-16T16:47:11.523 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.522+0000 7f8b071ff700 1 -- 172.21.15.93:0/3546104334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b001014b0 msgr2=0x7f8b00101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:11.523 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.522+0000 7f8b071ff700 1 --2- 172.21.15.93:0/3546104334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b001014b0 0x7f8b00101900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009990 tx=0x7f8aec009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.522+0000 7f8b071ff700 1 -- 172.21.15.93:0/3546104334 shutdown_connections 2023-11-16T16:47:11.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.522+0000 7f8b071ff700 1 --2- 172.21.15.93:0/3546104334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b00101e40 0x7f8b00108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.522+0000 7f8b071ff700 1 --2- 172.21.15.93:0/3546104334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b001002c0 0x7f8b001006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.522+0000 7f8b071ff700 1 --2- 172.21.15.93:0/3546104334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b001014b0 0x7f8b00101900 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.522+0000 7f8b071ff700 1 -- 172.21.15.93:0/3546104334 >> 172.21.15.93:0/3546104334 conn(0x7f8b000fb8a0 msgr2=0x7f8b000fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:11.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.522+0000 7f8b071ff700 1 -- 172.21.15.93:0/3546104334 shutdown_connections 2023-11-16T16:47:11.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.522+0000 7f8b071ff700 1 -- 172.21.15.93:0/3546104334 wait complete. 2023-11-16T16:47:11.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.523+0000 7f8b071ff700 1 Processor -- start 2023-11-16T16:47:11.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.523+0000 7f8b071ff700 1 -- start start 2023-11-16T16:47:11.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.523+0000 7f8b071ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b001002c0 0x7f8b000748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:11.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.523+0000 7f8b071ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b001014b0 0x7f8b00072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:11.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.523+0000 7f8b071ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b00101e40 0x7f8b00073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:11.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.523+0000 7f8b071ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b00073bc0 con 0x7f8b001014b0 2023-11-16T16:47:11.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.523+0000 7f8b071ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b00073d30 con 0x7f8b00101e40 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.523+0000 7f8b071ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b001a00a0 con 0x7f8b001002c0 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8afffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b001014b0 0x7f8b00072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8b0579c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b00101e40 0x7f8b00073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8b04f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b001002c0 0x7f8b000748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8afffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b001014b0 0x7f8b00072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42488/0 (socket says 172.21.15.93:42488) 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8afffff700 1 -- 172.21.15.93:0/3307203131 learned_addr learned my addr 172.21.15.93:0/3307203131 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8afffff700 1 -- 172.21.15.93:0/3307203131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b001002c0 msgr2=0x7f8b000748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8afffff700 1 --2- 172.21.15.93:0/3307203131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b001002c0 0x7f8b000748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8afffff700 1 -- 172.21.15.93:0/3307203131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b00101e40 msgr2=0x7f8b00073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8afffff700 1 --2- 172.21.15.93:0/3307203131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b00101e40 0x7f8b00073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8afffff700 1 -- 172.21.15.93:0/3307203131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec0095f0 con 0x7f8b001014b0 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8afffff700 1 --2- 172.21.15.93:0/3307203131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b001014b0 0x7f8b00072ee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00b620 tx=0x7f8aec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:11.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8afdffb700 1 -- 172.21.15.93:0/3307203131 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec01d070 con 0x7f8b001014b0 2023-11-16T16:47:11.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8afdffb700 1 -- 172.21.15.93:0/3307203131 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aec022470 con 0x7f8b001014b0 2023-11-16T16:47:11.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8b071ff700 1 -- 172.21.15.93:0/3307203131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0010bc70 con 0x7f8b001014b0 2023-11-16T16:47:11.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.524+0000 7f8afdffb700 1 -- 172.21.15.93:0/3307203131 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec017650 con 0x7f8b001014b0 2023-11-16T16:47:11.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.525+0000 7f8b071ff700 1 -- 172.21.15.93:0/3307203131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0010bae0 con 0x7f8b001014b0 2023-11-16T16:47:11.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.526+0000 7f8b071ff700 1 -- 172.21.15.93:0/3307203131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b001071f0 con 0x7f8b001014b0 2023-11-16T16:47:11.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.528+0000 7f8afdffb700 1 -- 172.21.15.93:0/3307203131 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8aec0225e0 con 0x7f8b001014b0 2023-11-16T16:47:11.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.529+0000 7f8afdffb700 1 --2- 172.21.15.93:0/3307203131 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8af00a11d0 0x7f8af00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:11.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.529+0000 7f8afdffb700 1 -- 172.21.15.93:0/3307203131 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f8aec0ce060 con 0x7f8b001014b0 2023-11-16T16:47:11.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.529+0000 7f8b04f9b700 1 --2- 172.21.15.93:0/3307203131 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8af00a11d0 0x7f8af00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:11.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.530+0000 7f8b04f9b700 1 --2- 172.21.15.93:0/3307203131 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8af00a11d0 0x7f8af00a3680 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b00101310 tx=0x7f8af4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:11.534 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.532+0000 7f8afdffb700 1 -- 172.21.15.93:0/3307203131 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8aec0974e0 con 0x7f8b001014b0 2023-11-16T16:47:11.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.765+0000 7f8b071ff700 1 -- 172.21.15.93:0/3307203131 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b00061c10 con 0x7f8af00a11d0 2023-11-16T16:47:11.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.791+0000 7f8afdffb700 1 -- 172.21.15.93:0/3307203131 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8b00061c10 con 0x7f8af00a11d0 2023-11-16T16:47:11.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.793+0000 7f8b071ff700 1 -- 172.21.15.93:0/3307203131 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8af00a11d0 msgr2=0x7f8af00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:11.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.793+0000 7f8b071ff700 1 --2- 172.21.15.93:0/3307203131 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8af00a11d0 0x7f8af00a3680 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b00101310 tx=0x7f8af4009260 comp rx=0 tx=0).stop 2023-11-16T16:47:11.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.793+0000 7f8b071ff700 1 -- 172.21.15.93:0/3307203131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b001014b0 msgr2=0x7f8b00072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:11.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.793+0000 7f8b071ff700 1 --2- 172.21.15.93:0/3307203131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b001014b0 0x7f8b00072ee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00b620 tx=0x7f8aec005740 comp rx=0 tx=0).stop 2023-11-16T16:47:11.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.793+0000 7f8b071ff700 1 -- 172.21.15.93:0/3307203131 shutdown_connections 2023-11-16T16:47:11.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.793+0000 7f8b071ff700 1 --2- 172.21.15.93:0/3307203131 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8af00a11d0 0x7f8af00a3680 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.793+0000 7f8b071ff700 1 --2- 172.21.15.93:0/3307203131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b00101e40 0x7f8b00073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.793+0000 7f8b071ff700 1 --2- 172.21.15.93:0/3307203131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b001002c0 0x7f8b000748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.793+0000 7f8b071ff700 1 --2- 172.21.15.93:0/3307203131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b001014b0 0x7f8b00072ee0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:11.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.793+0000 7f8b071ff700 1 -- 172.21.15.93:0/3307203131 >> 172.21.15.93:0/3307203131 conn(0x7f8b000fb8a0 msgr2=0x7f8b000fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:11.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.793+0000 7f8b071ff700 1 -- 172.21.15.93:0/3307203131 shutdown_connections 2023-11-16T16:47:11.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:11.793+0000 7f8b071ff700 1 -- 172.21.15.93:0/3307203131 wait complete. 2023-11-16T16:47:11.814 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-11-16T16:47:11.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:11 smithi093 ceph-mon[97863]: pgmap v300: 129 pgs: 129 active+clean; 582 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 938 B/s wr, 2 op/s 2023-11-16T16:47:11.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:11 smithi093 ceph-mon[97863]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:47:11.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:11 smithi159 ceph-mon[131528]: pgmap v300: 129 pgs: 129 active+clean; 582 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 938 B/s wr, 2 op/s 2023-11-16T16:47:11.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:11 smithi159 ceph-mon[131528]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:47:12.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.013+0000 7f451d1ff700 1 -- 172.21.15.93:0/2789563660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45181002d0 msgr2=0x7f45181006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.013+0000 7f451d1ff700 1 --2- 172.21.15.93:0/2789563660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45181002d0 0x7f45181006e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4508009990 tx=0x7f4508009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.013+0000 7f451d1ff700 1 -- 172.21.15.93:0/2789563660 shutdown_connections 2023-11-16T16:47:12.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.013+0000 7f451d1ff700 1 --2- 172.21.15.93:0/2789563660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45181002d0 0x7f45181006e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.013+0000 7f451d1ff700 1 --2- 172.21.15.93:0/2789563660 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4518101e50 0x7f4518108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.013+0000 7f451d1ff700 1 --2- 172.21.15.93:0/2789563660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45181014c0 0x7f4518101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.013+0000 7f451d1ff700 1 -- 172.21.15.93:0/2789563660 >> 172.21.15.93:0/2789563660 conn(0x7f45180fb8b0 msgr2=0x7f45180fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:12.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.013+0000 7f451d1ff700 1 -- 172.21.15.93:0/2789563660 shutdown_connections 2023-11-16T16:47:12.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.014+0000 7f451d1ff700 1 -- 172.21.15.93:0/2789563660 wait complete. 2023-11-16T16:47:12.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.014+0000 7f451d1ff700 1 Processor -- start 2023-11-16T16:47:12.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.015+0000 7f451d1ff700 1 -- start start 2023-11-16T16:47:12.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.015+0000 7f451d1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45181002d0 0x7f45181982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:12.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.015+0000 7f451d1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45181014c0 0x7f4518198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:12.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.015+0000 7f451d1ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4518101e50 0x7f451819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:12.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.015+0000 7f451d1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451819fe50 con 0x7f45181002d0 2023-11-16T16:47:12.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.015+0000 7f451d1ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451819ff90 con 0x7f4518101e50 2023-11-16T16:47:12.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.015+0000 7f451d1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45181a0120 con 0x7f45181014c0 2023-11-16T16:47:12.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.015+0000 7f4516d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45181002d0 0x7f45181982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:12.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.015+0000 7f4516d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45181002d0 0x7f45181982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42506/0 (socket says 172.21.15.93:42506) 2023-11-16T16:47:12.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.015+0000 7f4516d9d700 1 -- 172.21.15.93:0/65696381 learned_addr learned my addr 172.21.15.93:0/65696381 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:12.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.015+0000 7f451659c700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45181014c0 0x7f4518198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:12.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.015+0000 7f451759e700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4518101e50 0x7f451819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:12.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.016+0000 7f4516d9d700 1 -- 172.21.15.93:0/65696381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45181014c0 msgr2=0x7f4518198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.016+0000 7f4516d9d700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45181014c0 0x7f4518198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.016+0000 7f4516d9d700 1 -- 172.21.15.93:0/65696381 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4518101e50 msgr2=0x7f451819f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.016+0000 7f4516d9d700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4518101e50 0x7f451819f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.016+0000 7f4516d9d700 1 -- 172.21.15.93:0/65696381 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45080095f0 con 0x7f45181002d0 2023-11-16T16:47:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.016+0000 7f4516d9d700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45181002d0 0x7f45181982d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4508004730 tx=0x7f4508009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.016+0000 7f44fffff700 1 -- 172.21.15.93:0/65696381 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450801d070 con 0x7f45181002d0 2023-11-16T16:47:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.016+0000 7f44fffff700 1 -- 172.21.15.93:0/65696381 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4508004cd0 con 0x7f45181002d0 2023-11-16T16:47:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.016+0000 7f451d1ff700 1 -- 172.21.15.93:0/65696381 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451810bca0 con 0x7f45181002d0 2023-11-16T16:47:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.016+0000 7f44fffff700 1 -- 172.21.15.93:0/65696381 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508005530 con 0x7f45181002d0 2023-11-16T16:47:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.016+0000 7f451d1ff700 1 -- 172.21.15.93:0/65696381 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451810bb10 con 0x7f45181002d0 2023-11-16T16:47:12.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.017+0000 7f451d1ff700 1 -- 172.21.15.93:0/65696381 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4518192a00 con 0x7f45181002d0 2023-11-16T16:47:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.019+0000 7f44fffff700 1 -- 172.21.15.93:0/65696381 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f450800fbc0 con 0x7f45181002d0 2023-11-16T16:47:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.020+0000 7f44fffff700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f45000a11d0 0x7f45000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.021+0000 7f44fffff700 1 -- 172.21.15.93:0/65696381 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f45080cdf30 con 0x7f45181002d0 2023-11-16T16:47:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.021+0000 7f451659c700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f45000a11d0 0x7f45000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.021+0000 7f451659c700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f45000a11d0 0x7f45000a3680 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f450c009bf0 tx=0x7f450c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:12.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.024+0000 7f44fffff700 1 -- 172.21.15.93:0/65696381 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f450805a730 con 0x7f45181002d0 2023-11-16T16:47:12.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:11 smithi125 ceph-mon[131076]: pgmap v300: 129 pgs: 129 active+clean; 582 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 938 B/s wr, 2 op/s 2023-11-16T16:47:12.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:11 smithi125 ceph-mon[131076]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:47:12.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.259+0000 7f451d1ff700 1 -- 172.21.15.93:0/65696381 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4518061c40 con 0x7f45000a11d0 2023-11-16T16:47:12.264 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.263+0000 7f44fffff700 1 -- 172.21.15.93:0/65696381 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f4518061c40 con 0x7f45000a11d0 2023-11-16T16:47:12.264 INFO:teuthology.orchestra.run.smithi093.stdout:/volumes/_nogroup/sv_1/a7076a0f-3e60-4d59-9a3c-534477d252be 2023-11-16T16:47:12.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.265+0000 7f451d1ff700 1 -- 172.21.15.93:0/65696381 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f45000a11d0 msgr2=0x7f45000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.265+0000 7f451d1ff700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f45000a11d0 0x7f45000a3680 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f450c009bf0 tx=0x7f450c009260 comp rx=0 tx=0).stop 2023-11-16T16:47:12.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.265+0000 7f451d1ff700 1 -- 172.21.15.93:0/65696381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45181002d0 msgr2=0x7f45181982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.265+0000 7f451d1ff700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45181002d0 0x7f45181982d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4508004730 tx=0x7f4508009510 comp rx=0 tx=0).stop 2023-11-16T16:47:12.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.265+0000 7f451d1ff700 1 -- 172.21.15.93:0/65696381 shutdown_connections 2023-11-16T16:47:12.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.265+0000 7f451d1ff700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f45000a11d0 0x7f45000a3680 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.265+0000 7f451d1ff700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45181002d0 0x7f45181982d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.268 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.265+0000 7f451d1ff700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4518101e50 0x7f451819f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.268 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.265+0000 7f451d1ff700 1 --2- 172.21.15.93:0/65696381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45181014c0 0x7f4518198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.268 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.265+0000 7f451d1ff700 1 -- 172.21.15.93:0/65696381 >> 172.21.15.93:0/65696381 conn(0x7f45180fb8b0 msgr2=0x7f4518106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:12.268 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.265+0000 7f451d1ff700 1 -- 172.21.15.93:0/65696381 shutdown_connections 2023-11-16T16:47:12.268 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.265+0000 7f451d1ff700 1 -- 172.21.15.93:0/65696381 wait complete. 2023-11-16T16:47:12.286 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:47:12.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.481+0000 7f960e1ff700 1 -- 172.21.15.93:0/597325045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96081002b0 msgr2=0x7f96081006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.481+0000 7f960e1ff700 1 --2- 172.21.15.93:0/597325045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96081002b0 0x7f96081006c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009990 tx=0x7f95f8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.481+0000 7f960e1ff700 1 -- 172.21.15.93:0/597325045 shutdown_connections 2023-11-16T16:47:12.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.481+0000 7f960e1ff700 1 --2- 172.21.15.93:0/597325045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96081002b0 0x7f96081006c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.481+0000 7f960e1ff700 1 --2- 172.21.15.93:0/597325045 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9608101e30 0x7f96081086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.481+0000 7f960e1ff700 1 --2- 172.21.15.93:0/597325045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96081014a0 0x7f96081018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.481+0000 7f960e1ff700 1 -- 172.21.15.93:0/597325045 >> 172.21.15.93:0/597325045 conn(0x7f96080fb890 msgr2=0x7f96080fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:12.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.481+0000 7f960e1ff700 1 -- 172.21.15.93:0/597325045 shutdown_connections 2023-11-16T16:47:12.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.482+0000 7f960e1ff700 1 -- 172.21.15.93:0/597325045 wait complete. 2023-11-16T16:47:12.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.482+0000 7f960e1ff700 1 Processor -- start 2023-11-16T16:47:12.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.483+0000 7f960e1ff700 1 -- start start 2023-11-16T16:47:12.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.483+0000 7f960e1ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96081002b0 0x7f96080748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:12.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.483+0000 7f960e1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96081014a0 0x7f9608072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:12.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.483+0000 7f960e1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9608101e30 0x7f9608073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:12.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.483+0000 7f960e1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9608073bc0 con 0x7f96081014a0 2023-11-16T16:47:12.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.483+0000 7f960e1ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9608073d30 con 0x7f96081002b0 2023-11-16T16:47:12.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.483+0000 7f960e1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960819ffd0 con 0x7f9608101e30 2023-11-16T16:47:12.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.483+0000 7f9606ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96081014a0 0x7f9608072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:12.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.483+0000 7f9606ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96081014a0 0x7f9608072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42522/0 (socket says 172.21.15.93:42522) 2023-11-16T16:47:12.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.483+0000 7f9606ffd700 1 -- 172.21.15.93:0/2849243515 learned_addr learned my addr 172.21.15.93:0/2849243515 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:12.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.483+0000 7f96077fe700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96081002b0 0x7f96080748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:12.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.483+0000 7f9607fff700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9608101e30 0x7f9608073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:12.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.484+0000 7f9606ffd700 1 -- 172.21.15.93:0/2849243515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9608101e30 msgr2=0x7f9608073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.484+0000 7f9606ffd700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9608101e30 0x7f9608073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.484+0000 7f9606ffd700 1 -- 172.21.15.93:0/2849243515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96081002b0 msgr2=0x7f96080748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.484+0000 7f9606ffd700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96081002b0 0x7f96080748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.484+0000 7f9606ffd700 1 -- 172.21.15.93:0/2849243515 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f80095f0 con 0x7f96081014a0 2023-11-16T16:47:12.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.484+0000 7f96077fe700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96081002b0 0x7f96080748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:12.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.484+0000 7f9606ffd700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96081014a0 0x7f9608072ee0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00d980 tx=0x7f95fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:12.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.484+0000 7f9604ff9700 1 -- 172.21.15.93:0/2849243515 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc004220 con 0x7f96081014a0 2023-11-16T16:47:12.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.484+0000 7f9604ff9700 1 -- 172.21.15.93:0/2849243515 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95fc007640 con 0x7f96081014a0 2023-11-16T16:47:12.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.484+0000 7f9604ff9700 1 -- 172.21.15.93:0/2849243515 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc005690 con 0x7f96081014a0 2023-11-16T16:47:12.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.484+0000 7f960e1ff700 1 -- 172.21.15.93:0/2849243515 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960810bc10 con 0x7f96081014a0 2023-11-16T16:47:12.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.484+0000 7f960e1ff700 1 -- 172.21.15.93:0/2849243515 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960810ba80 con 0x7f96081014a0 2023-11-16T16:47:12.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.485+0000 7f960e1ff700 1 -- 172.21.15.93:0/2849243515 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96081a0670 con 0x7f96081014a0 2023-11-16T16:47:12.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.487+0000 7f9604ff9700 1 -- 172.21.15.93:0/2849243515 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f95fc004750 con 0x7f96081014a0 2023-11-16T16:47:12.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.489+0000 7f9604ff9700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f95f00a1150 0x7f95f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:12.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.489+0000 7f9604ff9700 1 -- 172.21.15.93:0/2849243515 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f95fc0cca90 con 0x7f96081014a0 2023-11-16T16:47:12.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.489+0000 7f96077fe700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f95f00a1150 0x7f95f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:12.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.490+0000 7f96077fe700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f95f00a1150 0x7f95f00a3600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f95f800b2e0 tx=0x7f95f8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:12.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.492+0000 7f9604ff9700 1 -- 172.21.15.93:0/2849243515 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f95fc095e80 con 0x7f96081014a0 2023-11-16T16:47:12.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.657+0000 7f9604ff9700 1 -- 172.21.15.93:0/2849243515 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f95fc04fb40 con 0x7f96081014a0 2023-11-16T16:47:12.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.751+0000 7f960e1ff700 1 -- 172.21.15.93:0/2849243515 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9608107200 con 0x7f96081014a0 2023-11-16T16:47:12.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.751+0000 7f9604ff9700 1 -- 172.21.15.93:0/2849243515 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10811 (secure 0 0 0) 0x7f95fc0abc10 con 0x7f96081014a0 2023-11-16T16:47:12.753 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:47:12.753 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3614592201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3614592201},{"type":"v1","addr":"172.21.15.93:6839","nonce":3614592201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4068789362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4068789362},{"type":"v1","addr":"172.21.15.93:6841","nonce":4068789362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:46:37.124338+0000","modified":"2023-11-16T16:47:10.311966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24541,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/778982589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":778982589},{"type":"v1","addr":"172.21.15.159:6837","nonce":778982589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6839/149300707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":149300707},{"type":"v1","addr":"172.21.15.159:6839","nonce":149300707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/3590027328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":3590027328},{"type":"v1","addr":"172.21.15.125:6839","nonce":3590027328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:47:12.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.754+0000 7f960e1ff700 1 -- 172.21.15.93:0/2849243515 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f95f00a1150 msgr2=0x7f95f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.754+0000 7f960e1ff700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f95f00a1150 0x7f95f00a3600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f95f800b2e0 tx=0x7f95f8005960 comp rx=0 tx=0).stop 2023-11-16T16:47:12.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.754+0000 7f960e1ff700 1 -- 172.21.15.93:0/2849243515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96081014a0 msgr2=0x7f9608072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.754+0000 7f960e1ff700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96081014a0 0x7f9608072ee0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00d980 tx=0x7f95fc00dc90 comp rx=0 tx=0).stop 2023-11-16T16:47:12.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.754+0000 7f960e1ff700 1 -- 172.21.15.93:0/2849243515 shutdown_connections 2023-11-16T16:47:12.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.754+0000 7f960e1ff700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f95f00a1150 0x7f95f00a3600 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.754+0000 7f960e1ff700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9608101e30 0x7f9608073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.754+0000 7f960e1ff700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96081002b0 0x7f96080748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.754+0000 7f960e1ff700 1 --2- 172.21.15.93:0/2849243515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96081014a0 0x7f9608072ee0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.754+0000 7f960e1ff700 1 -- 172.21.15.93:0/2849243515 >> 172.21.15.93:0/2849243515 conn(0x7f96080fb890 msgr2=0x7f96080fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:12.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.754+0000 7f960e1ff700 1 -- 172.21.15.93:0/2849243515 shutdown_connections 2023-11-16T16:47:12.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.754+0000 7f960e1ff700 1 -- 172.21.15.93:0/2849243515 wait complete. 2023-11-16T16:47:12.757 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 23 2023-11-16T16:47:12.774 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:47:12.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:12 smithi093 ceph-mon[97863]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:47:12.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:12 smithi093 ceph-mon[97863]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:47:12.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.970+0000 7fd6545ff700 1 -- 172.21.15.93:0/3490831467 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd64c101480 msgr2=0x7fd64c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.970+0000 7fd6545ff700 1 --2- 172.21.15.93:0/3490831467 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd64c101480 0x7fd64c1018d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009990 tx=0x7fd63c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.970+0000 7fd6545ff700 1 -- 172.21.15.93:0/3490831467 shutdown_connections 2023-11-16T16:47:12.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.970+0000 7fd6545ff700 1 --2- 172.21.15.93:0/3490831467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd64c101e10 0x7fd64c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.970+0000 7fd6545ff700 1 --2- 172.21.15.93:0/3490831467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd64c100290 0x7fd64c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.970+0000 7fd6545ff700 1 --2- 172.21.15.93:0/3490831467 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd64c101480 0x7fd64c1018d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.970+0000 7fd6545ff700 1 -- 172.21.15.93:0/3490831467 >> 172.21.15.93:0/3490831467 conn(0x7fd64c0fb870 msgr2=0x7fd64c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:12.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.970+0000 7fd6545ff700 1 -- 172.21.15.93:0/3490831467 shutdown_connections 2023-11-16T16:47:12.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.970+0000 7fd6545ff700 1 -- 172.21.15.93:0/3490831467 wait complete. 2023-11-16T16:47:12.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd6545ff700 1 Processor -- start 2023-11-16T16:47:12.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd6545ff700 1 -- start start 2023-11-16T16:47:12.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd6545ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd64c100290 0x7fd64c195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:12.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd6545ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd64c101480 0x7fd64c196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:12.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd6545ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd64c101e10 0x7fd64c19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:12.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd6545ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c19db50 con 0x7fd64c101480 2023-11-16T16:47:12.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd6545ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c19dc90 con 0x7fd64c100290 2023-11-16T16:47:12.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd6545ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c19de20 con 0x7fd64c101e10 2023-11-16T16:47:12.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd651b9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd64c101480 0x7fd64c196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:12.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd65239b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd64c100290 0x7fd64c195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:12.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd65239b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd64c100290 0x7fd64c195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:48826/0 (socket says 172.21.15.93:48826) 2023-11-16T16:47:12.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd652b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd64c101e10 0x7fd64c19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:12.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.971+0000 7fd65239b700 1 -- 172.21.15.93:0/180988935 learned_addr learned my addr 172.21.15.93:0/180988935 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:12.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.972+0000 7fd65239b700 1 -- 172.21.15.93:0/180988935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd64c101e10 msgr2=0x7fd64c19d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.972+0000 7fd65239b700 1 --2- 172.21.15.93:0/180988935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd64c101e10 0x7fd64c19d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.972+0000 7fd65239b700 1 -- 172.21.15.93:0/180988935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd64c101480 msgr2=0x7fd64c196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:12.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.972+0000 7fd65239b700 1 --2- 172.21.15.93:0/180988935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd64c101480 0x7fd64c196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:12.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.972+0000 7fd65239b700 1 -- 172.21.15.93:0/180988935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0095f0 con 0x7fd64c100290 2023-11-16T16:47:12.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.972+0000 7fd65239b700 1 --2- 172.21.15.93:0/180988935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd64c100290 0x7fd64c195fd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd64800e970 tx=0x7fd64800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:12.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.972+0000 7fd6437fe700 1 -- 172.21.15.93:0/180988935 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648009790 con 0x7fd64c100290 2023-11-16T16:47:12.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.972+0000 7fd6545ff700 1 -- 172.21.15.93:0/180988935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c10bc50 con 0x7fd64c100290 2023-11-16T16:47:12.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.972+0000 7fd6437fe700 1 -- 172.21.15.93:0/180988935 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd64800c7e0 con 0x7fd64c100290 2023-11-16T16:47:12.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.972+0000 7fd6437fe700 1 -- 172.21.15.93:0/180988935 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648003c70 con 0x7fd64c100290 2023-11-16T16:47:12.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.972+0000 7fd6545ff700 1 -- 172.21.15.93:0/180988935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c10bac0 con 0x7fd64c100290 2023-11-16T16:47:12.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.973+0000 7fd6545ff700 1 -- 172.21.15.93:0/180988935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64c067a60 con 0x7fd64c100290 2023-11-16T16:47:12.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.975+0000 7fd6437fe700 1 -- 172.21.15.93:0/180988935 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd64800cc50 con 0x7fd64c100290 2023-11-16T16:47:12.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.977+0000 7fd6437fe700 1 --2- 172.21.15.93:0/180988935 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd6380a11e0 0x7fd6380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:12.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.977+0000 7fd6437fe700 1 -- 172.21.15.93:0/180988935 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd648014070 con 0x7fd64c100290 2023-11-16T16:47:12.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.977+0000 7fd651b9a700 1 --2- 172.21.15.93:0/180988935 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd6380a11e0 0x7fd6380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:12.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.977+0000 7fd651b9a700 1 --2- 172.21.15.93:0/180988935 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd6380a11e0 0x7fd6380a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00b2e0 tx=0x7fd63c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:12.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:12.979+0000 7fd6437fe700 1 -- 172.21.15.93:0/180988935 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd648096880 con 0x7fd64c100290 2023-11-16T16:47:12.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:12 smithi159 ceph-mon[131528]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:47:12.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:12 smithi159 ceph-mon[131528]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:47:13.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:12 smithi125 ceph-mon[131076]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:47:13.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:12 smithi125 ceph-mon[131076]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:47:13.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.180+0000 7fd6545ff700 1 -- 172.21.15.93:0/180988935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd64c1071c0 con 0x7fd64c100290 2023-11-16T16:47:13.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.182+0000 7fd6437fe700 1 -- 172.21.15.93:0/180988935 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24468 (secure 0 0 0) 0x7fd648003dd0 con 0x7fd64c100290 2023-11-16T16:47:13.184 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:47:13.184 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":90,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:46:52.315313+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:47:13.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.185+0000 7fd6545ff700 1 -- 172.21.15.93:0/180988935 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd6380a11e0 msgr2=0x7fd6380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:13.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.185+0000 7fd6545ff700 1 --2- 172.21.15.93:0/180988935 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd6380a11e0 0x7fd6380a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00b2e0 tx=0x7fd63c005fb0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.185+0000 7fd6545ff700 1 -- 172.21.15.93:0/180988935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd64c100290 msgr2=0x7fd64c195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:13.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.185+0000 7fd6545ff700 1 --2- 172.21.15.93:0/180988935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd64c100290 0x7fd64c195fd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd64800e970 tx=0x7fd64800ec80 comp rx=0 tx=0).stop 2023-11-16T16:47:13.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.185+0000 7fd6545ff700 1 -- 172.21.15.93:0/180988935 shutdown_connections 2023-11-16T16:47:13.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.185+0000 7fd6545ff700 1 --2- 172.21.15.93:0/180988935 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd6380a11e0 0x7fd6380a3690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.185+0000 7fd6545ff700 1 --2- 172.21.15.93:0/180988935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd64c101e10 0x7fd64c19d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.185+0000 7fd6545ff700 1 --2- 172.21.15.93:0/180988935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd64c100290 0x7fd64c195fd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.185+0000 7fd6545ff700 1 --2- 172.21.15.93:0/180988935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd64c101480 0x7fd64c196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.185+0000 7fd6545ff700 1 -- 172.21.15.93:0/180988935 >> 172.21.15.93:0/180988935 conn(0x7fd64c0fb870 msgr2=0x7fd64c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:13.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.185+0000 7fd6545ff700 1 -- 172.21.15.93:0/180988935 shutdown_connections 2023-11-16T16:47:13.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.185+0000 7fd6545ff700 1 -- 172.21.15.93:0/180988935 wait complete. 2023-11-16T16:47:13.206 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-16T16:47:13.206 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-16T16:47:13.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.402+0000 7fd8bcdff700 1 -- 172.21.15.93:0/3023539940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8b8100290 msgr2=0x7fd8b81006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.402+0000 7fd8bcdff700 1 --2- 172.21.15.93:0/3023539940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8b8100290 0x7fd8b81006a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009990 tx=0x7fd8a8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.402+0000 7fd8bcdff700 1 -- 172.21.15.93:0/3023539940 shutdown_connections 2023-11-16T16:47:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.402+0000 7fd8bcdff700 1 --2- 172.21.15.93:0/3023539940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8b8100290 0x7fd8b81006a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.402+0000 7fd8bcdff700 1 --2- 172.21.15.93:0/3023539940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8b8101e10 0x7fd8b81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.402+0000 7fd8bcdff700 1 --2- 172.21.15.93:0/3023539940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8b8101480 0x7fd8b81018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.402+0000 7fd8bcdff700 1 -- 172.21.15.93:0/3023539940 >> 172.21.15.93:0/3023539940 conn(0x7fd8b80fb870 msgr2=0x7fd8b80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.402+0000 7fd8bcdff700 1 -- 172.21.15.93:0/3023539940 shutdown_connections 2023-11-16T16:47:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.402+0000 7fd8bcdff700 1 -- 172.21.15.93:0/3023539940 wait complete. 2023-11-16T16:47:13.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.403+0000 7fd8bcdff700 1 Processor -- start 2023-11-16T16:47:13.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.403+0000 7fd8bcdff700 1 -- start start 2023-11-16T16:47:13.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.403+0000 7fd8bcdff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8b8100290 0x7fd8b80748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:13.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.403+0000 7fd8bcdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8b8101480 0x7fd8b8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:13.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.403+0000 7fd8bcdff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8b8101e10 0x7fd8b8073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:13.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.403+0000 7fd8bcdff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b8073bc0 con 0x7fd8b8101e10 2023-11-16T16:47:13.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.403+0000 7fd8bcdff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b8073d30 con 0x7fd8b8100290 2023-11-16T16:47:13.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.403+0000 7fd8bcdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b81a0010 con 0x7fd8b8101480 2023-11-16T16:47:13.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8b5d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8b8101480 0x7fd8b8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:13.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8b659c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8b8100290 0x7fd8b80748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:13.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8b6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8b8101e10 0x7fd8b8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:13.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8b659c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8b8100290 0x7fd8b80748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:48834/0 (socket says 172.21.15.93:48834) 2023-11-16T16:47:13.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8b5d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8b8101480 0x7fd8b8072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:36524/0 (socket says 172.21.15.93:36524) 2023-11-16T16:47:13.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8b659c700 1 -- 172.21.15.93:0/1048850039 learned_addr learned my addr 172.21.15.93:0/1048850039 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:13.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8b659c700 1 -- 172.21.15.93:0/1048850039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8b8101480 msgr2=0x7fd8b8072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:13.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8b659c700 1 --2- 172.21.15.93:0/1048850039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8b8101480 0x7fd8b8072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8b659c700 1 -- 172.21.15.93:0/1048850039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8b8101e10 msgr2=0x7fd8b8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:13.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8b659c700 1 --2- 172.21.15.93:0/1048850039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8b8101e10 0x7fd8b8073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8b659c700 1 -- 172.21.15.93:0/1048850039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80095f0 con 0x7fd8b8100290 2023-11-16T16:47:13.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8b659c700 1 --2- 172.21.15.93:0/1048850039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8b8100290 0x7fd8b80748a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8005740 tx=0x7fd8a8005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:13.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.404+0000 7fd8a77fe700 1 -- 172.21.15.93:0/1048850039 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a801d070 con 0x7fd8b8100290 2023-11-16T16:47:13.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.405+0000 7fd8bcdff700 1 -- 172.21.15.93:0/1048850039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b810bc40 con 0x7fd8b8100290 2023-11-16T16:47:13.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.405+0000 7fd8bcdff700 1 -- 172.21.15.93:0/1048850039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b810bab0 con 0x7fd8b8100290 2023-11-16T16:47:13.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.405+0000 7fd8a77fe700 1 -- 172.21.15.93:0/1048850039 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8a8021410 con 0x7fd8b8100290 2023-11-16T16:47:13.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.405+0000 7fd8a77fe700 1 -- 172.21.15.93:0/1048850039 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8017650 con 0x7fd8b8100290 2023-11-16T16:47:13.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.406+0000 7fd8bcdff700 1 -- 172.21.15.93:0/1048850039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd898005320 con 0x7fd8b8100290 2023-11-16T16:47:13.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.408+0000 7fd8a77fe700 1 -- 172.21.15.93:0/1048850039 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd8a800fb40 con 0x7fd8b8100290 2023-11-16T16:47:13.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.409+0000 7fd8a77fe700 1 --2- 172.21.15.93:0/1048850039 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8a00a11e0 0x7fd8a00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.409+0000 7fd8a77fe700 1 -- 172.21.15.93:0/1048850039 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd8a80ce080 con 0x7fd8b8100290 2023-11-16T16:47:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.409+0000 7fd8b5d9b700 1 --2- 172.21.15.93:0/1048850039 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8a00a11e0 0x7fd8a00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.410+0000 7fd8b5d9b700 1 --2- 172.21.15.93:0/1048850039 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8a00a11e0 0x7fd8a00a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009c60 tx=0x7fd8ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.411+0000 7fd8a77fe700 1 -- 172.21.15.93:0/1048850039 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd8a80973a0 con 0x7fd8b8100290 2023-11-16T16:47:13.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.606+0000 7fd8bcdff700 1 -- 172.21.15.93:0/1048850039 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd898000bf0 con 0x7fd8a00a11e0 2023-11-16T16:47:13.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.621+0000 7fd8a77fe700 1 -- 172.21.15.93:0/1048850039 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422386 (secure 0 0 0) 0x7fd898000bf0 con 0x7fd8a00a11e0 2023-11-16T16:47:13.629 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:47:13.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.626+0000 7fd8bcdff700 1 -- 172.21.15.93:0/1048850039 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8a00a11e0 msgr2=0x7fd8a00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:13.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.626+0000 7fd8bcdff700 1 --2- 172.21.15.93:0/1048850039 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8a00a11e0 0x7fd8a00a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009c60 tx=0x7fd8ac009260 comp rx=0 tx=0).stop 2023-11-16T16:47:13.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.626+0000 7fd8bcdff700 1 -- 172.21.15.93:0/1048850039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8b8100290 msgr2=0x7fd8b80748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:13.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.626+0000 7fd8bcdff700 1 --2- 172.21.15.93:0/1048850039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8b8100290 0x7fd8b80748a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8005740 tx=0x7fd8a8005770 comp rx=0 tx=0).stop 2023-11-16T16:47:13.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.627+0000 7fd8bcdff700 1 -- 172.21.15.93:0/1048850039 shutdown_connections 2023-11-16T16:47:13.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.627+0000 7fd8bcdff700 1 --2- 172.21.15.93:0/1048850039 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd8a00a11e0 0x7fd8a00a3690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.627+0000 7fd8bcdff700 1 --2- 172.21.15.93:0/1048850039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8b8101e10 0x7fd8b8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.627+0000 7fd8bcdff700 1 --2- 172.21.15.93:0/1048850039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8b8100290 0x7fd8b80748a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.627+0000 7fd8bcdff700 1 --2- 172.21.15.93:0/1048850039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8b8101480 0x7fd8b8072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.627+0000 7fd8bcdff700 1 -- 172.21.15.93:0/1048850039 >> 172.21.15.93:0/1048850039 conn(0x7fd8b80fb870 msgr2=0x7fd8b81049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:13.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.627+0000 7fd8bcdff700 1 -- 172.21.15.93:0/1048850039 shutdown_connections 2023-11-16T16:47:13.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.627+0000 7fd8bcdff700 1 -- 172.21.15.93:0/1048850039 wait complete. 2023-11-16T16:47:13.633 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-11-16T16:47:13.649 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2023-11-16T16:47:12.522093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597607,"num_objects":54,"num_object_clones":0,"num_object_copies":163,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":142,"num_read_kb":123,"num_write":291,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":82312,"kb_used_data":13608,"kb_used_omap":0,"kb_used_meta":68608,"kb_avail":1124859512,"statfs":{"total":1151940427776,"available":1151856140288,"internally_reserved":0,"allocated":13934592,"data_stored":7663022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70254592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2788,"num_objects":30,"num_object_clones":0,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":62,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003725"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336272+0000","last_change":"2023-11-16T16:46:37.150429+0000","last_active":"2023-11-16T16:46:52.336272+0000","last_peered":"2023-11-16T16:46:52.336272+0000","last_clean":"2023-11-16T16:46:52.336272+0000","last_became_active":"2023-11-16T16:46:37.150133+0000","last_became_peered":"2023-11-16T16:46:37.150133+0000","last_unstale":"2023-11-16T16:46:52.336272+0000","last_undegraded":"2023-11-16T16:46:52.336272+0000","last_fullsized":"2023-11-16T16:46:52.336272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:23:35.649395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330560+0000","last_change":"2023-11-16T16:46:37.155729+0000","last_active":"2023-11-16T16:46:52.330560+0000","last_peered":"2023-11-16T16:46:52.330560+0000","last_clean":"2023-11-16T16:46:52.330560+0000","last_became_active":"2023-11-16T16:46:37.155178+0000","last_became_peered":"2023-11-16T16:46:37.155178+0000","last_unstale":"2023-11-16T16:46:52.330560+0000","last_undegraded":"2023-11-16T16:46:52.330560+0000","last_fullsized":"2023-11-16T16:46:52.330560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:32:07.453061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330033+0000","last_change":"2023-11-16T16:46:37.154669+0000","last_active":"2023-11-16T16:46:52.330033+0000","last_peered":"2023-11-16T16:46:52.330033+0000","last_clean":"2023-11-16T16:46:52.330033+0000","last_became_active":"2023-11-16T16:46:37.154331+0000","last_became_peered":"2023-11-16T16:46:37.154331+0000","last_unstale":"2023-11-16T16:46:52.330033+0000","last_undegraded":"2023-11-16T16:46:52.330033+0000","last_fullsized":"2023-11-16T16:46:52.330033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:18:33.416725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335112+0000","last_change":"2023-11-16T16:46:37.140627+0000","last_active":"2023-11-16T16:46:52.335112+0000","last_peered":"2023-11-16T16:46:52.335112+0000","last_clean":"2023-11-16T16:46:52.335112+0000","last_became_active":"2023-11-16T16:46:37.138386+0000","last_became_peered":"2023-11-16T16:46:37.138386+0000","last_unstale":"2023-11-16T16:46:52.335112+0000","last_undegraded":"2023-11-16T16:46:52.335112+0000","last_fullsized":"2023-11-16T16:46:52.335112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:47:01.255110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.332315+0000","last_change":"2023-11-16T16:46:37.154046+0000","last_active":"2023-11-16T16:46:52.332315+0000","last_peered":"2023-11-16T16:46:52.332315+0000","last_clean":"2023-11-16T16:46:52.332315+0000","last_became_active":"2023-11-16T16:46:37.153677+0000","last_became_peered":"2023-11-16T16:46:37.153677+0000","last_unstale":"2023-11-16T16:46:52.332315+0000","last_undegraded":"2023-11-16T16:46:52.332315+0000","last_fullsized":"2023-11-16T16:46:52.332315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:13:01.802905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044944+0000","last_change":"2023-11-16T16:46:37.153809+0000","last_active":"2023-11-16T16:46:53.044944+0000","last_peered":"2023-11-16T16:46:53.044944+0000","last_clean":"2023-11-16T16:46:53.044944+0000","last_became_active":"2023-11-16T16:46:37.153511+0000","last_became_peered":"2023-11-16T16:46:37.153511+0000","last_unstale":"2023-11-16T16:46:53.044944+0000","last_undegraded":"2023-11-16T16:46:53.044944+0000","last_fullsized":"2023-11-16T16:46:53.044944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:11:54.236854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330213+0000","last_change":"2023-11-16T16:46:37.142530+0000","last_active":"2023-11-16T16:46:52.330213+0000","last_peered":"2023-11-16T16:46:52.330213+0000","last_clean":"2023-11-16T16:46:52.330213+0000","last_became_active":"2023-11-16T16:46:37.142216+0000","last_became_peered":"2023-11-16T16:46:37.142216+0000","last_unstale":"2023-11-16T16:46:52.330213+0000","last_undegraded":"2023-11-16T16:46:52.330213+0000","last_fullsized":"2023-11-16T16:46:52.330213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:02:25.342439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044675+0000","last_change":"2023-11-16T16:46:37.151956+0000","last_active":"2023-11-16T16:46:53.044675+0000","last_peered":"2023-11-16T16:46:53.044675+0000","last_clean":"2023-11-16T16:46:53.044675+0000","last_became_active":"2023-11-16T16:46:37.149432+0000","last_became_peered":"2023-11-16T16:46:37.149432+0000","last_unstale":"2023-11-16T16:46:53.044675+0000","last_undegraded":"2023-11-16T16:46:53.044675+0000","last_fullsized":"2023-11-16T16:46:53.044675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:06:52.090307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333580+0000","last_change":"2023-11-16T16:46:37.148378+0000","last_active":"2023-11-16T16:46:52.333580+0000","last_peered":"2023-11-16T16:46:52.333580+0000","last_clean":"2023-11-16T16:46:52.333580+0000","last_became_active":"2023-11-16T16:46:37.148082+0000","last_became_peered":"2023-11-16T16:46:37.148082+0000","last_unstale":"2023-11-16T16:46:52.333580+0000","last_undegraded":"2023-11-16T16:46:52.333580+0000","last_fullsized":"2023-11-16T16:46:52.333580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:58:05.786729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.045350+0000","last_change":"2023-11-16T16:46:37.153895+0000","last_active":"2023-11-16T16:46:53.045350+0000","last_peered":"2023-11-16T16:46:53.045350+0000","last_clean":"2023-11-16T16:46:53.045350+0000","last_became_active":"2023-11-16T16:46:37.153491+0000","last_became_peered":"2023-11-16T16:46:37.153491+0000","last_unstale":"2023-11-16T16:46:53.045350+0000","last_undegraded":"2023-11-16T16:46:53.045350+0000","last_fullsized":"2023-11-16T16:46:53.045350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:32:45.116412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331206+0000","last_change":"2023-11-16T16:46:37.148036+0000","last_active":"2023-11-16T16:46:52.331206+0000","last_peered":"2023-11-16T16:46:52.331206+0000","last_clean":"2023-11-16T16:46:52.331206+0000","last_became_active":"2023-11-16T16:46:37.147372+0000","last_became_peered":"2023-11-16T16:46:37.147372+0000","last_unstale":"2023-11-16T16:46:52.331206+0000","last_undegraded":"2023-11-16T16:46:52.331206+0000","last_fullsized":"2023-11-16T16:46:52.331206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:00:49.806134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335024+0000","last_change":"2023-11-16T16:46:37.147120+0000","last_active":"2023-11-16T16:46:52.335024+0000","last_peered":"2023-11-16T16:46:52.335024+0000","last_clean":"2023-11-16T16:46:52.335024+0000","last_became_active":"2023-11-16T16:46:37.146615+0000","last_became_peered":"2023-11-16T16:46:37.146615+0000","last_unstale":"2023-11-16T16:46:52.335024+0000","last_undegraded":"2023-11-16T16:46:52.335024+0000","last_fullsized":"2023-11-16T16:46:52.335024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:56:35.911100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426417+0000","last_change":"2023-11-16T16:46:37.152029+0000","last_active":"2023-11-16T16:46:52.426417+0000","last_peered":"2023-11-16T16:46:52.426417+0000","last_clean":"2023-11-16T16:46:52.426417+0000","last_became_active":"2023-11-16T16:46:37.150662+0000","last_became_peered":"2023-11-16T16:46:37.150662+0000","last_unstale":"2023-11-16T16:46:52.426417+0000","last_undegraded":"2023-11-16T16:46:52.426417+0000","last_fullsized":"2023-11-16T16:46:52.426417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:16:19.042910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.329535+0000","last_change":"2023-11-16T16:46:37.160562+0000","last_active":"2023-11-16T16:46:52.329535+0000","last_peered":"2023-11-16T16:46:52.329535+0000","last_clean":"2023-11-16T16:46:52.329535+0000","last_became_active":"2023-11-16T16:46:37.158175+0000","last_became_peered":"2023-11-16T16:46:37.158175+0000","last_unstale":"2023-11-16T16:46:52.329535+0000","last_undegraded":"2023-11-16T16:46:52.329535+0000","last_fullsized":"2023-11-16T16:46:52.329535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:02:03.417017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.329392+0000","last_change":"2023-11-16T16:46:37.156381+0000","last_active":"2023-11-16T16:46:52.329392+0000","last_peered":"2023-11-16T16:46:52.329392+0000","last_clean":"2023-11-16T16:46:52.329392+0000","last_became_active":"2023-11-16T16:46:37.150366+0000","last_became_peered":"2023-11-16T16:46:37.150366+0000","last_unstale":"2023-11-16T16:46:52.329392+0000","last_undegraded":"2023-11-16T16:46:52.329392+0000","last_fullsized":"2023-11-16T16:46:52.329392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:50:21.478034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331876+0000","last_change":"2023-11-16T16:46:37.158070+0000","last_active":"2023-11-16T16:46:52.331876+0000","last_peered":"2023-11-16T16:46:52.331876+0000","last_clean":"2023-11-16T16:46:52.331876+0000","last_became_active":"2023-11-16T16:46:37.157755+0000","last_became_peered":"2023-11-16T16:46:37.157755+0000","last_unstale":"2023-11-16T16:46:52.331876+0000","last_undegraded":"2023-11-16T16:46:52.331876+0000","last_fullsized":"2023-11-16T16:46:52.331876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:38:14.007502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336484+0000","last_change":"2023-11-16T16:46:37.162616+0000","last_active":"2023-11-16T16:46:52.336484+0000","last_peered":"2023-11-16T16:46:52.336484+0000","last_clean":"2023-11-16T16:46:52.336484+0000","last_became_active":"2023-11-16T16:46:37.162281+0000","last_became_peered":"2023-11-16T16:46:37.162281+0000","last_unstale":"2023-11-16T16:46:52.336484+0000","last_undegraded":"2023-11-16T16:46:52.336484+0000","last_fullsized":"2023-11-16T16:46:52.336484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:13.452584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331314+0000","last_change":"2023-11-16T16:46:37.150337+0000","last_active":"2023-11-16T16:46:52.331314+0000","last_peered":"2023-11-16T16:46:52.331314+0000","last_clean":"2023-11-16T16:46:52.331314+0000","last_became_active":"2023-11-16T16:46:37.148826+0000","last_became_peered":"2023-11-16T16:46:37.148826+0000","last_unstale":"2023-11-16T16:46:52.331314+0000","last_undegraded":"2023-11-16T16:46:52.331314+0000","last_fullsized":"2023-11-16T16:46:52.331314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:28.902287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336555+0000","last_change":"2023-11-16T16:46:37.150253+0000","last_active":"2023-11-16T16:46:52.336555+0000","last_peered":"2023-11-16T16:46:52.336555+0000","last_clean":"2023-11-16T16:46:52.336555+0000","last_became_active":"2023-11-16T16:46:37.149962+0000","last_became_peered":"2023-11-16T16:46:37.149962+0000","last_unstale":"2023-11-16T16:46:52.336555+0000","last_undegraded":"2023-11-16T16:46:52.336555+0000","last_fullsized":"2023-11-16T16:46:52.336555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:35:48.970850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334978+0000","last_change":"2023-11-16T16:46:41.177888+0000","last_active":"2023-11-16T16:46:52.334978+0000","last_peered":"2023-11-16T16:46:52.334978+0000","last_clean":"2023-11-16T16:46:52.334978+0000","last_became_active":"2023-11-16T16:46:41.177442+0000","last_became_peered":"2023-11-16T16:46:41.177442+0000","last_unstale":"2023-11-16T16:46:52.334978+0000","last_undegraded":"2023-11-16T16:46:52.334978+0000","last_fullsized":"2023-11-16T16:46:52.334978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:11:25.938101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:09.650717+0000","last_change":"2023-11-16T16:46:36.595769+0000","last_active":"2023-11-16T16:47:09.650717+0000","last_peered":"2023-11-16T16:47:09.650717+0000","last_clean":"2023-11-16T16:47:09.650717+0000","last_became_active":"2023-11-16T16:46:36.593780+0000","last_became_peered":"2023-11-16T16:46:36.593780+0000","last_unstale":"2023-11-16T16:47:09.650717+0000","last_undegraded":"2023-11-16T16:47:09.650717+0000","last_fullsized":"2023-11-16T16:47:09.650717+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:56:43.139818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.045299+0000","last_change":"2023-11-16T16:46:37.146683+0000","last_active":"2023-11-16T16:46:53.045299+0000","last_peered":"2023-11-16T16:46:53.045299+0000","last_clean":"2023-11-16T16:46:53.045299+0000","last_became_active":"2023-11-16T16:46:37.146363+0000","last_became_peered":"2023-11-16T16:46:37.146363+0000","last_unstale":"2023-11-16T16:46:53.045299+0000","last_undegraded":"2023-11-16T16:46:53.045299+0000","last_fullsized":"2023-11-16T16:46:53.045299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:00:19.122906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333768+0000","last_change":"2023-11-16T16:46:41.174942+0000","last_active":"2023-11-16T16:46:52.333768+0000","last_peered":"2023-11-16T16:46:52.333768+0000","last_clean":"2023-11-16T16:46:52.333768+0000","last_became_active":"2023-11-16T16:46:41.174483+0000","last_became_peered":"2023-11-16T16:46:41.174483+0000","last_unstale":"2023-11-16T16:46:52.333768+0000","last_undegraded":"2023-11-16T16:46:52.333768+0000","last_fullsized":"2023-11-16T16:46:52.333768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:08:57.816394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044328+0000","last_change":"2023-11-16T16:46:36.607478+0000","last_active":"2023-11-16T16:46:53.044328+0000","last_peered":"2023-11-16T16:46:53.044328+0000","last_clean":"2023-11-16T16:46:53.044328+0000","last_became_active":"2023-11-16T16:46:36.607137+0000","last_became_peered":"2023-11-16T16:46:36.607137+0000","last_unstale":"2023-11-16T16:46:53.044328+0000","last_undegraded":"2023-11-16T16:46:53.044328+0000","last_fullsized":"2023-11-16T16:46:53.044328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:13:10.233490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330466+0000","last_change":"2023-11-16T16:46:37.148576+0000","last_active":"2023-11-16T16:46:52.330466+0000","last_peered":"2023-11-16T16:46:52.330466+0000","last_clean":"2023-11-16T16:46:52.330466+0000","last_became_active":"2023-11-16T16:46:37.148284+0000","last_became_peered":"2023-11-16T16:46:37.148284+0000","last_unstale":"2023-11-16T16:46:52.330466+0000","last_undegraded":"2023-11-16T16:46:52.330466+0000","last_fullsized":"2023-11-16T16:46:52.330466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:11:53.759257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333970+0000","last_change":"2023-11-16T16:46:41.178717+0000","last_active":"2023-11-16T16:46:52.333970+0000","last_peered":"2023-11-16T16:46:52.333970+0000","last_clean":"2023-11-16T16:46:52.333970+0000","last_became_active":"2023-11-16T16:46:41.178350+0000","last_became_peered":"2023-11-16T16:46:41.178350+0000","last_unstale":"2023-11-16T16:46:52.333970+0000","last_undegraded":"2023-11-16T16:46:52.333970+0000","last_fullsized":"2023-11-16T16:46:52.333970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:54:20.213711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:09.648970+0000","last_change":"2023-11-16T16:46:52.639272+0000","last_active":"2023-11-16T16:47:09.648970+0000","last_peered":"2023-11-16T16:47:09.648970+0000","last_clean":"2023-11-16T16:47:09.648970+0000","last_became_active":"2023-11-16T16:46:52.338319+0000","last_became_peered":"2023-11-16T16:46:52.338319+0000","last_unstale":"2023-11-16T16:47:09.648970+0000","last_undegraded":"2023-11-16T16:47:09.648970+0000","last_fullsized":"2023-11-16T16:47:09.648970+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:45:13.831749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.332231+0000","last_change":"2023-11-16T16:46:37.153404+0000","last_active":"2023-11-16T16:46:52.332231+0000","last_peered":"2023-11-16T16:46:52.332231+0000","last_clean":"2023-11-16T16:46:52.332231+0000","last_became_active":"2023-11-16T16:46:37.153064+0000","last_became_peered":"2023-11-16T16:46:37.153064+0000","last_unstale":"2023-11-16T16:46:52.332231+0000","last_undegraded":"2023-11-16T16:46:52.332231+0000","last_fullsized":"2023-11-16T16:46:52.332231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:56:27.226825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333139+0000","last_change":"2023-11-16T16:46:41.163307+0000","last_active":"2023-11-16T16:46:52.333139+0000","last_peered":"2023-11-16T16:46:52.333139+0000","last_clean":"2023-11-16T16:46:52.333139+0000","last_became_active":"2023-11-16T16:46:41.162822+0000","last_became_peered":"2023-11-16T16:46:41.162822+0000","last_unstale":"2023-11-16T16:46:52.333139+0000","last_undegraded":"2023-11-16T16:46:52.333139+0000","last_fullsized":"2023-11-16T16:46:52.333139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:44:51.683473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.332575+0000","last_change":"2023-11-16T16:46:36.595905+0000","last_active":"2023-11-16T16:46:52.332575+0000","last_peered":"2023-11-16T16:46:52.332575+0000","last_clean":"2023-11-16T16:46:52.332575+0000","last_became_active":"2023-11-16T16:46:36.595403+0000","last_became_peered":"2023-11-16T16:46:36.595403+0000","last_unstale":"2023-11-16T16:46:52.332575+0000","last_undegraded":"2023-11-16T16:46:52.332575+0000","last_fullsized":"2023-11-16T16:46:52.332575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:53:07.115959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334655+0000","last_change":"2023-11-16T16:46:52.334655+0000","last_active":"2023-11-16T16:46:52.334655+0000","last_peered":"2023-11-16T16:46:52.334655+0000","last_clean":"2023-11-16T16:46:52.334655+0000","last_became_active":"2023-11-16T16:46:52.334345+0000","last_became_peered":"2023-11-16T16:46:52.334345+0000","last_unstale":"2023-11-16T16:46:52.334655+0000","last_undegraded":"2023-11-16T16:46:52.334655+0000","last_fullsized":"2023-11-16T16:46:52.334655+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:17:06.138428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336484+0000","last_change":"2023-11-16T16:46:41.169041+0000","last_active":"2023-11-16T16:46:52.336484+0000","last_peered":"2023-11-16T16:46:52.336484+0000","last_clean":"2023-11-16T16:46:52.336484+0000","last_became_active":"2023-11-16T16:46:41.168589+0000","last_became_peered":"2023-11-16T16:46:41.168589+0000","last_unstale":"2023-11-16T16:46:52.336484+0000","last_undegraded":"2023-11-16T16:46:52.336484+0000","last_fullsized":"2023-11-16T16:46:52.336484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:36:18.896357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.643139+0000","last_change":"2023-11-16T16:46:36.595442+0000","last_active":"2023-11-16T16:47:08.643139+0000","last_peered":"2023-11-16T16:47:08.643139+0000","last_clean":"2023-11-16T16:47:08.643139+0000","last_became_active":"2023-11-16T16:46:36.595122+0000","last_became_peered":"2023-11-16T16:46:36.595122+0000","last_unstale":"2023-11-16T16:47:08.643139+0000","last_undegraded":"2023-11-16T16:47:08.643139+0000","last_fullsized":"2023-11-16T16:47:08.643139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:38:17.777801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.332585+0000","last_change":"2023-11-16T16:46:37.149218+0000","last_active":"2023-11-16T16:46:52.332585+0000","last_peered":"2023-11-16T16:46:52.332585+0000","last_clean":"2023-11-16T16:46:52.332585+0000","last_became_active":"2023-11-16T16:46:37.148908+0000","last_became_peered":"2023-11-16T16:46:37.148908+0000","last_unstale":"2023-11-16T16:46:52.332585+0000","last_undegraded":"2023-11-16T16:46:52.332585+0000","last_fullsized":"2023-11-16T16:46:52.332585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:03:58.945088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336993+0000","last_change":"2023-11-16T16:46:41.185001+0000","last_active":"2023-11-16T16:46:52.336993+0000","last_peered":"2023-11-16T16:46:52.336993+0000","last_clean":"2023-11-16T16:46:52.336993+0000","last_became_active":"2023-11-16T16:46:41.184603+0000","last_became_peered":"2023-11-16T16:46:41.184603+0000","last_unstale":"2023-11-16T16:46:52.336993+0000","last_undegraded":"2023-11-16T16:46:52.336993+0000","last_fullsized":"2023-11-16T16:46:52.336993+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:54:30.993576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044109+0000","last_change":"2023-11-16T16:46:36.611365+0000","last_active":"2023-11-16T16:46:53.044109+0000","last_peered":"2023-11-16T16:46:53.044109+0000","last_clean":"2023-11-16T16:46:53.044109+0000","last_became_active":"2023-11-16T16:46:36.611006+0000","last_became_peered":"2023-11-16T16:46:36.611006+0000","last_unstale":"2023-11-16T16:46:53.044109+0000","last_undegraded":"2023-11-16T16:46:53.044109+0000","last_fullsized":"2023-11-16T16:46:53.044109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:39:42.167891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327902+0000","last_change":"2023-11-16T16:46:37.156801+0000","last_active":"2023-11-16T16:46:52.327902+0000","last_peered":"2023-11-16T16:46:52.327902+0000","last_clean":"2023-11-16T16:46:52.327902+0000","last_became_active":"2023-11-16T16:46:37.156352+0000","last_became_peered":"2023-11-16T16:46:37.156352+0000","last_unstale":"2023-11-16T16:46:52.327902+0000","last_undegraded":"2023-11-16T16:46:52.327902+0000","last_fullsized":"2023-11-16T16:46:52.327902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:40:14.223607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336005+0000","last_change":"2023-11-16T16:46:41.182374+0000","last_active":"2023-11-16T16:46:52.336005+0000","last_peered":"2023-11-16T16:46:52.336005+0000","last_clean":"2023-11-16T16:46:52.336005+0000","last_became_active":"2023-11-16T16:46:41.181977+0000","last_became_peered":"2023-11-16T16:46:41.181977+0000","last_unstale":"2023-11-16T16:46:52.336005+0000","last_undegraded":"2023-11-16T16:46:52.336005+0000","last_fullsized":"2023-11-16T16:46:52.336005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:17:28.606630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:09.651645+0000","last_change":"2023-11-16T16:46:36.607647+0000","last_active":"2023-11-16T16:47:09.651645+0000","last_peered":"2023-11-16T16:47:09.651645+0000","last_clean":"2023-11-16T16:47:09.651645+0000","last_became_active":"2023-11-16T16:46:36.607282+0000","last_became_peered":"2023-11-16T16:46:36.607282+0000","last_unstale":"2023-11-16T16:47:09.651645+0000","last_undegraded":"2023-11-16T16:47:09.651645+0000","last_fullsized":"2023-11-16T16:47:09.651645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:27:34.869730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044528+0000","last_change":"2023-11-16T16:46:37.152659+0000","last_active":"2023-11-16T16:46:53.044528+0000","last_peered":"2023-11-16T16:46:53.044528+0000","last_clean":"2023-11-16T16:46:53.044528+0000","last_became_active":"2023-11-16T16:46:37.152313+0000","last_became_peered":"2023-11-16T16:46:37.152313+0000","last_unstale":"2023-11-16T16:46:53.044528+0000","last_undegraded":"2023-11-16T16:46:53.044528+0000","last_fullsized":"2023-11-16T16:46:53.044528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:20:05.081803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330459+0000","last_change":"2023-11-16T16:46:41.167312+0000","last_active":"2023-11-16T16:46:52.330459+0000","last_peered":"2023-11-16T16:46:52.330459+0000","last_clean":"2023-11-16T16:46:52.330459+0000","last_became_active":"2023-11-16T16:46:41.166211+0000","last_became_peered":"2023-11-16T16:46:41.166211+0000","last_unstale":"2023-11-16T16:46:52.330459+0000","last_undegraded":"2023-11-16T16:46:52.330459+0000","last_fullsized":"2023-11-16T16:46:52.330459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:32:06.730118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327225+0000","last_change":"2023-11-16T16:46:36.595888+0000","last_active":"2023-11-16T16:46:52.327225+0000","last_peered":"2023-11-16T16:46:52.327225+0000","last_clean":"2023-11-16T16:46:52.327225+0000","last_became_active":"2023-11-16T16:46:36.595602+0000","last_became_peered":"2023-11-16T16:46:36.595602+0000","last_unstale":"2023-11-16T16:46:52.327225+0000","last_undegraded":"2023-11-16T16:46:52.327225+0000","last_fullsized":"2023-11-16T16:46:52.327225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:29:12.613330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330260+0000","last_change":"2023-11-16T16:46:37.154431+0000","last_active":"2023-11-16T16:46:52.330260+0000","last_peered":"2023-11-16T16:46:52.330260+0000","last_clean":"2023-11-16T16:46:52.330260+0000","last_became_active":"2023-11-16T16:46:37.154106+0000","last_became_peered":"2023-11-16T16:46:37.154106+0000","last_unstale":"2023-11-16T16:46:52.330260+0000","last_undegraded":"2023-11-16T16:46:52.330260+0000","last_fullsized":"2023-11-16T16:46:52.330260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:07:37.024127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044743+0000","last_change":"2023-11-16T16:46:41.168644+0000","last_active":"2023-11-16T16:46:53.044743+0000","last_peered":"2023-11-16T16:46:53.044743+0000","last_clean":"2023-11-16T16:46:53.044743+0000","last_became_active":"2023-11-16T16:46:41.168121+0000","last_became_peered":"2023-11-16T16:46:41.168121+0000","last_unstale":"2023-11-16T16:46:53.044743+0000","last_undegraded":"2023-11-16T16:46:53.044743+0000","last_fullsized":"2023-11-16T16:46:53.044743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:07:29.914977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.643553+0000","last_change":"2023-11-16T16:46:36.597293+0000","last_active":"2023-11-16T16:47:08.643553+0000","last_peered":"2023-11-16T16:47:08.643553+0000","last_clean":"2023-11-16T16:47:08.643553+0000","last_became_active":"2023-11-16T16:46:36.596992+0000","last_became_peered":"2023-11-16T16:46:36.596992+0000","last_unstale":"2023-11-16T16:47:08.643553+0000","last_undegraded":"2023-11-16T16:47:08.643553+0000","last_fullsized":"2023-11-16T16:47:08.643553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:13:49.650449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333206+0000","last_change":"2023-11-16T16:46:37.149339+0000","last_active":"2023-11-16T16:46:52.333206+0000","last_peered":"2023-11-16T16:46:52.333206+0000","last_clean":"2023-11-16T16:46:52.333206+0000","last_became_active":"2023-11-16T16:46:37.149000+0000","last_became_peered":"2023-11-16T16:46:37.149000+0000","last_unstale":"2023-11-16T16:46:52.333206+0000","last_undegraded":"2023-11-16T16:46:52.333206+0000","last_fullsized":"2023-11-16T16:46:52.333206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:22:35.436696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.339769+0000","last_change":"2023-11-16T16:46:52.338731+0000","last_active":"2023-11-16T16:46:52.339769+0000","last_peered":"2023-11-16T16:46:52.339769+0000","last_clean":"2023-11-16T16:46:52.339769+0000","last_became_active":"2023-11-16T16:46:52.338254+0000","last_became_peered":"2023-11-16T16:46:52.338254+0000","last_unstale":"2023-11-16T16:46:52.339769+0000","last_undegraded":"2023-11-16T16:46:52.339769+0000","last_fullsized":"2023-11-16T16:46:52.339769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:41:05.865764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330874+0000","last_change":"2023-11-16T16:46:36.595407+0000","last_active":"2023-11-16T16:46:52.330874+0000","last_peered":"2023-11-16T16:46:52.330874+0000","last_clean":"2023-11-16T16:46:52.330874+0000","last_became_active":"2023-11-16T16:46:36.595132+0000","last_became_peered":"2023-11-16T16:46:36.595132+0000","last_unstale":"2023-11-16T16:46:52.330874+0000","last_undegraded":"2023-11-16T16:46:52.330874+0000","last_fullsized":"2023-11-16T16:46:52.330874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:20:59.670565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330345+0000","last_change":"2023-11-16T16:46:37.155037+0000","last_active":"2023-11-16T16:46:52.330345+0000","last_peered":"2023-11-16T16:46:52.330345+0000","last_clean":"2023-11-16T16:46:52.330345+0000","last_became_active":"2023-11-16T16:46:37.149822+0000","last_became_peered":"2023-11-16T16:46:37.149822+0000","last_unstale":"2023-11-16T16:46:52.330345+0000","last_undegraded":"2023-11-16T16:46:52.330345+0000","last_fullsized":"2023-11-16T16:46:52.330345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:56:10.462369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.339033+0000","last_change":"2023-11-16T16:46:52.339033+0000","last_active":"2023-11-16T16:46:52.339033+0000","last_peered":"2023-11-16T16:46:52.339033+0000","last_clean":"2023-11-16T16:46:52.339033+0000","last_became_active":"2023-11-16T16:46:52.338451+0000","last_became_peered":"2023-11-16T16:46:52.338451+0000","last_unstale":"2023-11-16T16:46:52.339033+0000","last_undegraded":"2023-11-16T16:46:52.339033+0000","last_fullsized":"2023-11-16T16:46:52.339033+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:04:46.441512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.328042+0000","last_change":"2023-11-16T16:46:36.596779+0000","last_active":"2023-11-16T16:46:52.328042+0000","last_peered":"2023-11-16T16:46:52.328042+0000","last_clean":"2023-11-16T16:46:52.328042+0000","last_became_active":"2023-11-16T16:46:36.596460+0000","last_became_peered":"2023-11-16T16:46:36.596460+0000","last_unstale":"2023-11-16T16:46:52.328042+0000","last_undegraded":"2023-11-16T16:46:52.328042+0000","last_fullsized":"2023-11-16T16:46:52.328042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:15:12.352768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.427912+0000","last_change":"2023-11-16T16:46:52.427912+0000","last_active":"2023-11-16T16:46:52.427912+0000","last_peered":"2023-11-16T16:46:52.427912+0000","last_clean":"2023-11-16T16:46:52.427912+0000","last_became_active":"2023-11-16T16:46:52.427426+0000","last_became_peered":"2023-11-16T16:46:52.427426+0000","last_unstale":"2023-11-16T16:46:52.427912+0000","last_undegraded":"2023-11-16T16:46:52.427912+0000","last_fullsized":"2023-11-16T16:46:52.427912+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:44:57.112343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330521+0000","last_change":"2023-11-16T16:46:41.177483+0000","last_active":"2023-11-16T16:46:52.330521+0000","last_peered":"2023-11-16T16:46:52.330521+0000","last_clean":"2023-11-16T16:46:52.330521+0000","last_became_active":"2023-11-16T16:46:41.173165+0000","last_became_peered":"2023-11-16T16:46:41.173165+0000","last_unstale":"2023-11-16T16:46:52.330521+0000","last_undegraded":"2023-11-16T16:46:52.330521+0000","last_fullsized":"2023-11-16T16:46:52.330521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:57:53.705807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.642841+0000","last_change":"2023-11-16T16:46:36.598279+0000","last_active":"2023-11-16T16:47:08.642841+0000","last_peered":"2023-11-16T16:47:08.642841+0000","last_clean":"2023-11-16T16:47:08.642841+0000","last_became_active":"2023-11-16T16:46:36.597975+0000","last_became_peered":"2023-11-16T16:46:36.597975+0000","last_unstale":"2023-11-16T16:47:08.642841+0000","last_undegraded":"2023-11-16T16:47:08.642841+0000","last_fullsized":"2023-11-16T16:47:08.642841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:36:36.186292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333136+0000","last_change":"2023-11-16T16:46:37.143014+0000","last_active":"2023-11-16T16:46:52.333136+0000","last_peered":"2023-11-16T16:46:52.333136+0000","last_clean":"2023-11-16T16:46:52.333136+0000","last_became_active":"2023-11-16T16:46:37.142662+0000","last_became_peered":"2023-11-16T16:46:37.142662+0000","last_unstale":"2023-11-16T16:46:52.333136+0000","last_undegraded":"2023-11-16T16:46:52.333136+0000","last_fullsized":"2023-11-16T16:46:52.333136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:51:30.485958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.328331+0000","last_change":"2023-11-16T16:46:41.169921+0000","last_active":"2023-11-16T16:46:52.328331+0000","last_peered":"2023-11-16T16:46:52.328331+0000","last_clean":"2023-11-16T16:46:52.328331+0000","last_became_active":"2023-11-16T16:46:41.169037+0000","last_became_peered":"2023-11-16T16:46:41.169037+0000","last_unstale":"2023-11-16T16:46:52.328331+0000","last_undegraded":"2023-11-16T16:46:52.328331+0000","last_fullsized":"2023-11-16T16:46:52.328331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:08:05.111113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336722+0000","last_change":"2023-11-16T16:46:36.596159+0000","last_active":"2023-11-16T16:46:52.336722+0000","last_peered":"2023-11-16T16:46:52.336722+0000","last_clean":"2023-11-16T16:46:52.336722+0000","last_became_active":"2023-11-16T16:46:36.595847+0000","last_became_peered":"2023-11-16T16:46:36.595847+0000","last_unstale":"2023-11-16T16:46:52.336722+0000","last_undegraded":"2023-11-16T16:46:52.336722+0000","last_fullsized":"2023-11-16T16:46:52.336722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:06:32.736611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426173+0000","last_change":"2023-11-16T16:46:37.156046+0000","last_active":"2023-11-16T16:46:52.426173+0000","last_peered":"2023-11-16T16:46:52.426173+0000","last_clean":"2023-11-16T16:46:52.426173+0000","last_became_active":"2023-11-16T16:46:37.155657+0000","last_became_peered":"2023-11-16T16:46:37.155657+0000","last_unstale":"2023-11-16T16:46:52.426173+0000","last_undegraded":"2023-11-16T16:46:52.426173+0000","last_fullsized":"2023-11-16T16:46:52.426173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:57.255975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334980+0000","last_change":"2023-11-16T16:46:41.182485+0000","last_active":"2023-11-16T16:46:52.334980+0000","last_peered":"2023-11-16T16:46:52.334980+0000","last_clean":"2023-11-16T16:46:52.334980+0000","last_became_active":"2023-11-16T16:46:41.181990+0000","last_became_peered":"2023-11-16T16:46:41.181990+0000","last_unstale":"2023-11-16T16:46:52.334980+0000","last_undegraded":"2023-11-16T16:46:52.334980+0000","last_fullsized":"2023-11-16T16:46:52.334980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:15:28.764485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.643137+0000","last_change":"2023-11-16T16:46:36.608501+0000","last_active":"2023-11-16T16:47:08.643137+0000","last_peered":"2023-11-16T16:47:08.643137+0000","last_clean":"2023-11-16T16:47:08.643137+0000","last_became_active":"2023-11-16T16:46:36.608210+0000","last_became_peered":"2023-11-16T16:46:36.608210+0000","last_unstale":"2023-11-16T16:47:08.643137+0000","last_undegraded":"2023-11-16T16:47:08.643137+0000","last_fullsized":"2023-11-16T16:47:08.643137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:25:48.287851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330892+0000","last_change":"2023-11-16T16:46:37.155431+0000","last_active":"2023-11-16T16:46:52.330892+0000","last_peered":"2023-11-16T16:46:52.330892+0000","last_clean":"2023-11-16T16:46:52.330892+0000","last_became_active":"2023-11-16T16:46:37.155106+0000","last_became_peered":"2023-11-16T16:46:37.155106+0000","last_unstale":"2023-11-16T16:46:52.330892+0000","last_undegraded":"2023-11-16T16:46:52.330892+0000","last_fullsized":"2023-11-16T16:46:52.330892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:40:10.184454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331332+0000","last_change":"2023-11-16T16:46:41.180903+0000","last_active":"2023-11-16T16:46:52.331332+0000","last_peered":"2023-11-16T16:46:52.331332+0000","last_clean":"2023-11-16T16:46:52.331332+0000","last_became_active":"2023-11-16T16:46:41.179842+0000","last_became_peered":"2023-11-16T16:46:41.179842+0000","last_unstale":"2023-11-16T16:46:52.331332+0000","last_undegraded":"2023-11-16T16:46:52.331332+0000","last_fullsized":"2023-11-16T16:46:52.331332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:08:39.409755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:11.326880+0000","last_change":"2023-11-16T16:46:36.592436+0000","last_active":"2023-11-16T16:47:11.326880+0000","last_peered":"2023-11-16T16:47:11.326880+0000","last_clean":"2023-11-16T16:47:11.326880+0000","last_became_active":"2023-11-16T16:46:36.592078+0000","last_became_peered":"2023-11-16T16:46:36.592078+0000","last_unstale":"2023-11-16T16:47:11.326880+0000","last_undegraded":"2023-11-16T16:47:11.326880+0000","last_fullsized":"2023-11-16T16:47:11.326880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:23:31.429710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330109+0000","last_change":"2023-11-16T16:46:37.153071+0000","last_active":"2023-11-16T16:46:52.330109+0000","last_peered":"2023-11-16T16:46:52.330109+0000","last_clean":"2023-11-16T16:46:52.330109+0000","last_became_active":"2023-11-16T16:46:37.152742+0000","last_became_peered":"2023-11-16T16:46:37.152742+0000","last_unstale":"2023-11-16T16:46:52.330109+0000","last_undegraded":"2023-11-16T16:46:52.330109+0000","last_fullsized":"2023-11-16T16:46:52.330109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:57:55.272027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426556+0000","last_change":"2023-11-16T16:46:41.176434+0000","last_active":"2023-11-16T16:46:52.426556+0000","last_peered":"2023-11-16T16:46:52.426556+0000","last_clean":"2023-11-16T16:46:52.426556+0000","last_became_active":"2023-11-16T16:46:41.175771+0000","last_became_peered":"2023-11-16T16:46:41.175771+0000","last_unstale":"2023-11-16T16:46:52.426556+0000","last_undegraded":"2023-11-16T16:46:52.426556+0000","last_fullsized":"2023-11-16T16:46:52.426556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:19:46.468931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:09.652768+0000","last_change":"2023-11-16T16:46:36.594148+0000","last_active":"2023-11-16T16:47:09.652768+0000","last_peered":"2023-11-16T16:47:09.652768+0000","last_clean":"2023-11-16T16:47:09.652768+0000","last_became_active":"2023-11-16T16:46:36.593437+0000","last_became_peered":"2023-11-16T16:46:36.593437+0000","last_unstale":"2023-11-16T16:47:09.652768+0000","last_undegraded":"2023-11-16T16:47:09.652768+0000","last_fullsized":"2023-11-16T16:47:09.652768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:45:12.599977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327398+0000","last_change":"2023-11-16T16:46:37.149044+0000","last_active":"2023-11-16T16:46:52.327398+0000","last_peered":"2023-11-16T16:46:52.327398+0000","last_clean":"2023-11-16T16:46:52.327398+0000","last_became_active":"2023-11-16T16:46:37.147215+0000","last_became_peered":"2023-11-16T16:46:37.147215+0000","last_unstale":"2023-11-16T16:46:52.327398+0000","last_undegraded":"2023-11-16T16:46:52.327398+0000","last_fullsized":"2023-11-16T16:46:52.327398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:58:43.048815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327370+0000","last_change":"2023-11-16T16:46:41.168754+0000","last_active":"2023-11-16T16:46:52.327370+0000","last_peered":"2023-11-16T16:46:52.327370+0000","last_clean":"2023-11-16T16:46:52.327370+0000","last_became_active":"2023-11-16T16:46:41.168056+0000","last_became_peered":"2023-11-16T16:46:41.168056+0000","last_unstale":"2023-11-16T16:46:52.327370+0000","last_undegraded":"2023-11-16T16:46:52.327370+0000","last_fullsized":"2023-11-16T16:46:52.327370+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:23:57.870297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333794+0000","last_change":"2023-11-16T16:46:36.594892+0000","last_active":"2023-11-16T16:46:52.333794+0000","last_peered":"2023-11-16T16:46:52.333794+0000","last_clean":"2023-11-16T16:46:52.333794+0000","last_became_active":"2023-11-16T16:46:36.594557+0000","last_became_peered":"2023-11-16T16:46:36.594557+0000","last_unstale":"2023-11-16T16:46:52.333794+0000","last_undegraded":"2023-11-16T16:46:52.333794+0000","last_fullsized":"2023-11-16T16:46:52.333794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:17:50.519104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335354+0000","last_change":"2023-11-16T16:46:37.148203+0000","last_active":"2023-11-16T16:46:52.335354+0000","last_peered":"2023-11-16T16:46:52.335354+0000","last_clean":"2023-11-16T16:46:52.335354+0000","last_became_active":"2023-11-16T16:46:37.147846+0000","last_became_peered":"2023-11-16T16:46:37.147846+0000","last_unstale":"2023-11-16T16:46:52.335354+0000","last_undegraded":"2023-11-16T16:46:52.335354+0000","last_fullsized":"2023-11-16T16:46:52.335354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:02:06.446791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.046150+0000","last_change":"2023-11-16T16:46:53.046150+0000","last_active":"2023-11-16T16:46:53.046150+0000","last_peered":"2023-11-16T16:46:53.046150+0000","last_clean":"2023-11-16T16:46:53.046150+0000","last_became_active":"2023-11-16T16:46:53.045619+0000","last_became_peered":"2023-11-16T16:46:53.045619+0000","last_unstale":"2023-11-16T16:46:53.046150+0000","last_undegraded":"2023-11-16T16:46:53.046150+0000","last_fullsized":"2023-11-16T16:46:53.046150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:21:37.959125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330958+0000","last_change":"2023-11-16T16:46:36.600779+0000","last_active":"2023-11-16T16:46:52.330958+0000","last_peered":"2023-11-16T16:46:52.330958+0000","last_clean":"2023-11-16T16:46:52.330958+0000","last_became_active":"2023-11-16T16:46:36.598664+0000","last_became_peered":"2023-11-16T16:46:36.598664+0000","last_unstale":"2023-11-16T16:46:52.330958+0000","last_undegraded":"2023-11-16T16:46:52.330958+0000","last_fullsized":"2023-11-16T16:46:52.330958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:30:23.163310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.045270+0000","last_change":"2023-11-16T16:46:37.155694+0000","last_active":"2023-11-16T16:46:53.045270+0000","last_peered":"2023-11-16T16:46:53.045270+0000","last_clean":"2023-11-16T16:46:53.045270+0000","last_became_active":"2023-11-16T16:46:37.155105+0000","last_became_peered":"2023-11-16T16:46:37.155105+0000","last_unstale":"2023-11-16T16:46:53.045270+0000","last_undegraded":"2023-11-16T16:46:53.045270+0000","last_fullsized":"2023-11-16T16:46:53.045270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:56:49.859400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426481+0000","last_change":"2023-11-16T16:46:41.172161+0000","last_active":"2023-11-16T16:46:52.426481+0000","last_peered":"2023-11-16T16:46:52.426481+0000","last_clean":"2023-11-16T16:46:52.426481+0000","last_became_active":"2023-11-16T16:46:41.169661+0000","last_became_peered":"2023-11-16T16:46:41.169661+0000","last_unstale":"2023-11-16T16:46:52.426481+0000","last_undegraded":"2023-11-16T16:46:52.426481+0000","last_fullsized":"2023-11-16T16:46:52.426481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:21:34.679139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330271+0000","last_change":"2023-11-16T16:46:36.611376+0000","last_active":"2023-11-16T16:46:52.330271+0000","last_peered":"2023-11-16T16:46:52.330271+0000","last_clean":"2023-11-16T16:46:52.330271+0000","last_became_active":"2023-11-16T16:46:36.611045+0000","last_became_peered":"2023-11-16T16:46:36.611045+0000","last_unstale":"2023-11-16T16:46:52.330271+0000","last_undegraded":"2023-11-16T16:46:52.330271+0000","last_fullsized":"2023-11-16T16:46:52.330271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:11:15.321886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044773+0000","last_change":"2023-11-16T16:46:37.164850+0000","last_active":"2023-11-16T16:46:53.044773+0000","last_peered":"2023-11-16T16:46:53.044773+0000","last_clean":"2023-11-16T16:46:53.044773+0000","last_became_active":"2023-11-16T16:46:37.164532+0000","last_became_peered":"2023-11-16T16:46:37.164532+0000","last_unstale":"2023-11-16T16:46:53.044773+0000","last_undegraded":"2023-11-16T16:46:53.044773+0000","last_fullsized":"2023-11-16T16:46:53.044773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:26:29.906483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426557+0000","last_change":"2023-11-16T16:46:41.203618+0000","last_active":"2023-11-16T16:46:52.426557+0000","last_peered":"2023-11-16T16:46:52.426557+0000","last_clean":"2023-11-16T16:46:52.426557+0000","last_became_active":"2023-11-16T16:46:41.202322+0000","last_became_peered":"2023-11-16T16:46:41.202322+0000","last_unstale":"2023-11-16T16:46:52.426557+0000","last_undegraded":"2023-11-16T16:46:52.426557+0000","last_fullsized":"2023-11-16T16:46:52.426557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:51:27.136384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327322+0000","last_change":"2023-11-16T16:46:37.154064+0000","last_active":"2023-11-16T16:46:52.327322+0000","last_peered":"2023-11-16T16:46:52.327322+0000","last_clean":"2023-11-16T16:46:52.327322+0000","last_became_active":"2023-11-16T16:46:37.153776+0000","last_became_peered":"2023-11-16T16:46:37.153776+0000","last_unstale":"2023-11-16T16:46:52.327322+0000","last_undegraded":"2023-11-16T16:46:52.327322+0000","last_fullsized":"2023-11-16T16:46:52.327322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:22:57.827996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:09.653103+0000","last_change":"2023-11-16T16:46:36.612613+0000","last_active":"2023-11-16T16:47:09.653103+0000","last_peered":"2023-11-16T16:47:09.653103+0000","last_clean":"2023-11-16T16:47:09.653103+0000","last_became_active":"2023-11-16T16:46:36.612303+0000","last_became_peered":"2023-11-16T16:46:36.612303+0000","last_unstale":"2023-11-16T16:47:09.653103+0000","last_undegraded":"2023-11-16T16:47:09.653103+0000","last_fullsized":"2023-11-16T16:47:09.653103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:17:46.318085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.046753+0000","last_change":"2023-11-16T16:46:53.046753+0000","last_active":"2023-11-16T16:46:53.046753+0000","last_peered":"2023-11-16T16:46:53.046753+0000","last_clean":"2023-11-16T16:46:53.046753+0000","last_became_active":"2023-11-16T16:46:53.046231+0000","last_became_peered":"2023-11-16T16:46:53.046231+0000","last_unstale":"2023-11-16T16:46:53.046753+0000","last_undegraded":"2023-11-16T16:46:53.046753+0000","last_fullsized":"2023-11-16T16:46:53.046753+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:56:01.077949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044682+0000","last_change":"2023-11-16T16:46:37.150627+0000","last_active":"2023-11-16T16:46:53.044682+0000","last_peered":"2023-11-16T16:46:53.044682+0000","last_clean":"2023-11-16T16:46:53.044682+0000","last_became_active":"2023-11-16T16:46:37.147364+0000","last_became_peered":"2023-11-16T16:46:37.147364+0000","last_unstale":"2023-11-16T16:46:53.044682+0000","last_undegraded":"2023-11-16T16:46:53.044682+0000","last_fullsized":"2023-11-16T16:46:53.044682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:51:22.419717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:10.320058+0000","last_change":"2023-11-16T16:46:36.597520+0000","last_active":"2023-11-16T16:47:10.320058+0000","last_peered":"2023-11-16T16:47:10.320058+0000","last_clean":"2023-11-16T16:47:10.320058+0000","last_became_active":"2023-11-16T16:46:36.596542+0000","last_became_peered":"2023-11-16T16:46:36.596542+0000","last_unstale":"2023-11-16T16:47:10.320058+0000","last_undegraded":"2023-11-16T16:47:10.320058+0000","last_fullsized":"2023-11-16T16:47:10.320058+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:49:35.975555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.329918+0000","last_change":"2023-11-16T16:45:27.937850+0000","last_active":"2023-11-16T16:46:52.329918+0000","last_peered":"2023-11-16T16:46:52.329918+0000","last_clean":"2023-11-16T16:46:52.329918+0000","last_became_active":"2023-11-16T16:45:27.937451+0000","last_became_peered":"2023-11-16T16:45:27.937451+0000","last_unstale":"2023-11-16T16:46:52.329918+0000","last_undegraded":"2023-11-16T16:46:52.329918+0000","last_fullsized":"2023-11-16T16:46:52.329918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:42:13.306703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:42:13.306703+0000","last_clean_scrub_stamp":"2023-11-16T16:42:13.306703+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:52:16.097593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327903+0000","last_change":"2023-11-16T16:46:41.203736+0000","last_active":"2023-11-16T16:46:52.327903+0000","last_peered":"2023-11-16T16:46:52.327903+0000","last_clean":"2023-11-16T16:46:52.327903+0000","last_became_active":"2023-11-16T16:46:41.203282+0000","last_became_peered":"2023-11-16T16:46:41.203282+0000","last_unstale":"2023-11-16T16:46:52.327903+0000","last_undegraded":"2023-11-16T16:46:52.327903+0000","last_fullsized":"2023-11-16T16:46:52.327903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:19:06.910333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.425944+0000","last_change":"2023-11-16T16:46:37.157408+0000","last_active":"2023-11-16T16:46:52.425944+0000","last_peered":"2023-11-16T16:46:52.425944+0000","last_clean":"2023-11-16T16:46:52.425944+0000","last_became_active":"2023-11-16T16:46:37.156972+0000","last_became_peered":"2023-11-16T16:46:37.156972+0000","last_unstale":"2023-11-16T16:46:52.425944+0000","last_undegraded":"2023-11-16T16:46:52.425944+0000","last_fullsized":"2023-11-16T16:46:52.425944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:42.100042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.329970+0000","last_change":"2023-11-16T16:46:36.594284+0000","last_active":"2023-11-16T16:46:52.329970+0000","last_peered":"2023-11-16T16:46:52.329970+0000","last_clean":"2023-11-16T16:46:52.329970+0000","last_became_active":"2023-11-16T16:46:36.593964+0000","last_became_peered":"2023-11-16T16:46:36.593964+0000","last_unstale":"2023-11-16T16:46:52.329970+0000","last_undegraded":"2023-11-16T16:46:52.329970+0000","last_fullsized":"2023-11-16T16:46:52.329970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:23:47.325260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335577+0000","last_change":"2023-11-16T16:46:41.173944+0000","last_active":"2023-11-16T16:46:52.335577+0000","last_peered":"2023-11-16T16:46:52.335577+0000","last_clean":"2023-11-16T16:46:52.335577+0000","last_became_active":"2023-11-16T16:46:41.173391+0000","last_became_peered":"2023-11-16T16:46:41.173391+0000","last_unstale":"2023-11-16T16:46:52.335577+0000","last_undegraded":"2023-11-16T16:46:52.335577+0000","last_fullsized":"2023-11-16T16:46:52.335577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:27:11.037240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.642849+0000","last_change":"2023-11-16T16:46:36.596360+0000","last_active":"2023-11-16T16:47:08.642849+0000","last_peered":"2023-11-16T16:47:08.642849+0000","last_clean":"2023-11-16T16:47:08.642849+0000","last_became_active":"2023-11-16T16:46:36.596042+0000","last_became_peered":"2023-11-16T16:46:36.596042+0000","last_unstale":"2023-11-16T16:47:08.642849+0000","last_undegraded":"2023-11-16T16:47:08.642849+0000","last_fullsized":"2023-11-16T16:47:08.642849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:05:25.948915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426327+0000","last_change":"2023-11-16T16:46:37.160277+0000","last_active":"2023-11-16T16:46:52.426327+0000","last_peered":"2023-11-16T16:46:52.426327+0000","last_clean":"2023-11-16T16:46:52.426327+0000","last_became_active":"2023-11-16T16:46:37.159047+0000","last_became_peered":"2023-11-16T16:46:37.159047+0000","last_unstale":"2023-11-16T16:46:52.426327+0000","last_undegraded":"2023-11-16T16:46:52.426327+0000","last_fullsized":"2023-11-16T16:46:52.426327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:35:16.786773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044472+0000","last_change":"2023-11-16T16:46:41.202909+0000","last_active":"2023-11-16T16:46:53.044472+0000","last_peered":"2023-11-16T16:46:53.044472+0000","last_clean":"2023-11-16T16:46:53.044472+0000","last_became_active":"2023-11-16T16:46:41.201969+0000","last_became_peered":"2023-11-16T16:46:41.201969+0000","last_unstale":"2023-11-16T16:46:53.044472+0000","last_undegraded":"2023-11-16T16:46:53.044472+0000","last_fullsized":"2023-11-16T16:46:53.044472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:45:49.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:09.651822+0000","last_change":"2023-11-16T16:46:36.599389+0000","last_active":"2023-11-16T16:47:09.651822+0000","last_peered":"2023-11-16T16:47:09.651822+0000","last_clean":"2023-11-16T16:47:09.651822+0000","last_became_active":"2023-11-16T16:46:36.599020+0000","last_became_peered":"2023-11-16T16:46:36.599020+0000","last_unstale":"2023-11-16T16:47:09.651822+0000","last_undegraded":"2023-11-16T16:47:09.651822+0000","last_fullsized":"2023-11-16T16:47:09.651822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:35:00.154371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.045358+0000","last_change":"2023-11-16T16:46:37.145412+0000","last_active":"2023-11-16T16:46:53.045358+0000","last_peered":"2023-11-16T16:46:53.045358+0000","last_clean":"2023-11-16T16:46:53.045358+0000","last_became_active":"2023-11-16T16:46:37.141380+0000","last_became_peered":"2023-11-16T16:46:37.141380+0000","last_unstale":"2023-11-16T16:46:53.045358+0000","last_undegraded":"2023-11-16T16:46:53.045358+0000","last_fullsized":"2023-11-16T16:46:53.045358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:08:48.448800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333854+0000","last_change":"2023-11-16T16:46:41.163578+0000","last_active":"2023-11-16T16:46:52.333854+0000","last_peered":"2023-11-16T16:46:52.333854+0000","last_clean":"2023-11-16T16:46:52.333854+0000","last_became_active":"2023-11-16T16:46:41.163153+0000","last_became_peered":"2023-11-16T16:46:41.163153+0000","last_unstale":"2023-11-16T16:46:52.333854+0000","last_undegraded":"2023-11-16T16:46:52.333854+0000","last_fullsized":"2023-11-16T16:46:52.333854+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:53:12.394116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.640943+0000","last_change":"2023-11-16T16:46:36.606973+0000","last_active":"2023-11-16T16:47:08.640943+0000","last_peered":"2023-11-16T16:47:08.640943+0000","last_clean":"2023-11-16T16:47:08.640943+0000","last_became_active":"2023-11-16T16:46:36.606744+0000","last_became_peered":"2023-11-16T16:46:36.606744+0000","last_unstale":"2023-11-16T16:47:08.640943+0000","last_undegraded":"2023-11-16T16:47:08.640943+0000","last_fullsized":"2023-11-16T16:47:08.640943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:55:14.632895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330683+0000","last_change":"2023-11-16T16:46:37.159696+0000","last_active":"2023-11-16T16:46:52.330683+0000","last_peered":"2023-11-16T16:46:52.330683+0000","last_clean":"2023-11-16T16:46:52.330683+0000","last_became_active":"2023-11-16T16:46:37.156808+0000","last_became_peered":"2023-11-16T16:46:37.156808+0000","last_unstale":"2023-11-16T16:46:52.330683+0000","last_undegraded":"2023-11-16T16:46:52.330683+0000","last_fullsized":"2023-11-16T16:46:52.330683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:21:37.020133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335419+0000","last_change":"2023-11-16T16:46:41.204645+0000","last_active":"2023-11-16T16:46:52.335419+0000","last_peered":"2023-11-16T16:46:52.335419+0000","last_clean":"2023-11-16T16:46:52.335419+0000","last_became_active":"2023-11-16T16:46:41.204167+0000","last_became_peered":"2023-11-16T16:46:41.204167+0000","last_unstale":"2023-11-16T16:46:52.335419+0000","last_undegraded":"2023-11-16T16:46:52.335419+0000","last_fullsized":"2023-11-16T16:46:52.335419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:56:09.635720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.642622+0000","last_change":"2023-11-16T16:46:52.334958+0000","last_active":"2023-11-16T16:47:08.642622+0000","last_peered":"2023-11-16T16:47:08.642622+0000","last_clean":"2023-11-16T16:47:08.642622+0000","last_became_active":"2023-11-16T16:46:52.334607+0000","last_became_peered":"2023-11-16T16:46:52.334607+0000","last_unstale":"2023-11-16T16:47:08.642622+0000","last_undegraded":"2023-11-16T16:47:08.642622+0000","last_fullsized":"2023-11-16T16:47:08.642622+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:59:19.819869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331676+0000","last_change":"2023-11-16T16:46:37.153357+0000","last_active":"2023-11-16T16:46:52.331676+0000","last_peered":"2023-11-16T16:46:52.331676+0000","last_clean":"2023-11-16T16:46:52.331676+0000","last_became_active":"2023-11-16T16:46:37.153064+0000","last_became_peered":"2023-11-16T16:46:37.153064+0000","last_unstale":"2023-11-16T16:46:52.331676+0000","last_undegraded":"2023-11-16T16:46:52.331676+0000","last_fullsized":"2023-11-16T16:46:52.331676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:54:56.156923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331313+0000","last_change":"2023-11-16T16:46:41.183938+0000","last_active":"2023-11-16T16:46:52.331313+0000","last_peered":"2023-11-16T16:46:52.331313+0000","last_clean":"2023-11-16T16:46:52.331313+0000","last_became_active":"2023-11-16T16:46:41.181850+0000","last_became_peered":"2023-11-16T16:46:41.181850+0000","last_unstale":"2023-11-16T16:46:52.331313+0000","last_undegraded":"2023-11-16T16:46:52.331313+0000","last_fullsized":"2023-11-16T16:46:52.331313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:03:51.603935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.643670+0000","last_change":"2023-11-16T16:46:36.603257+0000","last_active":"2023-11-16T16:47:08.643670+0000","last_peered":"2023-11-16T16:47:08.643670+0000","last_clean":"2023-11-16T16:47:08.643670+0000","last_became_active":"2023-11-16T16:46:36.602852+0000","last_became_peered":"2023-11-16T16:46:36.602852+0000","last_unstale":"2023-11-16T16:47:08.643670+0000","last_undegraded":"2023-11-16T16:47:08.643670+0000","last_fullsized":"2023-11-16T16:47:08.643670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:19:39.760959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335812+0000","last_change":"2023-11-16T16:46:37.162689+0000","last_active":"2023-11-16T16:46:52.335812+0000","last_peered":"2023-11-16T16:46:52.335812+0000","last_clean":"2023-11-16T16:46:52.335812+0000","last_became_active":"2023-11-16T16:46:37.162356+0000","last_became_peered":"2023-11-16T16:46:37.162356+0000","last_unstale":"2023-11-16T16:46:52.335812+0000","last_undegraded":"2023-11-16T16:46:52.335812+0000","last_fullsized":"2023-11-16T16:46:52.335812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:07:29.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334687+0000","last_change":"2023-11-16T16:46:41.182286+0000","last_active":"2023-11-16T16:46:52.334687+0000","last_peered":"2023-11-16T16:46:52.334687+0000","last_clean":"2023-11-16T16:46:52.334687+0000","last_became_active":"2023-11-16T16:46:41.181820+0000","last_became_peered":"2023-11-16T16:46:41.181820+0000","last_unstale":"2023-11-16T16:46:52.334687+0000","last_undegraded":"2023-11-16T16:46:52.334687+0000","last_fullsized":"2023-11-16T16:46:52.334687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:56:24.412595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334978+0000","last_change":"2023-11-16T16:46:36.598952+0000","last_active":"2023-11-16T16:46:52.334978+0000","last_peered":"2023-11-16T16:46:52.334978+0000","last_clean":"2023-11-16T16:46:52.334978+0000","last_became_active":"2023-11-16T16:46:36.598617+0000","last_became_peered":"2023-11-16T16:46:36.598617+0000","last_unstale":"2023-11-16T16:46:52.334978+0000","last_undegraded":"2023-11-16T16:46:52.334978+0000","last_fullsized":"2023-11-16T16:46:52.334978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:03:48.791066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.045102+0000","last_change":"2023-11-16T16:46:37.154350+0000","last_active":"2023-11-16T16:46:53.045102+0000","last_peered":"2023-11-16T16:46:53.045102+0000","last_clean":"2023-11-16T16:46:53.045102+0000","last_became_active":"2023-11-16T16:46:37.154045+0000","last_became_peered":"2023-11-16T16:46:37.154045+0000","last_unstale":"2023-11-16T16:46:53.045102+0000","last_undegraded":"2023-11-16T16:46:53.045102+0000","last_fullsized":"2023-11-16T16:46:53.045102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:22:21.421844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334540+0000","last_change":"2023-11-16T16:46:41.202563+0000","last_active":"2023-11-16T16:46:52.334540+0000","last_peered":"2023-11-16T16:46:52.334540+0000","last_clean":"2023-11-16T16:46:52.334540+0000","last_became_active":"2023-11-16T16:46:41.202125+0000","last_became_peered":"2023-11-16T16:46:41.202125+0000","last_unstale":"2023-11-16T16:46:52.334540+0000","last_undegraded":"2023-11-16T16:46:52.334540+0000","last_fullsized":"2023-11-16T16:46:52.334540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:15:43.049582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.643140+0000","last_change":"2023-11-16T16:46:52.341548+0000","last_active":"2023-11-16T16:47:08.643140+0000","last_peered":"2023-11-16T16:47:08.643140+0000","last_clean":"2023-11-16T16:47:08.643140+0000","last_became_active":"2023-11-16T16:46:52.337771+0000","last_became_peered":"2023-11-16T16:46:52.337771+0000","last_unstale":"2023-11-16T16:47:08.643140+0000","last_undegraded":"2023-11-16T16:47:08.643140+0000","last_fullsized":"2023-11-16T16:47:08.643140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:17.479630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330523+0000","last_change":"2023-11-16T16:46:37.159095+0000","last_active":"2023-11-16T16:46:52.330523+0000","last_peered":"2023-11-16T16:46:52.330523+0000","last_clean":"2023-11-16T16:46:52.330523+0000","last_became_active":"2023-11-16T16:46:37.158668+0000","last_became_peered":"2023-11-16T16:46:37.158668+0000","last_unstale":"2023-11-16T16:46:52.330523+0000","last_undegraded":"2023-11-16T16:46:52.330523+0000","last_fullsized":"2023-11-16T16:46:52.330523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:03:24.065868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.332116+0000","last_change":"2023-11-16T16:46:41.165572+0000","last_active":"2023-11-16T16:46:52.332116+0000","last_peered":"2023-11-16T16:46:52.332116+0000","last_clean":"2023-11-16T16:46:52.332116+0000","last_became_active":"2023-11-16T16:46:41.165174+0000","last_became_peered":"2023-11-16T16:46:41.165174+0000","last_unstale":"2023-11-16T16:46:52.332116+0000","last_undegraded":"2023-11-16T16:46:52.332116+0000","last_fullsized":"2023-11-16T16:46:52.332116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:54:13.439807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.337265+0000","last_change":"2023-11-16T16:46:36.593871+0000","last_active":"2023-11-16T16:46:52.337265+0000","last_peered":"2023-11-16T16:46:52.337265+0000","last_clean":"2023-11-16T16:46:52.337265+0000","last_became_active":"2023-11-16T16:46:36.593579+0000","last_became_peered":"2023-11-16T16:46:36.593579+0000","last_unstale":"2023-11-16T16:46:52.337265+0000","last_undegraded":"2023-11-16T16:46:52.337265+0000","last_fullsized":"2023-11-16T16:46:52.337265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:11:16.979596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331375+0000","last_change":"2023-11-16T16:46:37.140823+0000","last_active":"2023-11-16T16:46:52.331375+0000","last_peered":"2023-11-16T16:46:52.331375+0000","last_clean":"2023-11-16T16:46:52.331375+0000","last_became_active":"2023-11-16T16:46:37.140505+0000","last_became_peered":"2023-11-16T16:46:37.140505+0000","last_unstale":"2023-11-16T16:46:52.331375+0000","last_undegraded":"2023-11-16T16:46:52.331375+0000","last_fullsized":"2023-11-16T16:46:52.331375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:46:42.999149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.329464+0000","last_change":"2023-11-16T16:46:41.182041+0000","last_active":"2023-11-16T16:46:52.329464+0000","last_peered":"2023-11-16T16:46:52.329464+0000","last_clean":"2023-11-16T16:46:52.329464+0000","last_became_active":"2023-11-16T16:46:41.181420+0000","last_became_peered":"2023-11-16T16:46:41.181420+0000","last_unstale":"2023-11-16T16:46:52.329464+0000","last_undegraded":"2023-11-16T16:46:52.329464+0000","last_fullsized":"2023-11-16T16:46:52.329464+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:39:34.693599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.328130+0000","last_change":"2023-11-16T16:46:36.602184+0000","last_active":"2023-11-16T16:46:52.328130+0000","last_peered":"2023-11-16T16:46:52.328130+0000","last_clean":"2023-11-16T16:46:52.328130+0000","last_became_active":"2023-11-16T16:46:36.601893+0000","last_became_peered":"2023-11-16T16:46:36.601893+0000","last_unstale":"2023-11-16T16:46:52.328130+0000","last_undegraded":"2023-11-16T16:46:52.328130+0000","last_fullsized":"2023-11-16T16:46:52.328130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:07:44.033216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330760+0000","last_change":"2023-11-16T16:46:37.153649+0000","last_active":"2023-11-16T16:46:52.330760+0000","last_peered":"2023-11-16T16:46:52.330760+0000","last_clean":"2023-11-16T16:46:52.330760+0000","last_became_active":"2023-11-16T16:46:37.152420+0000","last_became_peered":"2023-11-16T16:46:37.152420+0000","last_unstale":"2023-11-16T16:46:52.330760+0000","last_undegraded":"2023-11-16T16:46:52.330760+0000","last_fullsized":"2023-11-16T16:46:52.330760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:26:13.077534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327051+0000","last_change":"2023-11-16T16:46:41.168363+0000","last_active":"2023-11-16T16:46:52.327051+0000","last_peered":"2023-11-16T16:46:52.327051+0000","last_clean":"2023-11-16T16:46:52.327051+0000","last_became_active":"2023-11-16T16:46:41.167615+0000","last_became_peered":"2023-11-16T16:46:41.167615+0000","last_unstale":"2023-11-16T16:46:52.327051+0000","last_undegraded":"2023-11-16T16:46:52.327051+0000","last_fullsized":"2023-11-16T16:46:52.327051+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:39:53.383890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426560+0000","last_change":"2023-11-16T16:46:36.611353+0000","last_active":"2023-11-16T16:46:52.426560+0000","last_peered":"2023-11-16T16:46:52.426560+0000","last_clean":"2023-11-16T16:46:52.426560+0000","last_became_active":"2023-11-16T16:46:36.610886+0000","last_became_peered":"2023-11-16T16:46:36.610886+0000","last_unstale":"2023-11-16T16:46:52.426560+0000","last_undegraded":"2023-11-16T16:46:52.426560+0000","last_fullsized":"2023-11-16T16:46:52.426560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:10:20.922368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331449+0000","last_change":"2023-11-16T16:46:37.146227+0000","last_active":"2023-11-16T16:46:52.331449+0000","last_peered":"2023-11-16T16:46:52.331449+0000","last_clean":"2023-11-16T16:46:52.331449+0000","last_became_active":"2023-11-16T16:46:37.143239+0000","last_became_peered":"2023-11-16T16:46:37.143239+0000","last_unstale":"2023-11-16T16:46:52.331449+0000","last_undegraded":"2023-11-16T16:46:52.331449+0000","last_fullsized":"2023-11-16T16:46:52.331449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:44:19.791930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335938+0000","last_change":"2023-11-16T16:46:37.153845+0000","last_active":"2023-11-16T16:46:52.335938+0000","last_peered":"2023-11-16T16:46:52.335938+0000","last_clean":"2023-11-16T16:46:52.335938+0000","last_became_active":"2023-11-16T16:46:37.153505+0000","last_became_peered":"2023-11-16T16:46:37.153505+0000","last_unstale":"2023-11-16T16:46:52.335938+0000","last_undegraded":"2023-11-16T16:46:52.335938+0000","last_fullsized":"2023-11-16T16:46:52.335938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:26:52.188071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335325+0000","last_change":"2023-11-16T16:46:37.148451+0000","last_active":"2023-11-16T16:46:52.335325+0000","last_peered":"2023-11-16T16:46:52.335325+0000","last_clean":"2023-11-16T16:46:52.335325+0000","last_became_active":"2023-11-16T16:46:37.148142+0000","last_became_peered":"2023-11-16T16:46:37.148142+0000","last_unstale":"2023-11-16T16:46:52.335325+0000","last_undegraded":"2023-11-16T16:46:52.335325+0000","last_fullsized":"2023-11-16T16:46:52.335325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:05:12.660394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334810+0000","last_change":"2023-11-16T16:46:37.150360+0000","last_active":"2023-11-16T16:46:52.334810+0000","last_peered":"2023-11-16T16:46:52.334810+0000","last_clean":"2023-11-16T16:46:52.334810+0000","last_became_active":"2023-11-16T16:46:37.150045+0000","last_became_peered":"2023-11-16T16:46:37.150045+0000","last_unstale":"2023-11-16T16:46:52.334810+0000","last_undegraded":"2023-11-16T16:46:52.334810+0000","last_fullsized":"2023-11-16T16:46:52.334810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:54:53.279485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.329861+0000","last_change":"2023-11-16T16:46:37.161736+0000","last_active":"2023-11-16T16:46:52.329861+0000","last_peered":"2023-11-16T16:46:52.329861+0000","last_clean":"2023-11-16T16:46:52.329861+0000","last_became_active":"2023-11-16T16:46:37.158273+0000","last_became_peered":"2023-11-16T16:46:37.158273+0000","last_unstale":"2023-11-16T16:46:52.329861+0000","last_undegraded":"2023-11-16T16:46:52.329861+0000","last_fullsized":"2023-11-16T16:46:52.329861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:38:18.599194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426259+0000","last_change":"2023-11-16T16:46:37.163725+0000","last_active":"2023-11-16T16:46:52.426259+0000","last_peered":"2023-11-16T16:46:52.426259+0000","last_clean":"2023-11-16T16:46:52.426259+0000","last_became_active":"2023-11-16T16:46:37.163180+0000","last_became_peered":"2023-11-16T16:46:37.163180+0000","last_unstale":"2023-11-16T16:46:52.426259+0000","last_undegraded":"2023-11-16T16:46:52.426259+0000","last_fullsized":"2023-11-16T16:46:52.426259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:57:30.898710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335260+0000","last_change":"2023-11-16T16:46:37.154475+0000","last_active":"2023-11-16T16:46:52.335260+0000","last_peered":"2023-11-16T16:46:52.335260+0000","last_clean":"2023-11-16T16:46:52.335260+0000","last_became_active":"2023-11-16T16:46:37.154161+0000","last_became_peered":"2023-11-16T16:46:37.154161+0000","last_unstale":"2023-11-16T16:46:52.335260+0000","last_undegraded":"2023-11-16T16:46:52.335260+0000","last_fullsized":"2023-11-16T16:46:52.335260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:28.955298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336724+0000","last_change":"2023-11-16T16:46:37.140293+0000","last_active":"2023-11-16T16:46:52.336724+0000","last_peered":"2023-11-16T16:46:52.336724+0000","last_clean":"2023-11-16T16:46:52.336724+0000","last_became_active":"2023-11-16T16:46:37.139966+0000","last_became_peered":"2023-11-16T16:46:37.139966+0000","last_unstale":"2023-11-16T16:46:52.336724+0000","last_undegraded":"2023-11-16T16:46:52.336724+0000","last_fullsized":"2023-11-16T16:46:52.336724+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:02:02.311770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.045428+0000","last_change":"2023-11-16T16:46:37.134807+0000","last_active":"2023-11-16T16:46:53.045428+0000","last_peered":"2023-11-16T16:46:53.045428+0000","last_clean":"2023-11-16T16:46:53.045428+0000","last_became_active":"2023-11-16T16:46:37.131833+0000","last_became_peered":"2023-11-16T16:46:37.131833+0000","last_unstale":"2023-11-16T16:46:53.045428+0000","last_undegraded":"2023-11-16T16:46:53.045428+0000","last_fullsized":"2023-11-16T16:46:53.045428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:05:04.767346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327181+0000","last_change":"2023-11-16T16:46:37.152424+0000","last_active":"2023-11-16T16:46:52.327181+0000","last_peered":"2023-11-16T16:46:52.327181+0000","last_clean":"2023-11-16T16:46:52.327181+0000","last_became_active":"2023-11-16T16:46:37.152082+0000","last_became_peered":"2023-11-16T16:46:37.152082+0000","last_unstale":"2023-11-16T16:46:52.327181+0000","last_undegraded":"2023-11-16T16:46:52.327181+0000","last_fullsized":"2023-11-16T16:46:52.327181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:14:36.269833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044568+0000","last_change":"2023-11-16T16:46:37.140267+0000","last_active":"2023-11-16T16:46:53.044568+0000","last_peered":"2023-11-16T16:46:53.044568+0000","last_clean":"2023-11-16T16:46:53.044568+0000","last_became_active":"2023-11-16T16:46:37.139689+0000","last_became_peered":"2023-11-16T16:46:37.139689+0000","last_unstale":"2023-11-16T16:46:53.044568+0000","last_undegraded":"2023-11-16T16:46:53.044568+0000","last_fullsized":"2023-11-16T16:46:53.044568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:55:24.624825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.332484+0000","last_change":"2023-11-16T16:46:37.151911+0000","last_active":"2023-11-16T16:46:52.332484+0000","last_peered":"2023-11-16T16:46:52.332484+0000","last_clean":"2023-11-16T16:46:52.332484+0000","last_became_active":"2023-11-16T16:46:37.151614+0000","last_became_peered":"2023-11-16T16:46:37.151614+0000","last_unstale":"2023-11-16T16:46:52.332484+0000","last_undegraded":"2023-11-16T16:46:52.332484+0000","last_fullsized":"2023-11-16T16:46:52.332484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:04:10.208002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426197+0000","last_change":"2023-11-16T16:46:37.153265+0000","last_active":"2023-11-16T16:46:52.426197+0000","last_peered":"2023-11-16T16:46:52.426197+0000","last_clean":"2023-11-16T16:46:52.426197+0000","last_became_active":"2023-11-16T16:46:37.152933+0000","last_became_peered":"2023-11-16T16:46:37.152933+0000","last_unstale":"2023-11-16T16:46:52.426197+0000","last_undegraded":"2023-11-16T16:46:52.426197+0000","last_fullsized":"2023-11-16T16:46:52.426197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:23:01.775314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336215+0000","last_change":"2023-11-16T16:46:37.142831+0000","last_active":"2023-11-16T16:46:52.336215+0000","last_peered":"2023-11-16T16:46:52.336215+0000","last_clean":"2023-11-16T16:46:52.336215+0000","last_became_active":"2023-11-16T16:46:37.142480+0000","last_became_peered":"2023-11-16T16:46:37.142480+0000","last_unstale":"2023-11-16T16:46:52.336215+0000","last_undegraded":"2023-11-16T16:46:52.336215+0000","last_fullsized":"2023-11-16T16:46:52.336215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:09:49.039282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7116,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":109,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":35351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6176,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93738976,"statfs":{"total":95995035648,"available":95988711424,"internally_reserved":0,"allocated":1007616,"data_stored":493331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44800000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.65300000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":0.73999999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72699999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.75}]},{"osd":10,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.63300000000000001}]}]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6900,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738252,"statfs":{"total":95995035648,"available":95987970048,"internally_reserved":0,"allocated":1617920,"data_stored":1083009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":1,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":4.1299999999999999,"5min":4.1299999999999999,"15min":4.1299999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":4.1769999999999996,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.56599999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.78000000000000003}]},{"osd":9,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.879},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.81799999999999995}]},{"osd":11,"last update":"Thu Nov 16 16:46:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.85599999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6384,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738768,"statfs":{"total":95995035648,"available":95988498432,"internally_reserved":0,"allocated":1024000,"data_stored":493567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.55100000000000005,"15min":0.56399999999999995},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72599999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.435},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.749,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.54900000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.495}]},{"osd":2,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72999999999999998,"15min":0.73899999999999999},"min":{"1min":0.47899999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96999999999999997,"5min":1.048,"15min":1.048},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.85899999999999999,"15min":0.89100000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98699999999999999,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67900000000000005,"15min":0.68999999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78000000000000003,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.79100000000000004,"15min":0.80900000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.056,"5min":1.24,"15min":1.24},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.82399999999999995,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1120000000000001,"5min":1.216,"15min":1.216},"last":0.84199999999999997}]},{"osd":5,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.82199999999999995,"15min":0.84399999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95899999999999996,"5min":1.304,"15min":1.304},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.86799999999999999,"15min":0.88600000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.145,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.88200000000000001}]},{"osd":6,"last update":"Thu Nov 16 16:46:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.92500000000000004},"min":{"1min":0.61899999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.111,"5min":1.282,"15min":1.282},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.98899999999999999,"15min":1.0229999999999999},"min":{"1min":0.57799999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0860000000000001,"5min":4.5650000000000004,"15min":4.5650000000000004},"last":0.86199999999999999}]},{"osd":8,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1,"15min":1.0389999999999999},"min":{"1min":0.55300000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.163,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.93799999999999994,"15min":0.96699999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.077,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.82699999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":4.4870000000000001,"5min":4.4870000000000001,"15min":4.4870000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.78700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":8,"up_from":59,"seq":253403070498,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6504,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738648,"statfs":{"total":95995035648,"available":95988375552,"internally_reserved":0,"allocated":1015808,"data_stored":489060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.498},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.59799999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.753,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71799999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.46400000000000002,"15min":0.47399999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.66900000000000004,"15min":0.68700000000000006},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81499999999999995,"5min":1.536,"15min":1.536},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.68200000000000005,"15min":0.70099999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78800000000000003,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82599999999999996,"5min":0.88,"15min":0.88},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.55200000000000005,"15min":0.56599999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73899999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84599999999999997,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.72699999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.74299999999999999,"15min":0.76700000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.871,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93100000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.80300000000000005}]},{"osd":9,"last update":"Thu Nov 16 16:47:03 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.79100000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.874}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7068,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738084,"statfs":{"total":95995035648,"available":95987798016,"internally_reserved":0,"allocated":1593344,"data_stored":1079272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.27100000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.60199999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68799999999999994,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.89500000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66700000000000004}]},{"osd":4,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.30399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.62},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84599999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81999999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71099999999999997,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.497,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88300000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65400000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.51700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.88900000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90600000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.60899999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93600000000000005,"5min":1.054,"15min":1.054},"last":1.0620000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.95499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6428,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738724,"statfs":{"total":95995035648,"available":95988453376,"internally_reserved":0,"allocated":1003520,"data_stored":488743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.36599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71999999999999997,"5min":0.87,"15min":0.87},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.36199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78300000000000003,"5min":1.139,"15min":1.139},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89000000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85299999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63,"15min":0.622},"min":{"1min":0.501,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.753,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.67500000000000004,"15min":0.69499999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":1.25,"15min":1.25},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.96699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.625,"15min":0.629},"min":{"1min":0.40600000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72499999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.53100000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89300000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73499999999999999,"15min":0.749},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88800000000000001,"5min":1.343,"15min":1.343},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.54600000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.91300000000000003,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.74299999999999999,"15min":0.73799999999999999},"min":{"1min":0.51600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.93500000000000005,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.73999999999999999,"15min":0.73299999999999998},"min":{"1min":0.57099999999999995,"5min":0.438,"15min":0.438},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.73799999999999999,"15min":0.72599999999999998},"min":{"1min":0.53800000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.90000000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.79000000000000004,"15min":0.79400000000000004},"min":{"1min":0.53600000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.0920000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79700000000000004,"15min":0.79900000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.97699999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.86699999999999999,"15min":0.877},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.089,"5min":1.165,"15min":1.165},"last":0.872},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84299999999999997,"15min":0.84699999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.097,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.88300000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.92700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7396,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737756,"statfs":{"total":95995035648,"available":95987462144,"internally_reserved":0,"allocated":1011712,"data_stored":492447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52200000000000002,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59899999999999998,"5min":1.431,"15min":1.431},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57299999999999995,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76500000000000001,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Nov 16 16:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.41299999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71199999999999997}]},{"osd":3,"last update":"Thu Nov 16 16:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.54500000000000004},"min":{"1min":0.41499999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83699999999999997,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59099999999999997,"15min":0.56999999999999995},"min":{"1min":0.46500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.89800000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72999999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.40899999999999997,"15min":0.39200000000000002},"min":{"1min":0.32200000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.432,"15min":0.40300000000000002},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.55900000000000005,"15min":0.52800000000000002},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.91400000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.58399999999999996,"15min":0.54600000000000004},"min":{"1min":0.60399999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92900000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.78000000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.58999999999999997},"min":{"1min":0.48199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85899999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.626,"15min":0.60199999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.48399999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.55700000000000005},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.44700000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72699999999999998,"15min":0.72299999999999998},"min":{"1min":0.51800000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94399999999999995,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.54900000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65900000000000003}]},{"osd":9,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94299999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.036,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.59799999999999998}]},{"osd":10,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.61599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7388,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737764,"statfs":{"total":95995035648,"available":95987470336,"internally_reserved":0,"allocated":1003520,"data_stored":488743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89100000000000001,"5min":1.01,"15min":1.01},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.61699999999999999,"15min":0.55900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.752}]},{"osd":2,"last update":"Thu Nov 16 16:46:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.58399999999999996,"15min":0.54600000000000004},"min":{"1min":0.44,"5min":0.373,"15min":0.373},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.55300000000000005},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92600000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.50900000000000001,"15min":0.46999999999999997},"min":{"1min":0.33300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.54200000000000004,"15min":0.51900000000000002},"min":{"1min":0.39100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77300000000000002,"5min":0.878,"15min":0.878},"last":0.72299999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.626,"15min":0.60299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60499999999999998,"15min":0.57899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96899999999999997,"5min":1.008,"15min":1.008},"last":0.877}]},{"osd":6,"last update":"Thu Nov 16 16:46:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.90900000000000003,"5min":1.03,"15min":1.03},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Nov 16 16:46:16 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.94199999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73899999999999999,"15min":0.73999999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.79100000000000004}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.79000000000000004,"15min":0.80500000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81100000000000005,"15min":0.81299999999999994},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.121,"5min":1.145,"15min":1.145},"last":1.0780000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":1.167}]},{"osd":11,"last update":"Thu Nov 16 16:46:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":1.214},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.133}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7400,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737752,"statfs":{"total":95995035648,"available":95987458048,"internally_reserved":0,"allocated":1015808,"data_stored":492481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.55600000000000005},"min":{"1min":0.40899999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70699999999999996,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56000000000000005,"15min":0.54000000000000004},"min":{"1min":0.378,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67800000000000005,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Nov 16 16:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71599999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.52500000000000002},"min":{"1min":0.48799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.69099999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.66700000000000004},"min":{"1min":0.44800000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.5509999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61499999999999999,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:46:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61399999999999999,"15min":0.63300000000000001},"min":{"1min":0.42199999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.73699999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.68100000000000005,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81200000000000006,"5min":4.181,"15min":4.181},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Nov 16 16:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45900000000000002,"15min":0.44600000000000001},"min":{"1min":0.33500000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78700000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.629},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.47499999999999998,"15min":0.46400000000000002},"min":{"1min":0.32100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67400000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67500000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64200000000000002,"15min":0.622},"min":{"1min":0.47499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.441,"5min":0.437,"15min":0.437},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.48499999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.84499999999999997,"5min":0.876,"15min":0.876},"last":0.76900000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64200000000000002,"15min":0.64500000000000002},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.88700000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.748},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.624,"15min":0.627},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81299999999999994,"5min":0.872,"15min":0.872},"last":0.84999999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.745,"15min":0.72999999999999998},"min":{"1min":0.63900000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.77500000000000002,"15min":0.76100000000000001},"min":{"1min":0.59099999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.95299999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.97799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6756,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738396,"statfs":{"total":95995035648,"available":95988117504,"internally_reserved":0,"allocated":1011712,"data_stored":488847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.42299999999999999,"15min":0.38600000000000001},"min":{"1min":0.30399999999999999,"5min":0.246,"15min":0.246},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.748},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.502,"15min":0.46500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.80400000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61399999999999999,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60099999999999998,"15min":0.57399999999999995},"min":{"1min":0.497,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Nov 16 16:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.64400000000000002,"15min":0.61599999999999999},"min":{"1min":0.47399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.016,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.621},"min":{"1min":0.443,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.98099999999999998,"5min":1.228,"15min":1.228},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.624,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.503}]},{"osd":5,"last update":"Thu Nov 16 16:46:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85699999999999998,"5min":1.133,"15min":1.133},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95399999999999996,"5min":1.149,"15min":1.149},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Nov 16 16:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.77100000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68999999999999995,"15min":0.68500000000000005},"min":{"1min":0.45400000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.84099999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.76700000000000002,"15min":0.76400000000000001},"min":{"1min":0.501,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.073,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.82799999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.753,"15min":0.753},"min":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1559999999999999,"5min":1.171,"15min":1.171},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.60699999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.306}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.871},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81599999999999995,"15min":0.81899999999999995},"min":{"1min":0.64500000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.84999999999999998}]},{"osd":10,"last update":"Thu Nov 16 16:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.61799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6628,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738524,"statfs":{"total":95995035648,"available":95988248576,"internally_reserved":0,"allocated":1011712,"data_stored":493404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.52500000000000002,"15min":0.501},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.377,"15min":0.34799999999999998},"min":{"1min":0.33700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.38},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.35199999999999998},"min":{"1min":0.26500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.5,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.36799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.754,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.53900000000000003,"15min":0.51000000000000001},"min":{"1min":0.45200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72599999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58399999999999996,"15min":0.53800000000000003},"min":{"1min":0.44400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75700000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.623},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82399999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54200000000000004,"15min":0.53100000000000003},"min":{"1min":0.41999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57799999999999996,"15min":0.55800000000000005},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Nov 16 16:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.40899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83699999999999997,"5min":1.097,"15min":1.097},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.496,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.60799999999999998,"15min":0.59199999999999997},"min":{"1min":0.47899999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.76900000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.498,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.68200000000000005,"15min":0.66600000000000004},"min":{"1min":0.54900000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85999999999999999}]},{"osd":9,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.80000000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.92900000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73099999999999998,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.878,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54600000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:46:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7284,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737868,"statfs":{"total":95995035648,"available":95987576832,"internally_reserved":0,"allocated":1617920,"data_stored":1080118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57399999999999995,"15min":0.56299999999999994},"min":{"1min":0.42699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77200000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.42899999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73199999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.503},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.755}]},{"osd":2,"last update":"Thu Nov 16 16:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.38900000000000001},"min":{"1min":0.308,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66700000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.40999999999999998,"15min":0.38800000000000001},"min":{"1min":0.28899999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79000000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.44,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57499999999999996,"15min":0.54700000000000004},"min":{"1min":0.46100000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.56000000000000005,"15min":0.54500000000000004},"min":{"1min":0.38400000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64900000000000002,"15min":0.63500000000000001},"min":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67300000000000004,"15min":0.65500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.57399999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.44400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.69099999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.67500000000000004}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.75,"15min":0.746},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.74099999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:46:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":21175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:47:13.659 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-16T16:47:13.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.886+0000 7fa331dff700 1 -- 172.21.15.93:0/159557591 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa32c074b30 msgr2=0x7fa32c072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:13.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.886+0000 7fa331dff700 1 --2- 172.21.15.93:0/159557591 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa32c074b30 0x7fa32c072f80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009990 tx=0x7fa31c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.887+0000 7fa331dff700 1 -- 172.21.15.93:0/159557591 shutdown_connections 2023-11-16T16:47:13.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.887+0000 7fa331dff700 1 --2- 172.21.15.93:0/159557591 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa32c074b30 0x7fa32c072f80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.887+0000 7fa331dff700 1 --2- 172.21.15.93:0/159557591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32c108150 0x7fa32c10a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.887+0000 7fa331dff700 1 --2- 172.21.15.93:0/159557591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa32c073550 0x7fa32c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.887+0000 7fa331dff700 1 -- 172.21.15.93:0/159557591 >> 172.21.15.93:0/159557591 conn(0x7fa32c0fb870 msgr2=0x7fa32c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:13.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.887+0000 7fa331dff700 1 -- 172.21.15.93:0/159557591 shutdown_connections 2023-11-16T16:47:13.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.887+0000 7fa331dff700 1 -- 172.21.15.93:0/159557591 wait complete. 2023-11-16T16:47:13.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa331dff700 1 Processor -- start 2023-11-16T16:47:13.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa331dff700 1 -- start start 2023-11-16T16:47:13.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa331dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa32c073550 0x7fa32c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:13.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa331dff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa32c074b30 0x7fa32c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:13.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa331dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32c108150 0x7fa32c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:13.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa331dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c19fda0 con 0x7fa32c073550 2023-11-16T16:47:13.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa331dff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c19fee0 con 0x7fa32c074b30 2023-11-16T16:47:13.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa331dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c1a0070 con 0x7fa32c108150 2023-11-16T16:47:13.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa32b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa32c073550 0x7fa32c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:13.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa32b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa32c073550 0x7fa32c198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42578/0 (socket says 172.21.15.93:42578) 2023-11-16T16:47:13.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa32b7fe700 1 -- 172.21.15.93:0/365728626 learned_addr learned my addr 172.21.15.93:0/365728626 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:13.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa32bfff700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32c108150 0x7fa32c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:13.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.888+0000 7fa323fff700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa32c074b30 0x7fa32c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:13.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.889+0000 7fa32b7fe700 1 -- 172.21.15.93:0/365728626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32c108150 msgr2=0x7fa32c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:13.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.889+0000 7fa32b7fe700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32c108150 0x7fa32c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.889+0000 7fa32b7fe700 1 -- 172.21.15.93:0/365728626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa32c074b30 msgr2=0x7fa32c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:13.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.889+0000 7fa32b7fe700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa32c074b30 0x7fa32c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:13.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.889+0000 7fa32b7fe700 1 -- 172.21.15.93:0/365728626 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c0095f0 con 0x7fa32c073550 2023-11-16T16:47:13.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.889+0000 7fa32b7fe700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa32c073550 0x7fa32c198220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00b680 tx=0x7fa31c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:13.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.889+0000 7fa3297fa700 1 -- 172.21.15.93:0/365728626 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c01d070 con 0x7fa32c073550 2023-11-16T16:47:13.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.889+0000 7fa3297fa700 1 -- 172.21.15.93:0/365728626 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa31c021410 con 0x7fa32c073550 2023-11-16T16:47:13.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.889+0000 7fa331dff700 1 -- 172.21.15.93:0/365728626 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c10de50 con 0x7fa32c073550 2023-11-16T16:47:13.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.889+0000 7fa3297fa700 1 -- 172.21.15.93:0/365728626 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c017650 con 0x7fa32c073550 2023-11-16T16:47:13.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.889+0000 7fa331dff700 1 -- 172.21.15.93:0/365728626 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c10dcc0 con 0x7fa32c073550 2023-11-16T16:47:13.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.890+0000 7fa331dff700 1 -- 172.21.15.93:0/365728626 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c106800 con 0x7fa32c073550 2023-11-16T16:47:13.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.892+0000 7fa3297fa700 1 -- 172.21.15.93:0/365728626 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa31c021580 con 0x7fa32c073550 2023-11-16T16:47:13.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.894+0000 7fa3297fa700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa3180a1260 0x7fa3180a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:13.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.895+0000 7fa3297fa700 1 -- 172.21.15.93:0/365728626 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa31c0ce1c0 con 0x7fa32c073550 2023-11-16T16:47:13.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.895+0000 7fa323fff700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa3180a1260 0x7fa3180a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:13.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.895+0000 7fa323fff700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa3180a1260 0x7fa3180a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa32c074850 tx=0x7fa314007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:13 smithi093 ceph-mon[97863]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:47:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:13 smithi093 ceph-mon[97863]: pgmap v301: 129 pgs: 129 active+clean; 584 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2023-11-16T16:47:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:13 smithi093 ceph-mon[97863]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-11-16T16:47:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:13 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2849243515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:47:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:13 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/180988935' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:13.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:13.899+0000 7fa3297fa700 1 -- 172.21.15.93:0/365728626 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa31c0974e0 con 0x7fa32c073550 2023-11-16T16:47:13.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:13 smithi159 ceph-mon[131528]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:47:13.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:13 smithi159 ceph-mon[131528]: pgmap v301: 129 pgs: 129 active+clean; 584 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2023-11-16T16:47:13.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:13 smithi159 ceph-mon[131528]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-11-16T16:47:13.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:13 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2849243515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:47:13.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:13 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/180988935' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:14.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:13 smithi125 ceph-mon[131076]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:47:14.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:13 smithi125 ceph-mon[131076]: pgmap v301: 129 pgs: 129 active+clean; 584 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2023-11-16T16:47:14.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:13 smithi125 ceph-mon[131076]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-11-16T16:47:14.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:13 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2849243515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:47:14.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:13 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/180988935' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:14.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.094+0000 7fa331dff700 1 -- 172.21.15.93:0/365728626 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa32c061cb0 con 0x7fa3180a1260 2023-11-16T16:47:14.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.107+0000 7fa3297fa700 1 -- 172.21.15.93:0/365728626 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422386 (secure 0 0 0) 0x7fa32c061cb0 con 0x7fa3180a1260 2023-11-16T16:47:14.115 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:47:14.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.113+0000 7fa331dff700 1 -- 172.21.15.93:0/365728626 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa3180a1260 msgr2=0x7fa3180a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:14.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.113+0000 7fa331dff700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa3180a1260 0x7fa3180a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa32c074850 tx=0x7fa314007400 comp rx=0 tx=0).stop 2023-11-16T16:47:14.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.113+0000 7fa331dff700 1 -- 172.21.15.93:0/365728626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa32c073550 msgr2=0x7fa32c198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:14.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.113+0000 7fa331dff700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa32c073550 0x7fa32c198220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00b680 tx=0x7fa31c005740 comp rx=0 tx=0).stop 2023-11-16T16:47:14.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.113+0000 7fa331dff700 1 -- 172.21.15.93:0/365728626 shutdown_connections 2023-11-16T16:47:14.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.113+0000 7fa331dff700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa3180a1260 0x7fa3180a3710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:14.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.113+0000 7fa331dff700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa32c073550 0x7fa32c198220 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:14.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.113+0000 7fa331dff700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32c108150 0x7fa32c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:14.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.113+0000 7fa331dff700 1 --2- 172.21.15.93:0/365728626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa32c074b30 0x7fa32c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:14.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.113+0000 7fa331dff700 1 -- 172.21.15.93:0/365728626 >> 172.21.15.93:0/365728626 conn(0x7fa32c0fb870 msgr2=0x7fa32c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:14.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.113+0000 7fa331dff700 1 -- 172.21.15.93:0/365728626 shutdown_connections 2023-11-16T16:47:14.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:14.113+0000 7fa331dff700 1 -- 172.21.15.93:0/365728626 wait complete. 2023-11-16T16:47:14.120 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-11-16T16:47:14.133 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2023-11-16T16:47:12.522093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597607,"num_objects":54,"num_object_clones":0,"num_object_copies":163,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":142,"num_read_kb":123,"num_write":291,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":82312,"kb_used_data":13608,"kb_used_omap":0,"kb_used_meta":68608,"kb_avail":1124859512,"statfs":{"total":1151940427776,"available":1151856140288,"internally_reserved":0,"allocated":13934592,"data_stored":7663022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70254592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2788,"num_objects":30,"num_object_clones":0,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":62,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003725"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336272+0000","last_change":"2023-11-16T16:46:37.150429+0000","last_active":"2023-11-16T16:46:52.336272+0000","last_peered":"2023-11-16T16:46:52.336272+0000","last_clean":"2023-11-16T16:46:52.336272+0000","last_became_active":"2023-11-16T16:46:37.150133+0000","last_became_peered":"2023-11-16T16:46:37.150133+0000","last_unstale":"2023-11-16T16:46:52.336272+0000","last_undegraded":"2023-11-16T16:46:52.336272+0000","last_fullsized":"2023-11-16T16:46:52.336272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:23:35.649395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330560+0000","last_change":"2023-11-16T16:46:37.155729+0000","last_active":"2023-11-16T16:46:52.330560+0000","last_peered":"2023-11-16T16:46:52.330560+0000","last_clean":"2023-11-16T16:46:52.330560+0000","last_became_active":"2023-11-16T16:46:37.155178+0000","last_became_peered":"2023-11-16T16:46:37.155178+0000","last_unstale":"2023-11-16T16:46:52.330560+0000","last_undegraded":"2023-11-16T16:46:52.330560+0000","last_fullsized":"2023-11-16T16:46:52.330560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:32:07.453061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330033+0000","last_change":"2023-11-16T16:46:37.154669+0000","last_active":"2023-11-16T16:46:52.330033+0000","last_peered":"2023-11-16T16:46:52.330033+0000","last_clean":"2023-11-16T16:46:52.330033+0000","last_became_active":"2023-11-16T16:46:37.154331+0000","last_became_peered":"2023-11-16T16:46:37.154331+0000","last_unstale":"2023-11-16T16:46:52.330033+0000","last_undegraded":"2023-11-16T16:46:52.330033+0000","last_fullsized":"2023-11-16T16:46:52.330033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:18:33.416725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335112+0000","last_change":"2023-11-16T16:46:37.140627+0000","last_active":"2023-11-16T16:46:52.335112+0000","last_peered":"2023-11-16T16:46:52.335112+0000","last_clean":"2023-11-16T16:46:52.335112+0000","last_became_active":"2023-11-16T16:46:37.138386+0000","last_became_peered":"2023-11-16T16:46:37.138386+0000","last_unstale":"2023-11-16T16:46:52.335112+0000","last_undegraded":"2023-11-16T16:46:52.335112+0000","last_fullsized":"2023-11-16T16:46:52.335112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:47:01.255110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.332315+0000","last_change":"2023-11-16T16:46:37.154046+0000","last_active":"2023-11-16T16:46:52.332315+0000","last_peered":"2023-11-16T16:46:52.332315+0000","last_clean":"2023-11-16T16:46:52.332315+0000","last_became_active":"2023-11-16T16:46:37.153677+0000","last_became_peered":"2023-11-16T16:46:37.153677+0000","last_unstale":"2023-11-16T16:46:52.332315+0000","last_undegraded":"2023-11-16T16:46:52.332315+0000","last_fullsized":"2023-11-16T16:46:52.332315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:13:01.802905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044944+0000","last_change":"2023-11-16T16:46:37.153809+0000","last_active":"2023-11-16T16:46:53.044944+0000","last_peered":"2023-11-16T16:46:53.044944+0000","last_clean":"2023-11-16T16:46:53.044944+0000","last_became_active":"2023-11-16T16:46:37.153511+0000","last_became_peered":"2023-11-16T16:46:37.153511+0000","last_unstale":"2023-11-16T16:46:53.044944+0000","last_undegraded":"2023-11-16T16:46:53.044944+0000","last_fullsized":"2023-11-16T16:46:53.044944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:11:54.236854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330213+0000","last_change":"2023-11-16T16:46:37.142530+0000","last_active":"2023-11-16T16:46:52.330213+0000","last_peered":"2023-11-16T16:46:52.330213+0000","last_clean":"2023-11-16T16:46:52.330213+0000","last_became_active":"2023-11-16T16:46:37.142216+0000","last_became_peered":"2023-11-16T16:46:37.142216+0000","last_unstale":"2023-11-16T16:46:52.330213+0000","last_undegraded":"2023-11-16T16:46:52.330213+0000","last_fullsized":"2023-11-16T16:46:52.330213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:02:25.342439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044675+0000","last_change":"2023-11-16T16:46:37.151956+0000","last_active":"2023-11-16T16:46:53.044675+0000","last_peered":"2023-11-16T16:46:53.044675+0000","last_clean":"2023-11-16T16:46:53.044675+0000","last_became_active":"2023-11-16T16:46:37.149432+0000","last_became_peered":"2023-11-16T16:46:37.149432+0000","last_unstale":"2023-11-16T16:46:53.044675+0000","last_undegraded":"2023-11-16T16:46:53.044675+0000","last_fullsized":"2023-11-16T16:46:53.044675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:06:52.090307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333580+0000","last_change":"2023-11-16T16:46:37.148378+0000","last_active":"2023-11-16T16:46:52.333580+0000","last_peered":"2023-11-16T16:46:52.333580+0000","last_clean":"2023-11-16T16:46:52.333580+0000","last_became_active":"2023-11-16T16:46:37.148082+0000","last_became_peered":"2023-11-16T16:46:37.148082+0000","last_unstale":"2023-11-16T16:46:52.333580+0000","last_undegraded":"2023-11-16T16:46:52.333580+0000","last_fullsized":"2023-11-16T16:46:52.333580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:58:05.786729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.045350+0000","last_change":"2023-11-16T16:46:37.153895+0000","last_active":"2023-11-16T16:46:53.045350+0000","last_peered":"2023-11-16T16:46:53.045350+0000","last_clean":"2023-11-16T16:46:53.045350+0000","last_became_active":"2023-11-16T16:46:37.153491+0000","last_became_peered":"2023-11-16T16:46:37.153491+0000","last_unstale":"2023-11-16T16:46:53.045350+0000","last_undegraded":"2023-11-16T16:46:53.045350+0000","last_fullsized":"2023-11-16T16:46:53.045350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:32:45.116412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331206+0000","last_change":"2023-11-16T16:46:37.148036+0000","last_active":"2023-11-16T16:46:52.331206+0000","last_peered":"2023-11-16T16:46:52.331206+0000","last_clean":"2023-11-16T16:46:52.331206+0000","last_became_active":"2023-11-16T16:46:37.147372+0000","last_became_peered":"2023-11-16T16:46:37.147372+0000","last_unstale":"2023-11-16T16:46:52.331206+0000","last_undegraded":"2023-11-16T16:46:52.331206+0000","last_fullsized":"2023-11-16T16:46:52.331206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:00:49.806134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335024+0000","last_change":"2023-11-16T16:46:37.147120+0000","last_active":"2023-11-16T16:46:52.335024+0000","last_peered":"2023-11-16T16:46:52.335024+0000","last_clean":"2023-11-16T16:46:52.335024+0000","last_became_active":"2023-11-16T16:46:37.146615+0000","last_became_peered":"2023-11-16T16:46:37.146615+0000","last_unstale":"2023-11-16T16:46:52.335024+0000","last_undegraded":"2023-11-16T16:46:52.335024+0000","last_fullsized":"2023-11-16T16:46:52.335024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:56:35.911100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426417+0000","last_change":"2023-11-16T16:46:37.152029+0000","last_active":"2023-11-16T16:46:52.426417+0000","last_peered":"2023-11-16T16:46:52.426417+0000","last_clean":"2023-11-16T16:46:52.426417+0000","last_became_active":"2023-11-16T16:46:37.150662+0000","last_became_peered":"2023-11-16T16:46:37.150662+0000","last_unstale":"2023-11-16T16:46:52.426417+0000","last_undegraded":"2023-11-16T16:46:52.426417+0000","last_fullsized":"2023-11-16T16:46:52.426417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:16:19.042910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.329535+0000","last_change":"2023-11-16T16:46:37.160562+0000","last_active":"2023-11-16T16:46:52.329535+0000","last_peered":"2023-11-16T16:46:52.329535+0000","last_clean":"2023-11-16T16:46:52.329535+0000","last_became_active":"2023-11-16T16:46:37.158175+0000","last_became_peered":"2023-11-16T16:46:37.158175+0000","last_unstale":"2023-11-16T16:46:52.329535+0000","last_undegraded":"2023-11-16T16:46:52.329535+0000","last_fullsized":"2023-11-16T16:46:52.329535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:02:03.417017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.329392+0000","last_change":"2023-11-16T16:46:37.156381+0000","last_active":"2023-11-16T16:46:52.329392+0000","last_peered":"2023-11-16T16:46:52.329392+0000","last_clean":"2023-11-16T16:46:52.329392+0000","last_became_active":"2023-11-16T16:46:37.150366+0000","last_became_peered":"2023-11-16T16:46:37.150366+0000","last_unstale":"2023-11-16T16:46:52.329392+0000","last_undegraded":"2023-11-16T16:46:52.329392+0000","last_fullsized":"2023-11-16T16:46:52.329392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:50:21.478034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331876+0000","last_change":"2023-11-16T16:46:37.158070+0000","last_active":"2023-11-16T16:46:52.331876+0000","last_peered":"2023-11-16T16:46:52.331876+0000","last_clean":"2023-11-16T16:46:52.331876+0000","last_became_active":"2023-11-16T16:46:37.157755+0000","last_became_peered":"2023-11-16T16:46:37.157755+0000","last_unstale":"2023-11-16T16:46:52.331876+0000","last_undegraded":"2023-11-16T16:46:52.331876+0000","last_fullsized":"2023-11-16T16:46:52.331876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:38:14.007502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336484+0000","last_change":"2023-11-16T16:46:37.162616+0000","last_active":"2023-11-16T16:46:52.336484+0000","last_peered":"2023-11-16T16:46:52.336484+0000","last_clean":"2023-11-16T16:46:52.336484+0000","last_became_active":"2023-11-16T16:46:37.162281+0000","last_became_peered":"2023-11-16T16:46:37.162281+0000","last_unstale":"2023-11-16T16:46:52.336484+0000","last_undegraded":"2023-11-16T16:46:52.336484+0000","last_fullsized":"2023-11-16T16:46:52.336484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:13.452584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331314+0000","last_change":"2023-11-16T16:46:37.150337+0000","last_active":"2023-11-16T16:46:52.331314+0000","last_peered":"2023-11-16T16:46:52.331314+0000","last_clean":"2023-11-16T16:46:52.331314+0000","last_became_active":"2023-11-16T16:46:37.148826+0000","last_became_peered":"2023-11-16T16:46:37.148826+0000","last_unstale":"2023-11-16T16:46:52.331314+0000","last_undegraded":"2023-11-16T16:46:52.331314+0000","last_fullsized":"2023-11-16T16:46:52.331314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:28.902287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336555+0000","last_change":"2023-11-16T16:46:37.150253+0000","last_active":"2023-11-16T16:46:52.336555+0000","last_peered":"2023-11-16T16:46:52.336555+0000","last_clean":"2023-11-16T16:46:52.336555+0000","last_became_active":"2023-11-16T16:46:37.149962+0000","last_became_peered":"2023-11-16T16:46:37.149962+0000","last_unstale":"2023-11-16T16:46:52.336555+0000","last_undegraded":"2023-11-16T16:46:52.336555+0000","last_fullsized":"2023-11-16T16:46:52.336555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:35:48.970850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334978+0000","last_change":"2023-11-16T16:46:41.177888+0000","last_active":"2023-11-16T16:46:52.334978+0000","last_peered":"2023-11-16T16:46:52.334978+0000","last_clean":"2023-11-16T16:46:52.334978+0000","last_became_active":"2023-11-16T16:46:41.177442+0000","last_became_peered":"2023-11-16T16:46:41.177442+0000","last_unstale":"2023-11-16T16:46:52.334978+0000","last_undegraded":"2023-11-16T16:46:52.334978+0000","last_fullsized":"2023-11-16T16:46:52.334978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:11:25.938101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:09.650717+0000","last_change":"2023-11-16T16:46:36.595769+0000","last_active":"2023-11-16T16:47:09.650717+0000","last_peered":"2023-11-16T16:47:09.650717+0000","last_clean":"2023-11-16T16:47:09.650717+0000","last_became_active":"2023-11-16T16:46:36.593780+0000","last_became_peered":"2023-11-16T16:46:36.593780+0000","last_unstale":"2023-11-16T16:47:09.650717+0000","last_undegraded":"2023-11-16T16:47:09.650717+0000","last_fullsized":"2023-11-16T16:47:09.650717+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:56:43.139818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.045299+0000","last_change":"2023-11-16T16:46:37.146683+0000","last_active":"2023-11-16T16:46:53.045299+0000","last_peered":"2023-11-16T16:46:53.045299+0000","last_clean":"2023-11-16T16:46:53.045299+0000","last_became_active":"2023-11-16T16:46:37.146363+0000","last_became_peered":"2023-11-16T16:46:37.146363+0000","last_unstale":"2023-11-16T16:46:53.045299+0000","last_undegraded":"2023-11-16T16:46:53.045299+0000","last_fullsized":"2023-11-16T16:46:53.045299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:00:19.122906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333768+0000","last_change":"2023-11-16T16:46:41.174942+0000","last_active":"2023-11-16T16:46:52.333768+0000","last_peered":"2023-11-16T16:46:52.333768+0000","last_clean":"2023-11-16T16:46:52.333768+0000","last_became_active":"2023-11-16T16:46:41.174483+0000","last_became_peered":"2023-11-16T16:46:41.174483+0000","last_unstale":"2023-11-16T16:46:52.333768+0000","last_undegraded":"2023-11-16T16:46:52.333768+0000","last_fullsized":"2023-11-16T16:46:52.333768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:08:57.816394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044328+0000","last_change":"2023-11-16T16:46:36.607478+0000","last_active":"2023-11-16T16:46:53.044328+0000","last_peered":"2023-11-16T16:46:53.044328+0000","last_clean":"2023-11-16T16:46:53.044328+0000","last_became_active":"2023-11-16T16:46:36.607137+0000","last_became_peered":"2023-11-16T16:46:36.607137+0000","last_unstale":"2023-11-16T16:46:53.044328+0000","last_undegraded":"2023-11-16T16:46:53.044328+0000","last_fullsized":"2023-11-16T16:46:53.044328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:13:10.233490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330466+0000","last_change":"2023-11-16T16:46:37.148576+0000","last_active":"2023-11-16T16:46:52.330466+0000","last_peered":"2023-11-16T16:46:52.330466+0000","last_clean":"2023-11-16T16:46:52.330466+0000","last_became_active":"2023-11-16T16:46:37.148284+0000","last_became_peered":"2023-11-16T16:46:37.148284+0000","last_unstale":"2023-11-16T16:46:52.330466+0000","last_undegraded":"2023-11-16T16:46:52.330466+0000","last_fullsized":"2023-11-16T16:46:52.330466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:11:53.759257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333970+0000","last_change":"2023-11-16T16:46:41.178717+0000","last_active":"2023-11-16T16:46:52.333970+0000","last_peered":"2023-11-16T16:46:52.333970+0000","last_clean":"2023-11-16T16:46:52.333970+0000","last_became_active":"2023-11-16T16:46:41.178350+0000","last_became_peered":"2023-11-16T16:46:41.178350+0000","last_unstale":"2023-11-16T16:46:52.333970+0000","last_undegraded":"2023-11-16T16:46:52.333970+0000","last_fullsized":"2023-11-16T16:46:52.333970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:54:20.213711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:09.648970+0000","last_change":"2023-11-16T16:46:52.639272+0000","last_active":"2023-11-16T16:47:09.648970+0000","last_peered":"2023-11-16T16:47:09.648970+0000","last_clean":"2023-11-16T16:47:09.648970+0000","last_became_active":"2023-11-16T16:46:52.338319+0000","last_became_peered":"2023-11-16T16:46:52.338319+0000","last_unstale":"2023-11-16T16:47:09.648970+0000","last_undegraded":"2023-11-16T16:47:09.648970+0000","last_fullsized":"2023-11-16T16:47:09.648970+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:45:13.831749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.332231+0000","last_change":"2023-11-16T16:46:37.153404+0000","last_active":"2023-11-16T16:46:52.332231+0000","last_peered":"2023-11-16T16:46:52.332231+0000","last_clean":"2023-11-16T16:46:52.332231+0000","last_became_active":"2023-11-16T16:46:37.153064+0000","last_became_peered":"2023-11-16T16:46:37.153064+0000","last_unstale":"2023-11-16T16:46:52.332231+0000","last_undegraded":"2023-11-16T16:46:52.332231+0000","last_fullsized":"2023-11-16T16:46:52.332231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:56:27.226825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333139+0000","last_change":"2023-11-16T16:46:41.163307+0000","last_active":"2023-11-16T16:46:52.333139+0000","last_peered":"2023-11-16T16:46:52.333139+0000","last_clean":"2023-11-16T16:46:52.333139+0000","last_became_active":"2023-11-16T16:46:41.162822+0000","last_became_peered":"2023-11-16T16:46:41.162822+0000","last_unstale":"2023-11-16T16:46:52.333139+0000","last_undegraded":"2023-11-16T16:46:52.333139+0000","last_fullsized":"2023-11-16T16:46:52.333139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:44:51.683473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.332575+0000","last_change":"2023-11-16T16:46:36.595905+0000","last_active":"2023-11-16T16:46:52.332575+0000","last_peered":"2023-11-16T16:46:52.332575+0000","last_clean":"2023-11-16T16:46:52.332575+0000","last_became_active":"2023-11-16T16:46:36.595403+0000","last_became_peered":"2023-11-16T16:46:36.595403+0000","last_unstale":"2023-11-16T16:46:52.332575+0000","last_undegraded":"2023-11-16T16:46:52.332575+0000","last_fullsized":"2023-11-16T16:46:52.332575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:53:07.115959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334655+0000","last_change":"2023-11-16T16:46:52.334655+0000","last_active":"2023-11-16T16:46:52.334655+0000","last_peered":"2023-11-16T16:46:52.334655+0000","last_clean":"2023-11-16T16:46:52.334655+0000","last_became_active":"2023-11-16T16:46:52.334345+0000","last_became_peered":"2023-11-16T16:46:52.334345+0000","last_unstale":"2023-11-16T16:46:52.334655+0000","last_undegraded":"2023-11-16T16:46:52.334655+0000","last_fullsized":"2023-11-16T16:46:52.334655+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:17:06.138428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336484+0000","last_change":"2023-11-16T16:46:41.169041+0000","last_active":"2023-11-16T16:46:52.336484+0000","last_peered":"2023-11-16T16:46:52.336484+0000","last_clean":"2023-11-16T16:46:52.336484+0000","last_became_active":"2023-11-16T16:46:41.168589+0000","last_became_peered":"2023-11-16T16:46:41.168589+0000","last_unstale":"2023-11-16T16:46:52.336484+0000","last_undegraded":"2023-11-16T16:46:52.336484+0000","last_fullsized":"2023-11-16T16:46:52.336484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:36:18.896357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.643139+0000","last_change":"2023-11-16T16:46:36.595442+0000","last_active":"2023-11-16T16:47:08.643139+0000","last_peered":"2023-11-16T16:47:08.643139+0000","last_clean":"2023-11-16T16:47:08.643139+0000","last_became_active":"2023-11-16T16:46:36.595122+0000","last_became_peered":"2023-11-16T16:46:36.595122+0000","last_unstale":"2023-11-16T16:47:08.643139+0000","last_undegraded":"2023-11-16T16:47:08.643139+0000","last_fullsized":"2023-11-16T16:47:08.643139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:38:17.777801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.332585+0000","last_change":"2023-11-16T16:46:37.149218+0000","last_active":"2023-11-16T16:46:52.332585+0000","last_peered":"2023-11-16T16:46:52.332585+0000","last_clean":"2023-11-16T16:46:52.332585+0000","last_became_active":"2023-11-16T16:46:37.148908+0000","last_became_peered":"2023-11-16T16:46:37.148908+0000","last_unstale":"2023-11-16T16:46:52.332585+0000","last_undegraded":"2023-11-16T16:46:52.332585+0000","last_fullsized":"2023-11-16T16:46:52.332585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:03:58.945088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336993+0000","last_change":"2023-11-16T16:46:41.185001+0000","last_active":"2023-11-16T16:46:52.336993+0000","last_peered":"2023-11-16T16:46:52.336993+0000","last_clean":"2023-11-16T16:46:52.336993+0000","last_became_active":"2023-11-16T16:46:41.184603+0000","last_became_peered":"2023-11-16T16:46:41.184603+0000","last_unstale":"2023-11-16T16:46:52.336993+0000","last_undegraded":"2023-11-16T16:46:52.336993+0000","last_fullsized":"2023-11-16T16:46:52.336993+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:54:30.993576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044109+0000","last_change":"2023-11-16T16:46:36.611365+0000","last_active":"2023-11-16T16:46:53.044109+0000","last_peered":"2023-11-16T16:46:53.044109+0000","last_clean":"2023-11-16T16:46:53.044109+0000","last_became_active":"2023-11-16T16:46:36.611006+0000","last_became_peered":"2023-11-16T16:46:36.611006+0000","last_unstale":"2023-11-16T16:46:53.044109+0000","last_undegraded":"2023-11-16T16:46:53.044109+0000","last_fullsized":"2023-11-16T16:46:53.044109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:39:42.167891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327902+0000","last_change":"2023-11-16T16:46:37.156801+0000","last_active":"2023-11-16T16:46:52.327902+0000","last_peered":"2023-11-16T16:46:52.327902+0000","last_clean":"2023-11-16T16:46:52.327902+0000","last_became_active":"2023-11-16T16:46:37.156352+0000","last_became_peered":"2023-11-16T16:46:37.156352+0000","last_unstale":"2023-11-16T16:46:52.327902+0000","last_undegraded":"2023-11-16T16:46:52.327902+0000","last_fullsized":"2023-11-16T16:46:52.327902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:40:14.223607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336005+0000","last_change":"2023-11-16T16:46:41.182374+0000","last_active":"2023-11-16T16:46:52.336005+0000","last_peered":"2023-11-16T16:46:52.336005+0000","last_clean":"2023-11-16T16:46:52.336005+0000","last_became_active":"2023-11-16T16:46:41.181977+0000","last_became_peered":"2023-11-16T16:46:41.181977+0000","last_unstale":"2023-11-16T16:46:52.336005+0000","last_undegraded":"2023-11-16T16:46:52.336005+0000","last_fullsized":"2023-11-16T16:46:52.336005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:17:28.606630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:09.651645+0000","last_change":"2023-11-16T16:46:36.607647+0000","last_active":"2023-11-16T16:47:09.651645+0000","last_peered":"2023-11-16T16:47:09.651645+0000","last_clean":"2023-11-16T16:47:09.651645+0000","last_became_active":"2023-11-16T16:46:36.607282+0000","last_became_peered":"2023-11-16T16:46:36.607282+0000","last_unstale":"2023-11-16T16:47:09.651645+0000","last_undegraded":"2023-11-16T16:47:09.651645+0000","last_fullsized":"2023-11-16T16:47:09.651645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:27:34.869730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044528+0000","last_change":"2023-11-16T16:46:37.152659+0000","last_active":"2023-11-16T16:46:53.044528+0000","last_peered":"2023-11-16T16:46:53.044528+0000","last_clean":"2023-11-16T16:46:53.044528+0000","last_became_active":"2023-11-16T16:46:37.152313+0000","last_became_peered":"2023-11-16T16:46:37.152313+0000","last_unstale":"2023-11-16T16:46:53.044528+0000","last_undegraded":"2023-11-16T16:46:53.044528+0000","last_fullsized":"2023-11-16T16:46:53.044528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:20:05.081803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330459+0000","last_change":"2023-11-16T16:46:41.167312+0000","last_active":"2023-11-16T16:46:52.330459+0000","last_peered":"2023-11-16T16:46:52.330459+0000","last_clean":"2023-11-16T16:46:52.330459+0000","last_became_active":"2023-11-16T16:46:41.166211+0000","last_became_peered":"2023-11-16T16:46:41.166211+0000","last_unstale":"2023-11-16T16:46:52.330459+0000","last_undegraded":"2023-11-16T16:46:52.330459+0000","last_fullsized":"2023-11-16T16:46:52.330459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:32:06.730118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327225+0000","last_change":"2023-11-16T16:46:36.595888+0000","last_active":"2023-11-16T16:46:52.327225+0000","last_peered":"2023-11-16T16:46:52.327225+0000","last_clean":"2023-11-16T16:46:52.327225+0000","last_became_active":"2023-11-16T16:46:36.595602+0000","last_became_peered":"2023-11-16T16:46:36.595602+0000","last_unstale":"2023-11-16T16:46:52.327225+0000","last_undegraded":"2023-11-16T16:46:52.327225+0000","last_fullsized":"2023-11-16T16:46:52.327225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:29:12.613330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330260+0000","last_change":"2023-11-16T16:46:37.154431+0000","last_active":"2023-11-16T16:46:52.330260+0000","last_peered":"2023-11-16T16:46:52.330260+0000","last_clean":"2023-11-16T16:46:52.330260+0000","last_became_active":"2023-11-16T16:46:37.154106+0000","last_became_peered":"2023-11-16T16:46:37.154106+0000","last_unstale":"2023-11-16T16:46:52.330260+0000","last_undegraded":"2023-11-16T16:46:52.330260+0000","last_fullsized":"2023-11-16T16:46:52.330260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:07:37.024127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044743+0000","last_change":"2023-11-16T16:46:41.168644+0000","last_active":"2023-11-16T16:46:53.044743+0000","last_peered":"2023-11-16T16:46:53.044743+0000","last_clean":"2023-11-16T16:46:53.044743+0000","last_became_active":"2023-11-16T16:46:41.168121+0000","last_became_peered":"2023-11-16T16:46:41.168121+0000","last_unstale":"2023-11-16T16:46:53.044743+0000","last_undegraded":"2023-11-16T16:46:53.044743+0000","last_fullsized":"2023-11-16T16:46:53.044743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:07:29.914977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.643553+0000","last_change":"2023-11-16T16:46:36.597293+0000","last_active":"2023-11-16T16:47:08.643553+0000","last_peered":"2023-11-16T16:47:08.643553+0000","last_clean":"2023-11-16T16:47:08.643553+0000","last_became_active":"2023-11-16T16:46:36.596992+0000","last_became_peered":"2023-11-16T16:46:36.596992+0000","last_unstale":"2023-11-16T16:47:08.643553+0000","last_undegraded":"2023-11-16T16:47:08.643553+0000","last_fullsized":"2023-11-16T16:47:08.643553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:13:49.650449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333206+0000","last_change":"2023-11-16T16:46:37.149339+0000","last_active":"2023-11-16T16:46:52.333206+0000","last_peered":"2023-11-16T16:46:52.333206+0000","last_clean":"2023-11-16T16:46:52.333206+0000","last_became_active":"2023-11-16T16:46:37.149000+0000","last_became_peered":"2023-11-16T16:46:37.149000+0000","last_unstale":"2023-11-16T16:46:52.333206+0000","last_undegraded":"2023-11-16T16:46:52.333206+0000","last_fullsized":"2023-11-16T16:46:52.333206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:22:35.436696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.339769+0000","last_change":"2023-11-16T16:46:52.338731+0000","last_active":"2023-11-16T16:46:52.339769+0000","last_peered":"2023-11-16T16:46:52.339769+0000","last_clean":"2023-11-16T16:46:52.339769+0000","last_became_active":"2023-11-16T16:46:52.338254+0000","last_became_peered":"2023-11-16T16:46:52.338254+0000","last_unstale":"2023-11-16T16:46:52.339769+0000","last_undegraded":"2023-11-16T16:46:52.339769+0000","last_fullsized":"2023-11-16T16:46:52.339769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:41:05.865764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330874+0000","last_change":"2023-11-16T16:46:36.595407+0000","last_active":"2023-11-16T16:46:52.330874+0000","last_peered":"2023-11-16T16:46:52.330874+0000","last_clean":"2023-11-16T16:46:52.330874+0000","last_became_active":"2023-11-16T16:46:36.595132+0000","last_became_peered":"2023-11-16T16:46:36.595132+0000","last_unstale":"2023-11-16T16:46:52.330874+0000","last_undegraded":"2023-11-16T16:46:52.330874+0000","last_fullsized":"2023-11-16T16:46:52.330874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:20:59.670565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330345+0000","last_change":"2023-11-16T16:46:37.155037+0000","last_active":"2023-11-16T16:46:52.330345+0000","last_peered":"2023-11-16T16:46:52.330345+0000","last_clean":"2023-11-16T16:46:52.330345+0000","last_became_active":"2023-11-16T16:46:37.149822+0000","last_became_peered":"2023-11-16T16:46:37.149822+0000","last_unstale":"2023-11-16T16:46:52.330345+0000","last_undegraded":"2023-11-16T16:46:52.330345+0000","last_fullsized":"2023-11-16T16:46:52.330345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:56:10.462369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.339033+0000","last_change":"2023-11-16T16:46:52.339033+0000","last_active":"2023-11-16T16:46:52.339033+0000","last_peered":"2023-11-16T16:46:52.339033+0000","last_clean":"2023-11-16T16:46:52.339033+0000","last_became_active":"2023-11-16T16:46:52.338451+0000","last_became_peered":"2023-11-16T16:46:52.338451+0000","last_unstale":"2023-11-16T16:46:52.339033+0000","last_undegraded":"2023-11-16T16:46:52.339033+0000","last_fullsized":"2023-11-16T16:46:52.339033+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:04:46.441512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.328042+0000","last_change":"2023-11-16T16:46:36.596779+0000","last_active":"2023-11-16T16:46:52.328042+0000","last_peered":"2023-11-16T16:46:52.328042+0000","last_clean":"2023-11-16T16:46:52.328042+0000","last_became_active":"2023-11-16T16:46:36.596460+0000","last_became_peered":"2023-11-16T16:46:36.596460+0000","last_unstale":"2023-11-16T16:46:52.328042+0000","last_undegraded":"2023-11-16T16:46:52.328042+0000","last_fullsized":"2023-11-16T16:46:52.328042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:15:12.352768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.427912+0000","last_change":"2023-11-16T16:46:52.427912+0000","last_active":"2023-11-16T16:46:52.427912+0000","last_peered":"2023-11-16T16:46:52.427912+0000","last_clean":"2023-11-16T16:46:52.427912+0000","last_became_active":"2023-11-16T16:46:52.427426+0000","last_became_peered":"2023-11-16T16:46:52.427426+0000","last_unstale":"2023-11-16T16:46:52.427912+0000","last_undegraded":"2023-11-16T16:46:52.427912+0000","last_fullsized":"2023-11-16T16:46:52.427912+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:44:57.112343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330521+0000","last_change":"2023-11-16T16:46:41.177483+0000","last_active":"2023-11-16T16:46:52.330521+0000","last_peered":"2023-11-16T16:46:52.330521+0000","last_clean":"2023-11-16T16:46:52.330521+0000","last_became_active":"2023-11-16T16:46:41.173165+0000","last_became_peered":"2023-11-16T16:46:41.173165+0000","last_unstale":"2023-11-16T16:46:52.330521+0000","last_undegraded":"2023-11-16T16:46:52.330521+0000","last_fullsized":"2023-11-16T16:46:52.330521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:57:53.705807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.642841+0000","last_change":"2023-11-16T16:46:36.598279+0000","last_active":"2023-11-16T16:47:08.642841+0000","last_peered":"2023-11-16T16:47:08.642841+0000","last_clean":"2023-11-16T16:47:08.642841+0000","last_became_active":"2023-11-16T16:46:36.597975+0000","last_became_peered":"2023-11-16T16:46:36.597975+0000","last_unstale":"2023-11-16T16:47:08.642841+0000","last_undegraded":"2023-11-16T16:47:08.642841+0000","last_fullsized":"2023-11-16T16:47:08.642841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:36:36.186292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333136+0000","last_change":"2023-11-16T16:46:37.143014+0000","last_active":"2023-11-16T16:46:52.333136+0000","last_peered":"2023-11-16T16:46:52.333136+0000","last_clean":"2023-11-16T16:46:52.333136+0000","last_became_active":"2023-11-16T16:46:37.142662+0000","last_became_peered":"2023-11-16T16:46:37.142662+0000","last_unstale":"2023-11-16T16:46:52.333136+0000","last_undegraded":"2023-11-16T16:46:52.333136+0000","last_fullsized":"2023-11-16T16:46:52.333136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:51:30.485958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.328331+0000","last_change":"2023-11-16T16:46:41.169921+0000","last_active":"2023-11-16T16:46:52.328331+0000","last_peered":"2023-11-16T16:46:52.328331+0000","last_clean":"2023-11-16T16:46:52.328331+0000","last_became_active":"2023-11-16T16:46:41.169037+0000","last_became_peered":"2023-11-16T16:46:41.169037+0000","last_unstale":"2023-11-16T16:46:52.328331+0000","last_undegraded":"2023-11-16T16:46:52.328331+0000","last_fullsized":"2023-11-16T16:46:52.328331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:08:05.111113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336722+0000","last_change":"2023-11-16T16:46:36.596159+0000","last_active":"2023-11-16T16:46:52.336722+0000","last_peered":"2023-11-16T16:46:52.336722+0000","last_clean":"2023-11-16T16:46:52.336722+0000","last_became_active":"2023-11-16T16:46:36.595847+0000","last_became_peered":"2023-11-16T16:46:36.595847+0000","last_unstale":"2023-11-16T16:46:52.336722+0000","last_undegraded":"2023-11-16T16:46:52.336722+0000","last_fullsized":"2023-11-16T16:46:52.336722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:06:32.736611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426173+0000","last_change":"2023-11-16T16:46:37.156046+0000","last_active":"2023-11-16T16:46:52.426173+0000","last_peered":"2023-11-16T16:46:52.426173+0000","last_clean":"2023-11-16T16:46:52.426173+0000","last_became_active":"2023-11-16T16:46:37.155657+0000","last_became_peered":"2023-11-16T16:46:37.155657+0000","last_unstale":"2023-11-16T16:46:52.426173+0000","last_undegraded":"2023-11-16T16:46:52.426173+0000","last_fullsized":"2023-11-16T16:46:52.426173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:57.255975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334980+0000","last_change":"2023-11-16T16:46:41.182485+0000","last_active":"2023-11-16T16:46:52.334980+0000","last_peered":"2023-11-16T16:46:52.334980+0000","last_clean":"2023-11-16T16:46:52.334980+0000","last_became_active":"2023-11-16T16:46:41.181990+0000","last_became_peered":"2023-11-16T16:46:41.181990+0000","last_unstale":"2023-11-16T16:46:52.334980+0000","last_undegraded":"2023-11-16T16:46:52.334980+0000","last_fullsized":"2023-11-16T16:46:52.334980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:15:28.764485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.643137+0000","last_change":"2023-11-16T16:46:36.608501+0000","last_active":"2023-11-16T16:47:08.643137+0000","last_peered":"2023-11-16T16:47:08.643137+0000","last_clean":"2023-11-16T16:47:08.643137+0000","last_became_active":"2023-11-16T16:46:36.608210+0000","last_became_peered":"2023-11-16T16:46:36.608210+0000","last_unstale":"2023-11-16T16:47:08.643137+0000","last_undegraded":"2023-11-16T16:47:08.643137+0000","last_fullsized":"2023-11-16T16:47:08.643137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:25:48.287851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330892+0000","last_change":"2023-11-16T16:46:37.155431+0000","last_active":"2023-11-16T16:46:52.330892+0000","last_peered":"2023-11-16T16:46:52.330892+0000","last_clean":"2023-11-16T16:46:52.330892+0000","last_became_active":"2023-11-16T16:46:37.155106+0000","last_became_peered":"2023-11-16T16:46:37.155106+0000","last_unstale":"2023-11-16T16:46:52.330892+0000","last_undegraded":"2023-11-16T16:46:52.330892+0000","last_fullsized":"2023-11-16T16:46:52.330892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:40:10.184454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331332+0000","last_change":"2023-11-16T16:46:41.180903+0000","last_active":"2023-11-16T16:46:52.331332+0000","last_peered":"2023-11-16T16:46:52.331332+0000","last_clean":"2023-11-16T16:46:52.331332+0000","last_became_active":"2023-11-16T16:46:41.179842+0000","last_became_peered":"2023-11-16T16:46:41.179842+0000","last_unstale":"2023-11-16T16:46:52.331332+0000","last_undegraded":"2023-11-16T16:46:52.331332+0000","last_fullsized":"2023-11-16T16:46:52.331332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:08:39.409755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:11.326880+0000","last_change":"2023-11-16T16:46:36.592436+0000","last_active":"2023-11-16T16:47:11.326880+0000","last_peered":"2023-11-16T16:47:11.326880+0000","last_clean":"2023-11-16T16:47:11.326880+0000","last_became_active":"2023-11-16T16:46:36.592078+0000","last_became_peered":"2023-11-16T16:46:36.592078+0000","last_unstale":"2023-11-16T16:47:11.326880+0000","last_undegraded":"2023-11-16T16:47:11.326880+0000","last_fullsized":"2023-11-16T16:47:11.326880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:23:31.429710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330109+0000","last_change":"2023-11-16T16:46:37.153071+0000","last_active":"2023-11-16T16:46:52.330109+0000","last_peered":"2023-11-16T16:46:52.330109+0000","last_clean":"2023-11-16T16:46:52.330109+0000","last_became_active":"2023-11-16T16:46:37.152742+0000","last_became_peered":"2023-11-16T16:46:37.152742+0000","last_unstale":"2023-11-16T16:46:52.330109+0000","last_undegraded":"2023-11-16T16:46:52.330109+0000","last_fullsized":"2023-11-16T16:46:52.330109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:57:55.272027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426556+0000","last_change":"2023-11-16T16:46:41.176434+0000","last_active":"2023-11-16T16:46:52.426556+0000","last_peered":"2023-11-16T16:46:52.426556+0000","last_clean":"2023-11-16T16:46:52.426556+0000","last_became_active":"2023-11-16T16:46:41.175771+0000","last_became_peered":"2023-11-16T16:46:41.175771+0000","last_unstale":"2023-11-16T16:46:52.426556+0000","last_undegraded":"2023-11-16T16:46:52.426556+0000","last_fullsized":"2023-11-16T16:46:52.426556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:19:46.468931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:09.652768+0000","last_change":"2023-11-16T16:46:36.594148+0000","last_active":"2023-11-16T16:47:09.652768+0000","last_peered":"2023-11-16T16:47:09.652768+0000","last_clean":"2023-11-16T16:47:09.652768+0000","last_became_active":"2023-11-16T16:46:36.593437+0000","last_became_peered":"2023-11-16T16:46:36.593437+0000","last_unstale":"2023-11-16T16:47:09.652768+0000","last_undegraded":"2023-11-16T16:47:09.652768+0000","last_fullsized":"2023-11-16T16:47:09.652768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:45:12.599977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327398+0000","last_change":"2023-11-16T16:46:37.149044+0000","last_active":"2023-11-16T16:46:52.327398+0000","last_peered":"2023-11-16T16:46:52.327398+0000","last_clean":"2023-11-16T16:46:52.327398+0000","last_became_active":"2023-11-16T16:46:37.147215+0000","last_became_peered":"2023-11-16T16:46:37.147215+0000","last_unstale":"2023-11-16T16:46:52.327398+0000","last_undegraded":"2023-11-16T16:46:52.327398+0000","last_fullsized":"2023-11-16T16:46:52.327398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:58:43.048815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327370+0000","last_change":"2023-11-16T16:46:41.168754+0000","last_active":"2023-11-16T16:46:52.327370+0000","last_peered":"2023-11-16T16:46:52.327370+0000","last_clean":"2023-11-16T16:46:52.327370+0000","last_became_active":"2023-11-16T16:46:41.168056+0000","last_became_peered":"2023-11-16T16:46:41.168056+0000","last_unstale":"2023-11-16T16:46:52.327370+0000","last_undegraded":"2023-11-16T16:46:52.327370+0000","last_fullsized":"2023-11-16T16:46:52.327370+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:23:57.870297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333794+0000","last_change":"2023-11-16T16:46:36.594892+0000","last_active":"2023-11-16T16:46:52.333794+0000","last_peered":"2023-11-16T16:46:52.333794+0000","last_clean":"2023-11-16T16:46:52.333794+0000","last_became_active":"2023-11-16T16:46:36.594557+0000","last_became_peered":"2023-11-16T16:46:36.594557+0000","last_unstale":"2023-11-16T16:46:52.333794+0000","last_undegraded":"2023-11-16T16:46:52.333794+0000","last_fullsized":"2023-11-16T16:46:52.333794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:17:50.519104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335354+0000","last_change":"2023-11-16T16:46:37.148203+0000","last_active":"2023-11-16T16:46:52.335354+0000","last_peered":"2023-11-16T16:46:52.335354+0000","last_clean":"2023-11-16T16:46:52.335354+0000","last_became_active":"2023-11-16T16:46:37.147846+0000","last_became_peered":"2023-11-16T16:46:37.147846+0000","last_unstale":"2023-11-16T16:46:52.335354+0000","last_undegraded":"2023-11-16T16:46:52.335354+0000","last_fullsized":"2023-11-16T16:46:52.335354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:02:06.446791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.046150+0000","last_change":"2023-11-16T16:46:53.046150+0000","last_active":"2023-11-16T16:46:53.046150+0000","last_peered":"2023-11-16T16:46:53.046150+0000","last_clean":"2023-11-16T16:46:53.046150+0000","last_became_active":"2023-11-16T16:46:53.045619+0000","last_became_peered":"2023-11-16T16:46:53.045619+0000","last_unstale":"2023-11-16T16:46:53.046150+0000","last_undegraded":"2023-11-16T16:46:53.046150+0000","last_fullsized":"2023-11-16T16:46:53.046150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:21:37.959125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330958+0000","last_change":"2023-11-16T16:46:36.600779+0000","last_active":"2023-11-16T16:46:52.330958+0000","last_peered":"2023-11-16T16:46:52.330958+0000","last_clean":"2023-11-16T16:46:52.330958+0000","last_became_active":"2023-11-16T16:46:36.598664+0000","last_became_peered":"2023-11-16T16:46:36.598664+0000","last_unstale":"2023-11-16T16:46:52.330958+0000","last_undegraded":"2023-11-16T16:46:52.330958+0000","last_fullsized":"2023-11-16T16:46:52.330958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:30:23.163310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.045270+0000","last_change":"2023-11-16T16:46:37.155694+0000","last_active":"2023-11-16T16:46:53.045270+0000","last_peered":"2023-11-16T16:46:53.045270+0000","last_clean":"2023-11-16T16:46:53.045270+0000","last_became_active":"2023-11-16T16:46:37.155105+0000","last_became_peered":"2023-11-16T16:46:37.155105+0000","last_unstale":"2023-11-16T16:46:53.045270+0000","last_undegraded":"2023-11-16T16:46:53.045270+0000","last_fullsized":"2023-11-16T16:46:53.045270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:56:49.859400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426481+0000","last_change":"2023-11-16T16:46:41.172161+0000","last_active":"2023-11-16T16:46:52.426481+0000","last_peered":"2023-11-16T16:46:52.426481+0000","last_clean":"2023-11-16T16:46:52.426481+0000","last_became_active":"2023-11-16T16:46:41.169661+0000","last_became_peered":"2023-11-16T16:46:41.169661+0000","last_unstale":"2023-11-16T16:46:52.426481+0000","last_undegraded":"2023-11-16T16:46:52.426481+0000","last_fullsized":"2023-11-16T16:46:52.426481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:21:34.679139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330271+0000","last_change":"2023-11-16T16:46:36.611376+0000","last_active":"2023-11-16T16:46:52.330271+0000","last_peered":"2023-11-16T16:46:52.330271+0000","last_clean":"2023-11-16T16:46:52.330271+0000","last_became_active":"2023-11-16T16:46:36.611045+0000","last_became_peered":"2023-11-16T16:46:36.611045+0000","last_unstale":"2023-11-16T16:46:52.330271+0000","last_undegraded":"2023-11-16T16:46:52.330271+0000","last_fullsized":"2023-11-16T16:46:52.330271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:11:15.321886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044773+0000","last_change":"2023-11-16T16:46:37.164850+0000","last_active":"2023-11-16T16:46:53.044773+0000","last_peered":"2023-11-16T16:46:53.044773+0000","last_clean":"2023-11-16T16:46:53.044773+0000","last_became_active":"2023-11-16T16:46:37.164532+0000","last_became_peered":"2023-11-16T16:46:37.164532+0000","last_unstale":"2023-11-16T16:46:53.044773+0000","last_undegraded":"2023-11-16T16:46:53.044773+0000","last_fullsized":"2023-11-16T16:46:53.044773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:26:29.906483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426557+0000","last_change":"2023-11-16T16:46:41.203618+0000","last_active":"2023-11-16T16:46:52.426557+0000","last_peered":"2023-11-16T16:46:52.426557+0000","last_clean":"2023-11-16T16:46:52.426557+0000","last_became_active":"2023-11-16T16:46:41.202322+0000","last_became_peered":"2023-11-16T16:46:41.202322+0000","last_unstale":"2023-11-16T16:46:52.426557+0000","last_undegraded":"2023-11-16T16:46:52.426557+0000","last_fullsized":"2023-11-16T16:46:52.426557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:51:27.136384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327322+0000","last_change":"2023-11-16T16:46:37.154064+0000","last_active":"2023-11-16T16:46:52.327322+0000","last_peered":"2023-11-16T16:46:52.327322+0000","last_clean":"2023-11-16T16:46:52.327322+0000","last_became_active":"2023-11-16T16:46:37.153776+0000","last_became_peered":"2023-11-16T16:46:37.153776+0000","last_unstale":"2023-11-16T16:46:52.327322+0000","last_undegraded":"2023-11-16T16:46:52.327322+0000","last_fullsized":"2023-11-16T16:46:52.327322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:22:57.827996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:09.653103+0000","last_change":"2023-11-16T16:46:36.612613+0000","last_active":"2023-11-16T16:47:09.653103+0000","last_peered":"2023-11-16T16:47:09.653103+0000","last_clean":"2023-11-16T16:47:09.653103+0000","last_became_active":"2023-11-16T16:46:36.612303+0000","last_became_peered":"2023-11-16T16:46:36.612303+0000","last_unstale":"2023-11-16T16:47:09.653103+0000","last_undegraded":"2023-11-16T16:47:09.653103+0000","last_fullsized":"2023-11-16T16:47:09.653103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:17:46.318085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.046753+0000","last_change":"2023-11-16T16:46:53.046753+0000","last_active":"2023-11-16T16:46:53.046753+0000","last_peered":"2023-11-16T16:46:53.046753+0000","last_clean":"2023-11-16T16:46:53.046753+0000","last_became_active":"2023-11-16T16:46:53.046231+0000","last_became_peered":"2023-11-16T16:46:53.046231+0000","last_unstale":"2023-11-16T16:46:53.046753+0000","last_undegraded":"2023-11-16T16:46:53.046753+0000","last_fullsized":"2023-11-16T16:46:53.046753+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:56:01.077949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044682+0000","last_change":"2023-11-16T16:46:37.150627+0000","last_active":"2023-11-16T16:46:53.044682+0000","last_peered":"2023-11-16T16:46:53.044682+0000","last_clean":"2023-11-16T16:46:53.044682+0000","last_became_active":"2023-11-16T16:46:37.147364+0000","last_became_peered":"2023-11-16T16:46:37.147364+0000","last_unstale":"2023-11-16T16:46:53.044682+0000","last_undegraded":"2023-11-16T16:46:53.044682+0000","last_fullsized":"2023-11-16T16:46:53.044682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:51:22.419717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:10.320058+0000","last_change":"2023-11-16T16:46:36.597520+0000","last_active":"2023-11-16T16:47:10.320058+0000","last_peered":"2023-11-16T16:47:10.320058+0000","last_clean":"2023-11-16T16:47:10.320058+0000","last_became_active":"2023-11-16T16:46:36.596542+0000","last_became_peered":"2023-11-16T16:46:36.596542+0000","last_unstale":"2023-11-16T16:47:10.320058+0000","last_undegraded":"2023-11-16T16:47:10.320058+0000","last_fullsized":"2023-11-16T16:47:10.320058+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:49:35.975555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.329918+0000","last_change":"2023-11-16T16:45:27.937850+0000","last_active":"2023-11-16T16:46:52.329918+0000","last_peered":"2023-11-16T16:46:52.329918+0000","last_clean":"2023-11-16T16:46:52.329918+0000","last_became_active":"2023-11-16T16:45:27.937451+0000","last_became_peered":"2023-11-16T16:45:27.937451+0000","last_unstale":"2023-11-16T16:46:52.329918+0000","last_undegraded":"2023-11-16T16:46:52.329918+0000","last_fullsized":"2023-11-16T16:46:52.329918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:42:13.306703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:42:13.306703+0000","last_clean_scrub_stamp":"2023-11-16T16:42:13.306703+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:52:16.097593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327903+0000","last_change":"2023-11-16T16:46:41.203736+0000","last_active":"2023-11-16T16:46:52.327903+0000","last_peered":"2023-11-16T16:46:52.327903+0000","last_clean":"2023-11-16T16:46:52.327903+0000","last_became_active":"2023-11-16T16:46:41.203282+0000","last_became_peered":"2023-11-16T16:46:41.203282+0000","last_unstale":"2023-11-16T16:46:52.327903+0000","last_undegraded":"2023-11-16T16:46:52.327903+0000","last_fullsized":"2023-11-16T16:46:52.327903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:19:06.910333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.425944+0000","last_change":"2023-11-16T16:46:37.157408+0000","last_active":"2023-11-16T16:46:52.425944+0000","last_peered":"2023-11-16T16:46:52.425944+0000","last_clean":"2023-11-16T16:46:52.425944+0000","last_became_active":"2023-11-16T16:46:37.156972+0000","last_became_peered":"2023-11-16T16:46:37.156972+0000","last_unstale":"2023-11-16T16:46:52.425944+0000","last_undegraded":"2023-11-16T16:46:52.425944+0000","last_fullsized":"2023-11-16T16:46:52.425944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:42.100042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.329970+0000","last_change":"2023-11-16T16:46:36.594284+0000","last_active":"2023-11-16T16:46:52.329970+0000","last_peered":"2023-11-16T16:46:52.329970+0000","last_clean":"2023-11-16T16:46:52.329970+0000","last_became_active":"2023-11-16T16:46:36.593964+0000","last_became_peered":"2023-11-16T16:46:36.593964+0000","last_unstale":"2023-11-16T16:46:52.329970+0000","last_undegraded":"2023-11-16T16:46:52.329970+0000","last_fullsized":"2023-11-16T16:46:52.329970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:23:47.325260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335577+0000","last_change":"2023-11-16T16:46:41.173944+0000","last_active":"2023-11-16T16:46:52.335577+0000","last_peered":"2023-11-16T16:46:52.335577+0000","last_clean":"2023-11-16T16:46:52.335577+0000","last_became_active":"2023-11-16T16:46:41.173391+0000","last_became_peered":"2023-11-16T16:46:41.173391+0000","last_unstale":"2023-11-16T16:46:52.335577+0000","last_undegraded":"2023-11-16T16:46:52.335577+0000","last_fullsized":"2023-11-16T16:46:52.335577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:27:11.037240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.642849+0000","last_change":"2023-11-16T16:46:36.596360+0000","last_active":"2023-11-16T16:47:08.642849+0000","last_peered":"2023-11-16T16:47:08.642849+0000","last_clean":"2023-11-16T16:47:08.642849+0000","last_became_active":"2023-11-16T16:46:36.596042+0000","last_became_peered":"2023-11-16T16:46:36.596042+0000","last_unstale":"2023-11-16T16:47:08.642849+0000","last_undegraded":"2023-11-16T16:47:08.642849+0000","last_fullsized":"2023-11-16T16:47:08.642849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:05:25.948915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426327+0000","last_change":"2023-11-16T16:46:37.160277+0000","last_active":"2023-11-16T16:46:52.426327+0000","last_peered":"2023-11-16T16:46:52.426327+0000","last_clean":"2023-11-16T16:46:52.426327+0000","last_became_active":"2023-11-16T16:46:37.159047+0000","last_became_peered":"2023-11-16T16:46:37.159047+0000","last_unstale":"2023-11-16T16:46:52.426327+0000","last_undegraded":"2023-11-16T16:46:52.426327+0000","last_fullsized":"2023-11-16T16:46:52.426327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:35:16.786773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044472+0000","last_change":"2023-11-16T16:46:41.202909+0000","last_active":"2023-11-16T16:46:53.044472+0000","last_peered":"2023-11-16T16:46:53.044472+0000","last_clean":"2023-11-16T16:46:53.044472+0000","last_became_active":"2023-11-16T16:46:41.201969+0000","last_became_peered":"2023-11-16T16:46:41.201969+0000","last_unstale":"2023-11-16T16:46:53.044472+0000","last_undegraded":"2023-11-16T16:46:53.044472+0000","last_fullsized":"2023-11-16T16:46:53.044472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:45:49.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:09.651822+0000","last_change":"2023-11-16T16:46:36.599389+0000","last_active":"2023-11-16T16:47:09.651822+0000","last_peered":"2023-11-16T16:47:09.651822+0000","last_clean":"2023-11-16T16:47:09.651822+0000","last_became_active":"2023-11-16T16:46:36.599020+0000","last_became_peered":"2023-11-16T16:46:36.599020+0000","last_unstale":"2023-11-16T16:47:09.651822+0000","last_undegraded":"2023-11-16T16:47:09.651822+0000","last_fullsized":"2023-11-16T16:47:09.651822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:35:00.154371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.045358+0000","last_change":"2023-11-16T16:46:37.145412+0000","last_active":"2023-11-16T16:46:53.045358+0000","last_peered":"2023-11-16T16:46:53.045358+0000","last_clean":"2023-11-16T16:46:53.045358+0000","last_became_active":"2023-11-16T16:46:37.141380+0000","last_became_peered":"2023-11-16T16:46:37.141380+0000","last_unstale":"2023-11-16T16:46:53.045358+0000","last_undegraded":"2023-11-16T16:46:53.045358+0000","last_fullsized":"2023-11-16T16:46:53.045358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:08:48.448800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.333854+0000","last_change":"2023-11-16T16:46:41.163578+0000","last_active":"2023-11-16T16:46:52.333854+0000","last_peered":"2023-11-16T16:46:52.333854+0000","last_clean":"2023-11-16T16:46:52.333854+0000","last_became_active":"2023-11-16T16:46:41.163153+0000","last_became_peered":"2023-11-16T16:46:41.163153+0000","last_unstale":"2023-11-16T16:46:52.333854+0000","last_undegraded":"2023-11-16T16:46:52.333854+0000","last_fullsized":"2023-11-16T16:46:52.333854+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:53:12.394116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.640943+0000","last_change":"2023-11-16T16:46:36.606973+0000","last_active":"2023-11-16T16:47:08.640943+0000","last_peered":"2023-11-16T16:47:08.640943+0000","last_clean":"2023-11-16T16:47:08.640943+0000","last_became_active":"2023-11-16T16:46:36.606744+0000","last_became_peered":"2023-11-16T16:46:36.606744+0000","last_unstale":"2023-11-16T16:47:08.640943+0000","last_undegraded":"2023-11-16T16:47:08.640943+0000","last_fullsized":"2023-11-16T16:47:08.640943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:55:14.632895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330683+0000","last_change":"2023-11-16T16:46:37.159696+0000","last_active":"2023-11-16T16:46:52.330683+0000","last_peered":"2023-11-16T16:46:52.330683+0000","last_clean":"2023-11-16T16:46:52.330683+0000","last_became_active":"2023-11-16T16:46:37.156808+0000","last_became_peered":"2023-11-16T16:46:37.156808+0000","last_unstale":"2023-11-16T16:46:52.330683+0000","last_undegraded":"2023-11-16T16:46:52.330683+0000","last_fullsized":"2023-11-16T16:46:52.330683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:21:37.020133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335419+0000","last_change":"2023-11-16T16:46:41.204645+0000","last_active":"2023-11-16T16:46:52.335419+0000","last_peered":"2023-11-16T16:46:52.335419+0000","last_clean":"2023-11-16T16:46:52.335419+0000","last_became_active":"2023-11-16T16:46:41.204167+0000","last_became_peered":"2023-11-16T16:46:41.204167+0000","last_unstale":"2023-11-16T16:46:52.335419+0000","last_undegraded":"2023-11-16T16:46:52.335419+0000","last_fullsized":"2023-11-16T16:46:52.335419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:56:09.635720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.642622+0000","last_change":"2023-11-16T16:46:52.334958+0000","last_active":"2023-11-16T16:47:08.642622+0000","last_peered":"2023-11-16T16:47:08.642622+0000","last_clean":"2023-11-16T16:47:08.642622+0000","last_became_active":"2023-11-16T16:46:52.334607+0000","last_became_peered":"2023-11-16T16:46:52.334607+0000","last_unstale":"2023-11-16T16:47:08.642622+0000","last_undegraded":"2023-11-16T16:47:08.642622+0000","last_fullsized":"2023-11-16T16:47:08.642622+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:59:19.819869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331676+0000","last_change":"2023-11-16T16:46:37.153357+0000","last_active":"2023-11-16T16:46:52.331676+0000","last_peered":"2023-11-16T16:46:52.331676+0000","last_clean":"2023-11-16T16:46:52.331676+0000","last_became_active":"2023-11-16T16:46:37.153064+0000","last_became_peered":"2023-11-16T16:46:37.153064+0000","last_unstale":"2023-11-16T16:46:52.331676+0000","last_undegraded":"2023-11-16T16:46:52.331676+0000","last_fullsized":"2023-11-16T16:46:52.331676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:54:56.156923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331313+0000","last_change":"2023-11-16T16:46:41.183938+0000","last_active":"2023-11-16T16:46:52.331313+0000","last_peered":"2023-11-16T16:46:52.331313+0000","last_clean":"2023-11-16T16:46:52.331313+0000","last_became_active":"2023-11-16T16:46:41.181850+0000","last_became_peered":"2023-11-16T16:46:41.181850+0000","last_unstale":"2023-11-16T16:46:52.331313+0000","last_undegraded":"2023-11-16T16:46:52.331313+0000","last_fullsized":"2023-11-16T16:46:52.331313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:03:51.603935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.643670+0000","last_change":"2023-11-16T16:46:36.603257+0000","last_active":"2023-11-16T16:47:08.643670+0000","last_peered":"2023-11-16T16:47:08.643670+0000","last_clean":"2023-11-16T16:47:08.643670+0000","last_became_active":"2023-11-16T16:46:36.602852+0000","last_became_peered":"2023-11-16T16:46:36.602852+0000","last_unstale":"2023-11-16T16:47:08.643670+0000","last_undegraded":"2023-11-16T16:47:08.643670+0000","last_fullsized":"2023-11-16T16:47:08.643670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:19:39.760959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335812+0000","last_change":"2023-11-16T16:46:37.162689+0000","last_active":"2023-11-16T16:46:52.335812+0000","last_peered":"2023-11-16T16:46:52.335812+0000","last_clean":"2023-11-16T16:46:52.335812+0000","last_became_active":"2023-11-16T16:46:37.162356+0000","last_became_peered":"2023-11-16T16:46:37.162356+0000","last_unstale":"2023-11-16T16:46:52.335812+0000","last_undegraded":"2023-11-16T16:46:52.335812+0000","last_fullsized":"2023-11-16T16:46:52.335812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:07:29.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334687+0000","last_change":"2023-11-16T16:46:41.182286+0000","last_active":"2023-11-16T16:46:52.334687+0000","last_peered":"2023-11-16T16:46:52.334687+0000","last_clean":"2023-11-16T16:46:52.334687+0000","last_became_active":"2023-11-16T16:46:41.181820+0000","last_became_peered":"2023-11-16T16:46:41.181820+0000","last_unstale":"2023-11-16T16:46:52.334687+0000","last_undegraded":"2023-11-16T16:46:52.334687+0000","last_fullsized":"2023-11-16T16:46:52.334687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:56:24.412595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334978+0000","last_change":"2023-11-16T16:46:36.598952+0000","last_active":"2023-11-16T16:46:52.334978+0000","last_peered":"2023-11-16T16:46:52.334978+0000","last_clean":"2023-11-16T16:46:52.334978+0000","last_became_active":"2023-11-16T16:46:36.598617+0000","last_became_peered":"2023-11-16T16:46:36.598617+0000","last_unstale":"2023-11-16T16:46:52.334978+0000","last_undegraded":"2023-11-16T16:46:52.334978+0000","last_fullsized":"2023-11-16T16:46:52.334978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:03:48.791066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.045102+0000","last_change":"2023-11-16T16:46:37.154350+0000","last_active":"2023-11-16T16:46:53.045102+0000","last_peered":"2023-11-16T16:46:53.045102+0000","last_clean":"2023-11-16T16:46:53.045102+0000","last_became_active":"2023-11-16T16:46:37.154045+0000","last_became_peered":"2023-11-16T16:46:37.154045+0000","last_unstale":"2023-11-16T16:46:53.045102+0000","last_undegraded":"2023-11-16T16:46:53.045102+0000","last_fullsized":"2023-11-16T16:46:53.045102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:22:21.421844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334540+0000","last_change":"2023-11-16T16:46:41.202563+0000","last_active":"2023-11-16T16:46:52.334540+0000","last_peered":"2023-11-16T16:46:52.334540+0000","last_clean":"2023-11-16T16:46:52.334540+0000","last_became_active":"2023-11-16T16:46:41.202125+0000","last_became_peered":"2023-11-16T16:46:41.202125+0000","last_unstale":"2023-11-16T16:46:52.334540+0000","last_undegraded":"2023-11-16T16:46:52.334540+0000","last_fullsized":"2023-11-16T16:46:52.334540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:15:43.049582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:47:08.643140+0000","last_change":"2023-11-16T16:46:52.341548+0000","last_active":"2023-11-16T16:47:08.643140+0000","last_peered":"2023-11-16T16:47:08.643140+0000","last_clean":"2023-11-16T16:47:08.643140+0000","last_became_active":"2023-11-16T16:46:52.337771+0000","last_became_peered":"2023-11-16T16:46:52.337771+0000","last_unstale":"2023-11-16T16:47:08.643140+0000","last_undegraded":"2023-11-16T16:47:08.643140+0000","last_fullsized":"2023-11-16T16:47:08.643140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:17.479630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330523+0000","last_change":"2023-11-16T16:46:37.159095+0000","last_active":"2023-11-16T16:46:52.330523+0000","last_peered":"2023-11-16T16:46:52.330523+0000","last_clean":"2023-11-16T16:46:52.330523+0000","last_became_active":"2023-11-16T16:46:37.158668+0000","last_became_peered":"2023-11-16T16:46:37.158668+0000","last_unstale":"2023-11-16T16:46:52.330523+0000","last_undegraded":"2023-11-16T16:46:52.330523+0000","last_fullsized":"2023-11-16T16:46:52.330523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:03:24.065868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.332116+0000","last_change":"2023-11-16T16:46:41.165572+0000","last_active":"2023-11-16T16:46:52.332116+0000","last_peered":"2023-11-16T16:46:52.332116+0000","last_clean":"2023-11-16T16:46:52.332116+0000","last_became_active":"2023-11-16T16:46:41.165174+0000","last_became_peered":"2023-11-16T16:46:41.165174+0000","last_unstale":"2023-11-16T16:46:52.332116+0000","last_undegraded":"2023-11-16T16:46:52.332116+0000","last_fullsized":"2023-11-16T16:46:52.332116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:54:13.439807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.337265+0000","last_change":"2023-11-16T16:46:36.593871+0000","last_active":"2023-11-16T16:46:52.337265+0000","last_peered":"2023-11-16T16:46:52.337265+0000","last_clean":"2023-11-16T16:46:52.337265+0000","last_became_active":"2023-11-16T16:46:36.593579+0000","last_became_peered":"2023-11-16T16:46:36.593579+0000","last_unstale":"2023-11-16T16:46:52.337265+0000","last_undegraded":"2023-11-16T16:46:52.337265+0000","last_fullsized":"2023-11-16T16:46:52.337265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:11:16.979596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331375+0000","last_change":"2023-11-16T16:46:37.140823+0000","last_active":"2023-11-16T16:46:52.331375+0000","last_peered":"2023-11-16T16:46:52.331375+0000","last_clean":"2023-11-16T16:46:52.331375+0000","last_became_active":"2023-11-16T16:46:37.140505+0000","last_became_peered":"2023-11-16T16:46:37.140505+0000","last_unstale":"2023-11-16T16:46:52.331375+0000","last_undegraded":"2023-11-16T16:46:52.331375+0000","last_fullsized":"2023-11-16T16:46:52.331375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:46:42.999149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.329464+0000","last_change":"2023-11-16T16:46:41.182041+0000","last_active":"2023-11-16T16:46:52.329464+0000","last_peered":"2023-11-16T16:46:52.329464+0000","last_clean":"2023-11-16T16:46:52.329464+0000","last_became_active":"2023-11-16T16:46:41.181420+0000","last_became_peered":"2023-11-16T16:46:41.181420+0000","last_unstale":"2023-11-16T16:46:52.329464+0000","last_undegraded":"2023-11-16T16:46:52.329464+0000","last_fullsized":"2023-11-16T16:46:52.329464+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:39:34.693599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.328130+0000","last_change":"2023-11-16T16:46:36.602184+0000","last_active":"2023-11-16T16:46:52.328130+0000","last_peered":"2023-11-16T16:46:52.328130+0000","last_clean":"2023-11-16T16:46:52.328130+0000","last_became_active":"2023-11-16T16:46:36.601893+0000","last_became_peered":"2023-11-16T16:46:36.601893+0000","last_unstale":"2023-11-16T16:46:52.328130+0000","last_undegraded":"2023-11-16T16:46:52.328130+0000","last_fullsized":"2023-11-16T16:46:52.328130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:07:44.033216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.330760+0000","last_change":"2023-11-16T16:46:37.153649+0000","last_active":"2023-11-16T16:46:52.330760+0000","last_peered":"2023-11-16T16:46:52.330760+0000","last_clean":"2023-11-16T16:46:52.330760+0000","last_became_active":"2023-11-16T16:46:37.152420+0000","last_became_peered":"2023-11-16T16:46:37.152420+0000","last_unstale":"2023-11-16T16:46:52.330760+0000","last_undegraded":"2023-11-16T16:46:52.330760+0000","last_fullsized":"2023-11-16T16:46:52.330760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:26:13.077534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327051+0000","last_change":"2023-11-16T16:46:41.168363+0000","last_active":"2023-11-16T16:46:52.327051+0000","last_peered":"2023-11-16T16:46:52.327051+0000","last_clean":"2023-11-16T16:46:52.327051+0000","last_became_active":"2023-11-16T16:46:41.167615+0000","last_became_peered":"2023-11-16T16:46:41.167615+0000","last_unstale":"2023-11-16T16:46:52.327051+0000","last_undegraded":"2023-11-16T16:46:52.327051+0000","last_fullsized":"2023-11-16T16:46:52.327051+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:39:53.383890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426560+0000","last_change":"2023-11-16T16:46:36.611353+0000","last_active":"2023-11-16T16:46:52.426560+0000","last_peered":"2023-11-16T16:46:52.426560+0000","last_clean":"2023-11-16T16:46:52.426560+0000","last_became_active":"2023-11-16T16:46:36.610886+0000","last_became_peered":"2023-11-16T16:46:36.610886+0000","last_unstale":"2023-11-16T16:46:52.426560+0000","last_undegraded":"2023-11-16T16:46:52.426560+0000","last_fullsized":"2023-11-16T16:46:52.426560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:10:20.922368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.331449+0000","last_change":"2023-11-16T16:46:37.146227+0000","last_active":"2023-11-16T16:46:52.331449+0000","last_peered":"2023-11-16T16:46:52.331449+0000","last_clean":"2023-11-16T16:46:52.331449+0000","last_became_active":"2023-11-16T16:46:37.143239+0000","last_became_peered":"2023-11-16T16:46:37.143239+0000","last_unstale":"2023-11-16T16:46:52.331449+0000","last_undegraded":"2023-11-16T16:46:52.331449+0000","last_fullsized":"2023-11-16T16:46:52.331449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:44:19.791930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335938+0000","last_change":"2023-11-16T16:46:37.153845+0000","last_active":"2023-11-16T16:46:52.335938+0000","last_peered":"2023-11-16T16:46:52.335938+0000","last_clean":"2023-11-16T16:46:52.335938+0000","last_became_active":"2023-11-16T16:46:37.153505+0000","last_became_peered":"2023-11-16T16:46:37.153505+0000","last_unstale":"2023-11-16T16:46:52.335938+0000","last_undegraded":"2023-11-16T16:46:52.335938+0000","last_fullsized":"2023-11-16T16:46:52.335938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:26:52.188071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335325+0000","last_change":"2023-11-16T16:46:37.148451+0000","last_active":"2023-11-16T16:46:52.335325+0000","last_peered":"2023-11-16T16:46:52.335325+0000","last_clean":"2023-11-16T16:46:52.335325+0000","last_became_active":"2023-11-16T16:46:37.148142+0000","last_became_peered":"2023-11-16T16:46:37.148142+0000","last_unstale":"2023-11-16T16:46:52.335325+0000","last_undegraded":"2023-11-16T16:46:52.335325+0000","last_fullsized":"2023-11-16T16:46:52.335325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:05:12.660394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.334810+0000","last_change":"2023-11-16T16:46:37.150360+0000","last_active":"2023-11-16T16:46:52.334810+0000","last_peered":"2023-11-16T16:46:52.334810+0000","last_clean":"2023-11-16T16:46:52.334810+0000","last_became_active":"2023-11-16T16:46:37.150045+0000","last_became_peered":"2023-11-16T16:46:37.150045+0000","last_unstale":"2023-11-16T16:46:52.334810+0000","last_undegraded":"2023-11-16T16:46:52.334810+0000","last_fullsized":"2023-11-16T16:46:52.334810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:54:53.279485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.329861+0000","last_change":"2023-11-16T16:46:37.161736+0000","last_active":"2023-11-16T16:46:52.329861+0000","last_peered":"2023-11-16T16:46:52.329861+0000","last_clean":"2023-11-16T16:46:52.329861+0000","last_became_active":"2023-11-16T16:46:37.158273+0000","last_became_peered":"2023-11-16T16:46:37.158273+0000","last_unstale":"2023-11-16T16:46:52.329861+0000","last_undegraded":"2023-11-16T16:46:52.329861+0000","last_fullsized":"2023-11-16T16:46:52.329861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:38:18.599194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426259+0000","last_change":"2023-11-16T16:46:37.163725+0000","last_active":"2023-11-16T16:46:52.426259+0000","last_peered":"2023-11-16T16:46:52.426259+0000","last_clean":"2023-11-16T16:46:52.426259+0000","last_became_active":"2023-11-16T16:46:37.163180+0000","last_became_peered":"2023-11-16T16:46:37.163180+0000","last_unstale":"2023-11-16T16:46:52.426259+0000","last_undegraded":"2023-11-16T16:46:52.426259+0000","last_fullsized":"2023-11-16T16:46:52.426259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:57:30.898710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.335260+0000","last_change":"2023-11-16T16:46:37.154475+0000","last_active":"2023-11-16T16:46:52.335260+0000","last_peered":"2023-11-16T16:46:52.335260+0000","last_clean":"2023-11-16T16:46:52.335260+0000","last_became_active":"2023-11-16T16:46:37.154161+0000","last_became_peered":"2023-11-16T16:46:37.154161+0000","last_unstale":"2023-11-16T16:46:52.335260+0000","last_undegraded":"2023-11-16T16:46:52.335260+0000","last_fullsized":"2023-11-16T16:46:52.335260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:28.955298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336724+0000","last_change":"2023-11-16T16:46:37.140293+0000","last_active":"2023-11-16T16:46:52.336724+0000","last_peered":"2023-11-16T16:46:52.336724+0000","last_clean":"2023-11-16T16:46:52.336724+0000","last_became_active":"2023-11-16T16:46:37.139966+0000","last_became_peered":"2023-11-16T16:46:37.139966+0000","last_unstale":"2023-11-16T16:46:52.336724+0000","last_undegraded":"2023-11-16T16:46:52.336724+0000","last_fullsized":"2023-11-16T16:46:52.336724+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:02:02.311770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.045428+0000","last_change":"2023-11-16T16:46:37.134807+0000","last_active":"2023-11-16T16:46:53.045428+0000","last_peered":"2023-11-16T16:46:53.045428+0000","last_clean":"2023-11-16T16:46:53.045428+0000","last_became_active":"2023-11-16T16:46:37.131833+0000","last_became_peered":"2023-11-16T16:46:37.131833+0000","last_unstale":"2023-11-16T16:46:53.045428+0000","last_undegraded":"2023-11-16T16:46:53.045428+0000","last_fullsized":"2023-11-16T16:46:53.045428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:05:04.767346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.327181+0000","last_change":"2023-11-16T16:46:37.152424+0000","last_active":"2023-11-16T16:46:52.327181+0000","last_peered":"2023-11-16T16:46:52.327181+0000","last_clean":"2023-11-16T16:46:52.327181+0000","last_became_active":"2023-11-16T16:46:37.152082+0000","last_became_peered":"2023-11-16T16:46:37.152082+0000","last_unstale":"2023-11-16T16:46:52.327181+0000","last_undegraded":"2023-11-16T16:46:52.327181+0000","last_fullsized":"2023-11-16T16:46:52.327181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:14:36.269833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:53.044568+0000","last_change":"2023-11-16T16:46:37.140267+0000","last_active":"2023-11-16T16:46:53.044568+0000","last_peered":"2023-11-16T16:46:53.044568+0000","last_clean":"2023-11-16T16:46:53.044568+0000","last_became_active":"2023-11-16T16:46:37.139689+0000","last_became_peered":"2023-11-16T16:46:37.139689+0000","last_unstale":"2023-11-16T16:46:53.044568+0000","last_undegraded":"2023-11-16T16:46:53.044568+0000","last_fullsized":"2023-11-16T16:46:53.044568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:55:24.624825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.332484+0000","last_change":"2023-11-16T16:46:37.151911+0000","last_active":"2023-11-16T16:46:52.332484+0000","last_peered":"2023-11-16T16:46:52.332484+0000","last_clean":"2023-11-16T16:46:52.332484+0000","last_became_active":"2023-11-16T16:46:37.151614+0000","last_became_peered":"2023-11-16T16:46:37.151614+0000","last_unstale":"2023-11-16T16:46:52.332484+0000","last_undegraded":"2023-11-16T16:46:52.332484+0000","last_fullsized":"2023-11-16T16:46:52.332484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:04:10.208002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.426197+0000","last_change":"2023-11-16T16:46:37.153265+0000","last_active":"2023-11-16T16:46:52.426197+0000","last_peered":"2023-11-16T16:46:52.426197+0000","last_clean":"2023-11-16T16:46:52.426197+0000","last_became_active":"2023-11-16T16:46:37.152933+0000","last_became_peered":"2023-11-16T16:46:37.152933+0000","last_unstale":"2023-11-16T16:46:52.426197+0000","last_undegraded":"2023-11-16T16:46:52.426197+0000","last_fullsized":"2023-11-16T16:46:52.426197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:23:01.775314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-16T16:46:52.336215+0000","last_change":"2023-11-16T16:46:37.142831+0000","last_active":"2023-11-16T16:46:52.336215+0000","last_peered":"2023-11-16T16:46:52.336215+0000","last_clean":"2023-11-16T16:46:52.336215+0000","last_became_active":"2023-11-16T16:46:37.142480+0000","last_became_peered":"2023-11-16T16:46:37.142480+0000","last_unstale":"2023-11-16T16:46:52.336215+0000","last_undegraded":"2023-11-16T16:46:52.336215+0000","last_fullsized":"2023-11-16T16:46:52.336215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:09:49.039282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7116,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":109,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":35351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6176,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93738976,"statfs":{"total":95995035648,"available":95988711424,"internally_reserved":0,"allocated":1007616,"data_stored":493331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44800000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.65300000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":0.73999999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72699999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.75}]},{"osd":10,"last update":"Thu Nov 16 16:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.63300000000000001}]}]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6900,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738252,"statfs":{"total":95995035648,"available":95987970048,"internally_reserved":0,"allocated":1617920,"data_stored":1083009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":1,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":4.1299999999999999,"5min":4.1299999999999999,"15min":4.1299999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":4.1769999999999996,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.56599999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.78000000000000003}]},{"osd":9,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.879},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.81799999999999995}]},{"osd":11,"last update":"Thu Nov 16 16:46:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.85599999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6384,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738768,"statfs":{"total":95995035648,"available":95988498432,"internally_reserved":0,"allocated":1024000,"data_stored":493567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.55100000000000005,"15min":0.56399999999999995},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72599999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.435},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.749,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.54900000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.495}]},{"osd":2,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72999999999999998,"15min":0.73899999999999999},"min":{"1min":0.47899999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96999999999999997,"5min":1.048,"15min":1.048},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.85899999999999999,"15min":0.89100000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98699999999999999,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67900000000000005,"15min":0.68999999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78000000000000003,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.79100000000000004,"15min":0.80900000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.056,"5min":1.24,"15min":1.24},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.82399999999999995,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1120000000000001,"5min":1.216,"15min":1.216},"last":0.84199999999999997}]},{"osd":5,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.82199999999999995,"15min":0.84399999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95899999999999996,"5min":1.304,"15min":1.304},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.86799999999999999,"15min":0.88600000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.145,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.88200000000000001}]},{"osd":6,"last update":"Thu Nov 16 16:46:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.92500000000000004},"min":{"1min":0.61899999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.111,"5min":1.282,"15min":1.282},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.98899999999999999,"15min":1.0229999999999999},"min":{"1min":0.57799999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0860000000000001,"5min":4.5650000000000004,"15min":4.5650000000000004},"last":0.86199999999999999}]},{"osd":8,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1,"15min":1.0389999999999999},"min":{"1min":0.55300000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.163,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.93799999999999994,"15min":0.96699999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.077,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.82699999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":4.4870000000000001,"5min":4.4870000000000001,"15min":4.4870000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.78700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":8,"up_from":59,"seq":253403070498,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6504,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738648,"statfs":{"total":95995035648,"available":95988375552,"internally_reserved":0,"allocated":1015808,"data_stored":489060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.498},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.59799999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.753,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71799999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.46400000000000002,"15min":0.47399999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.66900000000000004,"15min":0.68700000000000006},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81499999999999995,"5min":1.536,"15min":1.536},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.68200000000000005,"15min":0.70099999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78800000000000003,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82599999999999996,"5min":0.88,"15min":0.88},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.55200000000000005,"15min":0.56599999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73899999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84599999999999997,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.72699999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.74299999999999999,"15min":0.76700000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.871,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93100000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.80300000000000005}]},{"osd":9,"last update":"Thu Nov 16 16:47:03 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.79100000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.874}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7068,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738084,"statfs":{"total":95995035648,"available":95987798016,"internally_reserved":0,"allocated":1593344,"data_stored":1079272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.27100000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.60199999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68799999999999994,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.89500000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.52100000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66700000000000004}]},{"osd":4,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.30399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.62},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84599999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81999999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71099999999999997,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.497,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88300000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65400000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.51700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.88900000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90600000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.60899999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93600000000000005,"5min":1.054,"15min":1.054},"last":1.0620000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.95499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6428,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738724,"statfs":{"total":95995035648,"available":95988453376,"internally_reserved":0,"allocated":1003520,"data_stored":488743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.36599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71999999999999997,"5min":0.87,"15min":0.87},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.36199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78300000000000003,"5min":1.139,"15min":1.139},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89000000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85299999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63,"15min":0.622},"min":{"1min":0.501,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.753,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.67500000000000004,"15min":0.69499999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":1.25,"15min":1.25},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.96699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.625,"15min":0.629},"min":{"1min":0.40600000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72499999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.53100000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89300000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73499999999999999,"15min":0.749},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88800000000000001,"5min":1.343,"15min":1.343},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.54600000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.91300000000000003,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.74299999999999999,"15min":0.73799999999999999},"min":{"1min":0.51600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.93500000000000005,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.73999999999999999,"15min":0.73299999999999998},"min":{"1min":0.57099999999999995,"5min":0.438,"15min":0.438},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.73799999999999999,"15min":0.72599999999999998},"min":{"1min":0.53800000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.90000000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.79000000000000004,"15min":0.79400000000000004},"min":{"1min":0.53600000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.0920000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79700000000000004,"15min":0.79900000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.97699999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.86699999999999999,"15min":0.877},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.089,"5min":1.165,"15min":1.165},"last":0.872},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84299999999999997,"15min":0.84699999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.097,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.88300000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.92700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7396,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737756,"statfs":{"total":95995035648,"available":95987462144,"internally_reserved":0,"allocated":1011712,"data_stored":492447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52200000000000002,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59899999999999998,"5min":1.431,"15min":1.431},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57299999999999995,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76500000000000001,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Nov 16 16:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.41299999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71199999999999997}]},{"osd":3,"last update":"Thu Nov 16 16:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.54500000000000004},"min":{"1min":0.41499999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83699999999999997,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59099999999999997,"15min":0.56999999999999995},"min":{"1min":0.46500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.89800000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72999999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.40899999999999997,"15min":0.39200000000000002},"min":{"1min":0.32200000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.432,"15min":0.40300000000000002},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.55900000000000005,"15min":0.52800000000000002},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.91400000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.58399999999999996,"15min":0.54600000000000004},"min":{"1min":0.60399999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92900000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.78000000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.58999999999999997},"min":{"1min":0.48199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85899999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.626,"15min":0.60199999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.48399999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.55700000000000005},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.44700000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72699999999999998,"15min":0.72299999999999998},"min":{"1min":0.51800000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94399999999999995,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.54900000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65900000000000003}]},{"osd":9,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94299999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.036,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.59799999999999998}]},{"osd":10,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.61599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7388,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737764,"statfs":{"total":95995035648,"available":95987470336,"internally_reserved":0,"allocated":1003520,"data_stored":488743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89100000000000001,"5min":1.01,"15min":1.01},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.61699999999999999,"15min":0.55900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.752}]},{"osd":2,"last update":"Thu Nov 16 16:46:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.58399999999999996,"15min":0.54600000000000004},"min":{"1min":0.44,"5min":0.373,"15min":0.373},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.55300000000000005},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92600000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.50900000000000001,"15min":0.46999999999999997},"min":{"1min":0.33300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.54200000000000004,"15min":0.51900000000000002},"min":{"1min":0.39100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77300000000000002,"5min":0.878,"15min":0.878},"last":0.72299999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.626,"15min":0.60299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60499999999999998,"15min":0.57899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96899999999999997,"5min":1.008,"15min":1.008},"last":0.877}]},{"osd":6,"last update":"Thu Nov 16 16:46:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.90900000000000003,"5min":1.03,"15min":1.03},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Nov 16 16:46:16 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.94199999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73899999999999999,"15min":0.73999999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.79100000000000004}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.79000000000000004,"15min":0.80500000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81100000000000005,"15min":0.81299999999999994},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.121,"5min":1.145,"15min":1.145},"last":1.0780000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":1.167}]},{"osd":11,"last update":"Thu Nov 16 16:46:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":1.214},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.133}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7400,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737752,"statfs":{"total":95995035648,"available":95987458048,"internally_reserved":0,"allocated":1015808,"data_stored":492481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.55600000000000005},"min":{"1min":0.40899999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70699999999999996,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56000000000000005,"15min":0.54000000000000004},"min":{"1min":0.378,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67800000000000005,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Nov 16 16:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71599999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.52500000000000002},"min":{"1min":0.48799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.69099999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.66700000000000004},"min":{"1min":0.44800000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.5509999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61499999999999999,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:46:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61399999999999999,"15min":0.63300000000000001},"min":{"1min":0.42199999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.73699999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.68100000000000005,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81200000000000006,"5min":4.181,"15min":4.181},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Nov 16 16:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45900000000000002,"15min":0.44600000000000001},"min":{"1min":0.33500000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78700000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.629},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.47499999999999998,"15min":0.46400000000000002},"min":{"1min":0.32100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67400000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67500000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64200000000000002,"15min":0.622},"min":{"1min":0.47499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.441,"5min":0.437,"15min":0.437},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76100000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.48499999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.84499999999999997,"5min":0.876,"15min":0.876},"last":0.76900000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64200000000000002,"15min":0.64500000000000002},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.88700000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.748},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.624,"15min":0.627},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81299999999999994,"5min":0.872,"15min":0.872},"last":0.84999999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.745,"15min":0.72999999999999998},"min":{"1min":0.63900000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.77500000000000002,"15min":0.76100000000000001},"min":{"1min":0.59099999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.95299999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.97799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6756,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738396,"statfs":{"total":95995035648,"available":95988117504,"internally_reserved":0,"allocated":1011712,"data_stored":488847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.42299999999999999,"15min":0.38600000000000001},"min":{"1min":0.30399999999999999,"5min":0.246,"15min":0.246},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.748},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.502,"15min":0.46500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.80400000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61399999999999999,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60099999999999998,"15min":0.57399999999999995},"min":{"1min":0.497,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Nov 16 16:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.64400000000000002,"15min":0.61599999999999999},"min":{"1min":0.47399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.016,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.621},"min":{"1min":0.443,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.98099999999999998,"5min":1.228,"15min":1.228},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.624,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.503}]},{"osd":5,"last update":"Thu Nov 16 16:46:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85699999999999998,"5min":1.133,"15min":1.133},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.48499999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95399999999999996,"5min":1.149,"15min":1.149},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Nov 16 16:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.77100000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68999999999999995,"15min":0.68500000000000005},"min":{"1min":0.45400000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.84099999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.76700000000000002,"15min":0.76400000000000001},"min":{"1min":0.501,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.073,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.82799999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.753,"15min":0.753},"min":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1559999999999999,"5min":1.171,"15min":1.171},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.60699999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.306}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.871},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81599999999999995,"15min":0.81899999999999995},"min":{"1min":0.64500000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.84999999999999998}]},{"osd":10,"last update":"Thu Nov 16 16:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.61799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6628,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738524,"statfs":{"total":95995035648,"available":95988248576,"internally_reserved":0,"allocated":1011712,"data_stored":493404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.52500000000000002,"15min":0.501},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.377,"15min":0.34799999999999998},"min":{"1min":0.33700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.38},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.35199999999999998},"min":{"1min":0.26500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.5,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.36799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.754,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.53900000000000003,"15min":0.51000000000000001},"min":{"1min":0.45200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72599999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58399999999999996,"15min":0.53800000000000003},"min":{"1min":0.44400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75700000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.623},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82399999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54200000000000004,"15min":0.53100000000000003},"min":{"1min":0.41999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57799999999999996,"15min":0.55800000000000005},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Nov 16 16:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.40899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83699999999999997,"5min":1.097,"15min":1.097},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.496,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.60799999999999998,"15min":0.59199999999999997},"min":{"1min":0.47899999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.76900000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.498,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.68200000000000005,"15min":0.66600000000000004},"min":{"1min":0.54900000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85999999999999999}]},{"osd":9,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.80000000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.92900000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73099999999999998,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.878,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54600000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:46:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7284,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737868,"statfs":{"total":95995035648,"available":95987576832,"internally_reserved":0,"allocated":1617920,"data_stored":1080118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57399999999999995,"15min":0.56299999999999994},"min":{"1min":0.42699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77200000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.42899999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73199999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.503},"min":{"1min":0.41499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.755}]},{"osd":2,"last update":"Thu Nov 16 16:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.38900000000000001},"min":{"1min":0.308,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66700000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.40999999999999998,"15min":0.38800000000000001},"min":{"1min":0.28899999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79000000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.44,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57499999999999996,"15min":0.54700000000000004},"min":{"1min":0.46100000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.56000000000000005,"15min":0.54500000000000004},"min":{"1min":0.38400000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64900000000000002,"15min":0.63500000000000001},"min":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67300000000000004,"15min":0.65500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.57399999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.44400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.69099999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.67500000000000004}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.75,"15min":0.746},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.74099999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:46:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":21175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:47:14.141 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-16T16:47:14.141 INFO:tasks.cephadm:Setting up client nodes... 2023-11-16T16:47:14.142 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-11-16T16:47:14.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:14 smithi159 ceph-mon[131528]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:47:14.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:14 smithi159 ceph-mon[131528]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:47:15.021 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:15.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:14 smithi125 ceph-mon[131076]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:47:15.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:14 smithi125 ceph-mon[131076]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:47:15.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:14 smithi093 ceph-mon[97863]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:47:15.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:14 smithi093 ceph-mon[97863]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:47:15.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:15 smithi093 ceph-mon[97863]: pgmap v302: 129 pgs: 129 active+clean; 618 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.5 KiB/s wr, 10 op/s 2023-11-16T16:47:15.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:15 smithi159 ceph-mon[131528]: pgmap v302: 129 pgs: 129 active+clean; 618 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.5 KiB/s wr, 10 op/s 2023-11-16T16:47:16.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:15 smithi125 ceph-mon[131076]: pgmap v302: 129 pgs: 129 active+clean; 618 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.5 KiB/s wr, 10 op/s 2023-11-16T16:47:16.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.431+0000 7f9d25c1d700 1 -- 172.21.15.93:0/2655271492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d20100370 msgr2=0x7f9d20100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:16.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.431+0000 7f9d25c1d700 1 --2- 172.21.15.93:0/2655271492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d20100370 0x7f9d20100780 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009990 tx=0x7f9d10009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:16.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.431+0000 7f9d25c1d700 1 -- 172.21.15.93:0/2655271492 shutdown_connections 2023-11-16T16:47:16.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.431+0000 7f9d25c1d700 1 --2- 172.21.15.93:0/2655271492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d20100370 0x7f9d20100780 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:16.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.431+0000 7f9d25c1d700 1 --2- 172.21.15.93:0/2655271492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d20101f00 0x7f9d201087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:16.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.431+0000 7f9d25c1d700 1 --2- 172.21.15.93:0/2655271492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20101570 0x7f9d201019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:16.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.431+0000 7f9d25c1d700 1 -- 172.21.15.93:0/2655271492 >> 172.21.15.93:0/2655271492 conn(0x7f9d200fb9a0 msgr2=0x7f9d200fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:16.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.431+0000 7f9d25c1d700 1 -- 172.21.15.93:0/2655271492 shutdown_connections 2023-11-16T16:47:16.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.432+0000 7f9d25c1d700 1 -- 172.21.15.93:0/2655271492 wait complete. 2023-11-16T16:47:16.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.433+0000 7f9d25c1d700 1 Processor -- start 2023-11-16T16:47:16.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.433+0000 7f9d25c1d700 1 -- start start 2023-11-16T16:47:16.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.433+0000 7f9d25c1d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d20100370 0x7f9d20198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:16.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.433+0000 7f9d25c1d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d20101570 0x7f9d201987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:16.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.433+0000 7f9d25c1d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20101f00 0x7f9d2019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:16.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.433+0000 7f9d25c1d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2019fe10 con 0x7f9d20100370 2023-11-16T16:47:16.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.433+0000 7f9d25c1d700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2019ff50 con 0x7f9d20101570 2023-11-16T16:47:16.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.433+0000 7f9d25c1d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d201a00e0 con 0x7f9d20101f00 2023-11-16T16:47:16.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.433+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d20100370 0x7f9d20198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:16.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.433+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d20100370 0x7f9d20198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42584/0 (socket says 172.21.15.93:42584) 2023-11-16T16:47:16.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.433+0000 7f9d1f7fe700 1 -- 172.21.15.93:0/3542214331 learned_addr learned my addr 172.21.15.93:0/3542214331 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:16.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.434+0000 7f9d1ffff700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20101f00 0x7f9d2019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:16.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.434+0000 7f9d1effd700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d20101570 0x7f9d201987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:16.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.434+0000 7f9d1f7fe700 1 -- 172.21.15.93:0/3542214331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20101f00 msgr2=0x7f9d2019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:16.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.434+0000 7f9d1f7fe700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20101f00 0x7f9d2019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:16.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.434+0000 7f9d1f7fe700 1 -- 172.21.15.93:0/3542214331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d20101570 msgr2=0x7f9d201987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:16.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.434+0000 7f9d1f7fe700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d20101570 0x7f9d201987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:16.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.434+0000 7f9d1f7fe700 1 -- 172.21.15.93:0/3542214331 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d100095f0 con 0x7f9d20100370 2023-11-16T16:47:16.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.434+0000 7f9d1f7fe700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d20100370 0x7f9d20198280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9d10004740 tx=0x7f9d10009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:16.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.434+0000 7f9d1cff9700 1 -- 172.21.15.93:0/3542214331 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1001d070 con 0x7f9d20100370 2023-11-16T16:47:16.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.434+0000 7f9d1cff9700 1 -- 172.21.15.93:0/3542214331 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d10021410 con 0x7f9d20100370 2023-11-16T16:47:16.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.434+0000 7f9d25c1d700 1 -- 172.21.15.93:0/3542214331 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2010bd10 con 0x7f9d20100370 2023-11-16T16:47:16.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.434+0000 7f9d1cff9700 1 -- 172.21.15.93:0/3542214331 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10017650 con 0x7f9d20100370 2023-11-16T16:47:16.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.435+0000 7f9d25c1d700 1 -- 172.21.15.93:0/3542214331 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2010bb80 con 0x7f9d20100370 2023-11-16T16:47:16.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.436+0000 7f9d25c1d700 1 -- 172.21.15.93:0/3542214331 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d201a0670 con 0x7f9d20100370 2023-11-16T16:47:16.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.437+0000 7f9d1cff9700 1 -- 172.21.15.93:0/3542214331 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9d10021580 con 0x7f9d20100370 2023-11-16T16:47:16.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.439+0000 7f9d1cff9700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9d080a11e0 0x7f9d080a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:16.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.439+0000 7f9d1cff9700 1 -- 172.21.15.93:0/3542214331 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9d100ce200 con 0x7f9d20100370 2023-11-16T16:47:16.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.439+0000 7f9d1effd700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9d080a11e0 0x7f9d080a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:16.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.440+0000 7f9d1effd700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9d080a11e0 0x7f9d080a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009c60 tx=0x7f9d14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:16.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.443+0000 7f9d1cff9700 1 -- 172.21.15.93:0/3542214331 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9d10097520 con 0x7f9d20100370 2023-11-16T16:47:16.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.722+0000 7f9d25c1d700 1 -- 172.21.15.93:0/3542214331 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f9d201a0c20 con 0x7f9d20100370 2023-11-16T16:47:16.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.727+0000 7f9d1cff9700 1 -- 172.21.15.93:0/3542214331 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f9d1009ab40 con 0x7f9d20100370 2023-11-16T16:47:16.729 INFO:teuthology.orchestra.run.smithi093.stdout:[client.0] 2023-11-16T16:47:16.729 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCUR1ZlhrUqKxAANhnXpSsA4y9A2fMwy++4iw== 2023-11-16T16:47:16.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.729+0000 7f9d25c1d700 1 -- 172.21.15.93:0/3542214331 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9d080a11e0 msgr2=0x7f9d080a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:16.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.729+0000 7f9d25c1d700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9d080a11e0 0x7f9d080a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009c60 tx=0x7f9d14009260 comp rx=0 tx=0).stop 2023-11-16T16:47:16.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.729+0000 7f9d25c1d700 1 -- 172.21.15.93:0/3542214331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d20100370 msgr2=0x7f9d20198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:16.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.729+0000 7f9d25c1d700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d20100370 0x7f9d20198280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9d10004740 tx=0x7f9d10009580 comp rx=0 tx=0).stop 2023-11-16T16:47:16.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.729+0000 7f9d25c1d700 1 -- 172.21.15.93:0/3542214331 shutdown_connections 2023-11-16T16:47:16.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.729+0000 7f9d25c1d700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9d080a11e0 0x7f9d080a3690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:16.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.730+0000 7f9d25c1d700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d20100370 0x7f9d20198280 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:16.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.730+0000 7f9d25c1d700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20101f00 0x7f9d2019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:16.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.730+0000 7f9d25c1d700 1 --2- 172.21.15.93:0/3542214331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d20101570 0x7f9d201987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:16.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.730+0000 7f9d25c1d700 1 -- 172.21.15.93:0/3542214331 >> 172.21.15.93:0/3542214331 conn(0x7f9d200fb9a0 msgr2=0x7f9d200fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:16.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.730+0000 7f9d25c1d700 1 -- 172.21.15.93:0/3542214331 shutdown_connections 2023-11-16T16:47:16.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:16.730+0000 7f9d25c1d700 1 -- 172.21.15.93:0/3542214331 wait complete. 2023-11-16T16:47:17.218 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:47:17.218 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-11-16T16:47:17.218 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-16T16:47:17.319 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-16T16:47:17.319 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-11-16T16:47:17.319 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph mgr dump --format=json 2023-11-16T16:47:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:17 smithi093 ceph-mon[97863]: pgmap v303: 129 pgs: 129 active+clean; 618 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2023-11-16T16:47:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:17 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3542214331' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-16T16:47:17.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:17 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3542214331' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-16T16:47:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:17 smithi159 ceph-mon[131528]: pgmap v303: 129 pgs: 129 active+clean; 618 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2023-11-16T16:47:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:17 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3542214331' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-16T16:47:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:17 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3542214331' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-16T16:47:18.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:17 smithi125 ceph-mon[131076]: pgmap v303: 129 pgs: 129 active+clean; 618 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2023-11-16T16:47:18.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:17 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3542214331' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-16T16:47:18.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:17 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3542214331' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-16T16:47:18.200 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:19.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.605+0000 7f9d17f7d700 1 -- 172.21.15.93:0/1091415141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d10100ee0 msgr2=0x7f9d10104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.605+0000 7f9d17f7d700 1 --2- 172.21.15.93:0/1091415141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d10100ee0 0x7f9d10104eb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9d000099c0 tx=0x7f9d00009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.605+0000 7f9d17f7d700 1 -- 172.21.15.93:0/1091415141 shutdown_connections 2023-11-16T16:47:19.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.605+0000 7f9d17f7d700 1 --2- 172.21.15.93:0/1091415141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d101055e0 0x7f9d1010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.605+0000 7f9d17f7d700 1 --2- 172.21.15.93:0/1091415141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d10100540 0x7f9d10100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.605+0000 7f9d17f7d700 1 --2- 172.21.15.93:0/1091415141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d10100ee0 0x7f9d10104eb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.605+0000 7f9d17f7d700 1 -- 172.21.15.93:0/1091415141 >> 172.21.15.93:0/1091415141 conn(0x7f9d100fbdc0 msgr2=0x7f9d100fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.606+0000 7f9d17f7d700 1 -- 172.21.15.93:0/1091415141 shutdown_connections 2023-11-16T16:47:19.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.606+0000 7f9d17f7d700 1 -- 172.21.15.93:0/1091415141 wait complete. 2023-11-16T16:47:19.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.607+0000 7f9d17f7d700 1 Processor -- start 2023-11-16T16:47:19.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.607+0000 7f9d17f7d700 1 -- start start 2023-11-16T16:47:19.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.607+0000 7f9d17f7d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d10100540 0x7f9d101985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.607+0000 7f9d17f7d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d10100ee0 0x7f9d10198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.607+0000 7f9d17f7d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d101055e0 0x7f9d1019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.607+0000 7f9d17f7d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1019d5a0 con 0x7f9d101055e0 2023-11-16T16:47:19.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.607+0000 7f9d17f7d700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1019d710 con 0x7f9d10100540 2023-11-16T16:47:19.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.607+0000 7f9d17f7d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1019d8d0 con 0x7f9d10100ee0 2023-11-16T16:47:19.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.607+0000 7f9d1651a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d101055e0 0x7f9d1019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.607+0000 7f9d1651a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d101055e0 0x7f9d1019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46486/0 (socket says 172.21.15.93:46486) 2023-11-16T16:47:19.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.607+0000 7f9d1651a700 1 -- 172.21.15.93:0/3715599169 learned_addr learned my addr 172.21.15.93:0/3715599169 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:19.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.607+0000 7f9d15518700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d10100ee0 0x7f9d10198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.608+0000 7f9d15d19700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d10100540 0x7f9d101985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.608+0000 7f9d1651a700 1 -- 172.21.15.93:0/3715599169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d10100ee0 msgr2=0x7f9d10198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.608+0000 7f9d1651a700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d10100ee0 0x7f9d10198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.608+0000 7f9d1651a700 1 -- 172.21.15.93:0/3715599169 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d10100540 msgr2=0x7f9d101985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.608+0000 7f9d1651a700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d10100540 0x7f9d101985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.608+0000 7f9d1651a700 1 -- 172.21.15.93:0/3715599169 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d000095f0 con 0x7f9d101055e0 2023-11-16T16:47:19.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.608+0000 7f9d1651a700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d101055e0 0x7f9d1019cf90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9d08007ee0 tx=0x7f9d0800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.608+0000 7f9d06ffd700 1 -- 172.21.15.93:0/3715599169 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d08017070 con 0x7f9d101055e0 2023-11-16T16:47:19.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.608+0000 7f9d17f7d700 1 -- 172.21.15.93:0/3715599169 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d10110360 con 0x7f9d101055e0 2023-11-16T16:47:19.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.608+0000 7f9d06ffd700 1 -- 172.21.15.93:0/3715599169 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d0800f040 con 0x7f9d101055e0 2023-11-16T16:47:19.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.609+0000 7f9d06ffd700 1 -- 172.21.15.93:0/3715599169 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d08005560 con 0x7f9d101055e0 2023-11-16T16:47:19.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.609+0000 7f9d17f7d700 1 -- 172.21.15.93:0/3715599169 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d101101d0 con 0x7f9d101055e0 2023-11-16T16:47:19.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.610+0000 7f9d17f7d700 1 -- 172.21.15.93:0/3715599169 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d101104f0 con 0x7f9d101055e0 2023-11-16T16:47:19.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.611+0000 7f9d06ffd700 1 -- 172.21.15.93:0/3715599169 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9d0800db20 con 0x7f9d101055e0 2023-11-16T16:47:19.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.613+0000 7f9d06ffd700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9cfc0a11e0 0x7f9cfc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.613+0000 7f9d06ffd700 1 -- 172.21.15.93:0/3715599169 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9d080cbee0 con 0x7f9d101055e0 2023-11-16T16:47:19.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.613+0000 7f9d15d19700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9cfc0a11e0 0x7f9cfc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.614+0000 7f9d15d19700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9cfc0a11e0 0x7f9cfc0a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c005950 tx=0x7f9d0c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.617+0000 7f9d06ffd700 1 -- 172.21.15.93:0/3715599169 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9d08095170 con 0x7f9d101055e0 2023-11-16T16:47:19.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.867+0000 7f9d17f7d700 1 -- 172.21.15.93:0/3715599169 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9d10066ea0 con 0x7f9d101055e0 2023-11-16T16:47:19.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.872+0000 7f9d06ffd700 1 -- 172.21.15.93:0/3715599169 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260126 (secure 0 0 0) 0x7f9d08098790 con 0x7f9d101055e0 2023-11-16T16:47:19.877 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:47:19.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.876+0000 7f9d17f7d700 1 -- 172.21.15.93:0/3715599169 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9cfc0a11e0 msgr2=0x7f9cfc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.876+0000 7f9d17f7d700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9cfc0a11e0 0x7f9cfc0a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c005950 tx=0x7f9d0c00b410 comp rx=0 tx=0).stop 2023-11-16T16:47:19.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.876+0000 7f9d17f7d700 1 -- 172.21.15.93:0/3715599169 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d101055e0 msgr2=0x7f9d1019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.876+0000 7f9d17f7d700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d101055e0 0x7f9d1019cf90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9d08007ee0 tx=0x7f9d0800d3b0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.876+0000 7f9d17f7d700 1 -- 172.21.15.93:0/3715599169 shutdown_connections 2023-11-16T16:47:19.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.876+0000 7f9d17f7d700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9cfc0a11e0 0x7f9cfc0a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.876+0000 7f9d17f7d700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d101055e0 0x7f9d1019cf90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.876+0000 7f9d17f7d700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d10100540 0x7f9d101985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.876+0000 7f9d17f7d700 1 --2- 172.21.15.93:0/3715599169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d10100ee0 0x7f9d10198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.876+0000 7f9d17f7d700 1 -- 172.21.15.93:0/3715599169 >> 172.21.15.93:0/3715599169 conn(0x7f9d100fbdc0 msgr2=0x7f9d100fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.876+0000 7f9d17f7d700 1 -- 172.21.15.93:0/3715599169 shutdown_connections 2023-11-16T16:47:19.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:19.876+0000 7f9d17f7d700 1 -- 172.21.15.93:0/3715599169 wait complete. 2023-11-16T16:47:19.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:19 smithi159 ceph-mon[131528]: pgmap v304: 129 pgs: 129 active+clean; 618 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2023-11-16T16:47:20.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:19 smithi125 ceph-mon[131076]: pgmap v304: 129 pgs: 129 active+clean; 618 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2023-11-16T16:47:20.095 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:19 smithi093 ceph-mon[97863]: pgmap v304: 129 pgs: 129 active+clean; 618 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2023-11-16T16:47:20.391 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3283196047},{"type":"v1","addr":"172.21.15.93:6801","nonce":3283196047}]},"active_addr":"172.21.15.93:6801/3283196047","active_change":"2023-11-16T16:39:50.432504+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24115,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.93:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":3600267289}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":3165408888}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":4047308833}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":1038570028}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":2511574730}]}]} 2023-11-16T16:47:20.395 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-11-16T16:47:20.396 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-11-16T16:47:20.396 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd dump --format=json 2023-11-16T16:47:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:20 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3715599169' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-16T16:47:21.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:20 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3715599169' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-16T16:47:21.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:20 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3715599169' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-16T16:47:21.246 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:21.745 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:21 smithi093 ceph-mon[97863]: pgmap v305: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2023-11-16T16:47:21.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:21 smithi159 ceph-mon[131528]: pgmap v305: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2023-11-16T16:47:22.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:21 smithi125 ceph-mon[131076]: pgmap v305: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2023-11-16T16:47:22.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.681+0000 7efccbcf2700 1 -- 172.21.15.93:0/3423311837 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcc4100ee0 msgr2=0x7efcc4104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:22.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.681+0000 7efccbcf2700 1 --2- 172.21.15.93:0/3423311837 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcc4100ee0 0x7efcc4104eb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efcb40099c0 tx=0x7efcb4009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:22.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.681+0000 7efccbcf2700 1 -- 172.21.15.93:0/3423311837 shutdown_connections 2023-11-16T16:47:22.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.681+0000 7efccbcf2700 1 --2- 172.21.15.93:0/3423311837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcc41055e0 0x7efcc410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:22.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.681+0000 7efccbcf2700 1 --2- 172.21.15.93:0/3423311837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efcc4100540 0x7efcc4100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:22.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.681+0000 7efccbcf2700 1 --2- 172.21.15.93:0/3423311837 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcc4100ee0 0x7efcc4104eb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:22.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.681+0000 7efccbcf2700 1 -- 172.21.15.93:0/3423311837 >> 172.21.15.93:0/3423311837 conn(0x7efcc40fbdc0 msgr2=0x7efcc40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:22.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.681+0000 7efccbcf2700 1 -- 172.21.15.93:0/3423311837 shutdown_connections 2023-11-16T16:47:22.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.681+0000 7efccbcf2700 1 -- 172.21.15.93:0/3423311837 wait complete. 2023-11-16T16:47:22.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.682+0000 7efccbcf2700 1 Processor -- start 2023-11-16T16:47:22.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.682+0000 7efccbcf2700 1 -- start start 2023-11-16T16:47:22.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.682+0000 7efccbcf2700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efcc4100540 0x7efcc41963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:22.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efccbcf2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcc4100ee0 0x7efcc4196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:22.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efccbcf2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcc41055e0 0x7efcc419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efccbcf2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc419b390 con 0x7efcc4100ee0 2023-11-16T16:47:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efccbcf2700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc419b500 con 0x7efcc4100540 2023-11-16T16:47:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efccbcf2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc419b6c0 con 0x7efcc41055e0 2023-11-16T16:47:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efcc928d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcc4100ee0 0x7efcc4196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efcc928d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcc4100ee0 0x7efcc4196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46498/0 (socket says 172.21.15.93:46498) 2023-11-16T16:47:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efcc9a8e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efcc4100540 0x7efcc41963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efcc928d700 1 -- 172.21.15.93:0/1128746385 learned_addr learned my addr 172.21.15.93:0/1128746385 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efcca28f700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcc41055e0 0x7efcc419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efcc928d700 1 -- 172.21.15.93:0/1128746385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcc41055e0 msgr2=0x7efcc419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efcc928d700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcc41055e0 0x7efcc419ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efcc928d700 1 -- 172.21.15.93:0/1128746385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efcc4100540 msgr2=0x7efcc41963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efcc928d700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efcc4100540 0x7efcc41963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efcc928d700 1 -- 172.21.15.93:0/1128746385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb40095f0 con 0x7efcc4100ee0 2023-11-16T16:47:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.683+0000 7efcc928d700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcc4100ee0 0x7efcc4196910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efcb40098e0 tx=0x7efcb400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.684+0000 7efcbaffd700 1 -- 172.21.15.93:0/1128746385 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb401d070 con 0x7efcc4100ee0 2023-11-16T16:47:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.684+0000 7efcbaffd700 1 -- 172.21.15.93:0/1128746385 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcb4022470 con 0x7efcc4100ee0 2023-11-16T16:47:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.684+0000 7efccbcf2700 1 -- 172.21.15.93:0/1128746385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc4110360 con 0x7efcc4100ee0 2023-11-16T16:47:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.684+0000 7efcbaffd700 1 -- 172.21.15.93:0/1128746385 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4017650 con 0x7efcc4100ee0 2023-11-16T16:47:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.684+0000 7efccbcf2700 1 -- 172.21.15.93:0/1128746385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc41101d0 con 0x7efcc4100ee0 2023-11-16T16:47:22.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.685+0000 7efccbcf2700 1 -- 172.21.15.93:0/1128746385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc41104f0 con 0x7efcc4100ee0 2023-11-16T16:47:22.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.687+0000 7efcbaffd700 1 -- 172.21.15.93:0/1128746385 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efcb4022850 con 0x7efcc4100ee0 2023-11-16T16:47:22.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.688+0000 7efcbaffd700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efcb00a1260 0x7efcb00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:22.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.689+0000 7efcbaffd700 1 -- 172.21.15.93:0/1128746385 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7efcb40cdb50 con 0x7efcc4100ee0 2023-11-16T16:47:22.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.689+0000 7efcc9a8e700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efcb00a1260 0x7efcb00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:22.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.689+0000 7efcc9a8e700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efcb00a1260 0x7efcb00a3710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009c60 tx=0x7efcc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:22.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.691+0000 7efcbaffd700 1 -- 172.21.15.93:0/1128746385 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efcb40970a0 con 0x7efcc4100ee0 2023-11-16T16:47:22.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.890+0000 7efccbcf2700 1 -- 172.21.15.93:0/1128746385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcc4066ea0 con 0x7efcc4100ee0 2023-11-16T16:47:22.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.891+0000 7efcbaffd700 1 -- 172.21.15.93:0/1128746385 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24468 (secure 0 0 0) 0x7efcb409a6c0 con 0x7efcc4100ee0 2023-11-16T16:47:22.894 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:47:22.894 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":90,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:46:52.315313+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:47:22.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.894+0000 7efccbcf2700 1 -- 172.21.15.93:0/1128746385 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efcb00a1260 msgr2=0x7efcb00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:22.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.894+0000 7efccbcf2700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efcb00a1260 0x7efcb00a3710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009c60 tx=0x7efcc0009260 comp rx=0 tx=0).stop 2023-11-16T16:47:22.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.894+0000 7efccbcf2700 1 -- 172.21.15.93:0/1128746385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcc4100ee0 msgr2=0x7efcc4196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:22.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.894+0000 7efccbcf2700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcc4100ee0 0x7efcc4196910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efcb40098e0 tx=0x7efcb400f690 comp rx=0 tx=0).stop 2023-11-16T16:47:22.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.894+0000 7efccbcf2700 1 -- 172.21.15.93:0/1128746385 shutdown_connections 2023-11-16T16:47:22.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.894+0000 7efccbcf2700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efcb00a1260 0x7efcb00a3710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:22.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.894+0000 7efccbcf2700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcc41055e0 0x7efcc419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:22.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.894+0000 7efccbcf2700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efcc4100540 0x7efcc41963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:22.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.894+0000 7efccbcf2700 1 --2- 172.21.15.93:0/1128746385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcc4100ee0 0x7efcc4196910 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:22.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.894+0000 7efccbcf2700 1 -- 172.21.15.93:0/1128746385 >> 172.21.15.93:0/1128746385 conn(0x7efcc40fbdc0 msgr2=0x7efcc40fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:22.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.894+0000 7efccbcf2700 1 -- 172.21.15.93:0/1128746385 shutdown_connections 2023-11-16T16:47:22.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:22.894+0000 7efccbcf2700 1 -- 172.21.15.93:0/1128746385 wait complete. 2023-11-16T16:47:23.444 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-11-16T16:47:23.444 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd dump --format=json 2023-11-16T16:47:23.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:23 smithi093 ceph-mon[97863]: pgmap v306: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2023-11-16T16:47:23.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:23 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1128746385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:23.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:23 smithi159 ceph-mon[131528]: pgmap v306: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2023-11-16T16:47:23.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:23 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1128746385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:24.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:23 smithi125 ceph-mon[131076]: pgmap v306: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2023-11-16T16:47:24.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:23 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1128746385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:24.291 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:25.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.764+0000 7f864c567700 1 -- 172.21.15.93:0/1437281575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8644100ee0 msgr2=0x7f8644104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:25.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.764+0000 7f864c567700 1 --2- 172.21.15.93:0/1437281575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8644100ee0 0x7f8644104eb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f86340099c0 tx=0x7f8634009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:25.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.764+0000 7f864c567700 1 -- 172.21.15.93:0/1437281575 shutdown_connections 2023-11-16T16:47:25.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.764+0000 7f864c567700 1 --2- 172.21.15.93:0/1437281575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86441055e0 0x7f864410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:25.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.764+0000 7f864c567700 1 --2- 172.21.15.93:0/1437281575 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8644100540 0x7f8644100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:25.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.764+0000 7f864c567700 1 --2- 172.21.15.93:0/1437281575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8644100ee0 0x7f8644104eb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:25.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.764+0000 7f864c567700 1 -- 172.21.15.93:0/1437281575 >> 172.21.15.93:0/1437281575 conn(0x7f86440fbdc0 msgr2=0x7f86440fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:25.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.764+0000 7f864c567700 1 -- 172.21.15.93:0/1437281575 shutdown_connections 2023-11-16T16:47:25.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.764+0000 7f864c567700 1 -- 172.21.15.93:0/1437281575 wait complete. 2023-11-16T16:47:25.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.765+0000 7f864c567700 1 Processor -- start 2023-11-16T16:47:25.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.765+0000 7f864c567700 1 -- start start 2023-11-16T16:47:25.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864c567700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8644100540 0x7f86441985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:25.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864c567700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8644100ee0 0x7f8644198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:25.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864c567700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86441055e0 0x7f864419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:25.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864c567700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864419d580 con 0x7f86441055e0 2023-11-16T16:47:25.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864c567700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864419d6f0 con 0x7f8644100540 2023-11-16T16:47:25.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864c567700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864419d8b0 con 0x7f8644100ee0 2023-11-16T16:47:25.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864ab04700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86441055e0 0x7f864419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:25.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864ab04700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86441055e0 0x7f864419cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46512/0 (socket says 172.21.15.93:46512) 2023-11-16T16:47:25.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864ab04700 1 -- 172.21.15.93:0/2735083625 learned_addr learned my addr 172.21.15.93:0/2735083625 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:25.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864a303700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8644100540 0x7f86441985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:25.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f8649b02700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8644100ee0 0x7f8644198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:25.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864ab04700 1 -- 172.21.15.93:0/2735083625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8644100ee0 msgr2=0x7f8644198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:25.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864ab04700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8644100ee0 0x7f8644198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:25.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864ab04700 1 -- 172.21.15.93:0/2735083625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8644100540 msgr2=0x7f86441985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:25.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864ab04700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8644100540 0x7f86441985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:25.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.766+0000 7f864ab04700 1 -- 172.21.15.93:0/2735083625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86340095f0 con 0x7f86441055e0 2023-11-16T16:47:25.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.767+0000 7f864ab04700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86441055e0 0x7f864419cf70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f863c00ef30 tx=0x7f863c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:25.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.767+0000 7f863b7fe700 1 -- 172.21.15.93:0/2735083625 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c018070 con 0x7f86441055e0 2023-11-16T16:47:25.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.767+0000 7f863b7fe700 1 -- 172.21.15.93:0/2735083625 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f863c00f040 con 0x7f86441055e0 2023-11-16T16:47:25.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.767+0000 7f864c567700 1 -- 172.21.15.93:0/2735083625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8644110360 con 0x7f86441055e0 2023-11-16T16:47:25.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.767+0000 7f863b7fe700 1 -- 172.21.15.93:0/2735083625 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c00cd00 con 0x7f86441055e0 2023-11-16T16:47:25.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.767+0000 7f864c567700 1 -- 172.21.15.93:0/2735083625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86441101d0 con 0x7f86441055e0 2023-11-16T16:47:25.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.768+0000 7f864c567700 1 -- 172.21.15.93:0/2735083625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86441104f0 con 0x7f86441055e0 2023-11-16T16:47:25.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.770+0000 7f863b7fe700 1 -- 172.21.15.93:0/2735083625 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f863c003a00 con 0x7f86441055e0 2023-11-16T16:47:25.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.771+0000 7f863b7fe700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f86300a11e0 0x7f86300a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:25.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.772+0000 7f863b7fe700 1 -- 172.21.15.93:0/2735083625 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f863c0cc870 con 0x7f86441055e0 2023-11-16T16:47:25.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.772+0000 7f864a303700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f86300a11e0 0x7f86300a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:25.774 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.772+0000 7f864a303700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f86300a11e0 0x7f86300a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8640009c60 tx=0x7f8640009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:25.777 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.776+0000 7f863b7fe700 1 -- 172.21.15.93:0/2735083625 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f863c095b90 con 0x7f86441055e0 2023-11-16T16:47:25.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:25 smithi093 ceph-mon[97863]: pgmap v307: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2023-11-16T16:47:25.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.977+0000 7f864c567700 1 -- 172.21.15.93:0/2735083625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8644066ea0 con 0x7f86441055e0 2023-11-16T16:47:25.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.979+0000 7f863b7fe700 1 -- 172.21.15.93:0/2735083625 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24468 (secure 0 0 0) 0x7f863c0991b0 con 0x7f86441055e0 2023-11-16T16:47:25.981 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:47:25.981 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":90,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:46:52.315313+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:47:25.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.981+0000 7f864c567700 1 -- 172.21.15.93:0/2735083625 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f86300a11e0 msgr2=0x7f86300a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:25.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.981+0000 7f864c567700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f86300a11e0 0x7f86300a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8640009c60 tx=0x7f8640009260 comp rx=0 tx=0).stop 2023-11-16T16:47:25.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.981+0000 7f864c567700 1 -- 172.21.15.93:0/2735083625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86441055e0 msgr2=0x7f864419cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:25.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.981+0000 7f864c567700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86441055e0 0x7f864419cf70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f863c00ef30 tx=0x7f863c00c3a0 comp rx=0 tx=0).stop 2023-11-16T16:47:25.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.981+0000 7f864c567700 1 -- 172.21.15.93:0/2735083625 shutdown_connections 2023-11-16T16:47:25.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.981+0000 7f864c567700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f86300a11e0 0x7f86300a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:25.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.981+0000 7f864c567700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86441055e0 0x7f864419cf70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:25.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.981+0000 7f864c567700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8644100540 0x7f86441985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:25.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.981+0000 7f864c567700 1 --2- 172.21.15.93:0/2735083625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8644100ee0 0x7f8644198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:25.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.981+0000 7f864c567700 1 -- 172.21.15.93:0/2735083625 >> 172.21.15.93:0/2735083625 conn(0x7f86440fbdc0 msgr2=0x7f86440fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:25.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.981+0000 7f864c567700 1 -- 172.21.15.93:0/2735083625 shutdown_connections 2023-11-16T16:47:25.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:25.981+0000 7f864c567700 1 -- 172.21.15.93:0/2735083625 wait complete. 2023-11-16T16:47:25.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:25 smithi159 ceph-mon[131528]: pgmap v307: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2023-11-16T16:47:26.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:25 smithi125 ceph-mon[131076]: pgmap v307: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2023-11-16T16:47:26.484 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-11-16T16:47:26.485 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-11-16T16:47:26.485 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-11-16T16:47:26.486 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-11-16T16:47:26.486 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-11-16T16:47:26.487 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-11-16T16:47:26.487 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-11-16T16:47:26.488 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-11-16T16:47:26.488 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-11-16T16:47:26.489 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-11-16T16:47:26.489 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-11-16T16:47:26.489 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-11-16T16:47:26.732 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:26 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2735083625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:26.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:26 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2735083625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:27.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:26 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2735083625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:47:27.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:27 smithi159 ceph-mon[131528]: pgmap v308: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-16T16:47:28.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:27 smithi125 ceph-mon[131076]: pgmap v308: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-16T16:47:28.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:27 smithi093 ceph-mon[97863]: pgmap v308: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-16T16:47:28.231 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:28.293 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:28.613 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:28.851 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:29.037 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:29.469 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:29.506 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:29.772 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:29.918 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:29.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:29 smithi159 ceph-mon[131528]: pgmap v309: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-11-16T16:47:30.054 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:29 smithi093 ceph-mon[97863]: pgmap v309: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-11-16T16:47:30.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:29 smithi125 ceph-mon[131076]: pgmap v309: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-11-16T16:47:30.662 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:30.883 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:30.888 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:31.380 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:31 smithi093 ceph-mon[97863]: pgmap v310: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-16T16:47:31.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:31 smithi125 ceph-mon[131076]: pgmap v310: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-16T16:47:31.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:31 smithi159 ceph-mon[131528]: pgmap v310: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-16T16:47:33.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.641+0000 7f2871830700 1 -- 172.21.15.93:0/3108822701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f286c100540 msgr2=0x7f286c100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:33.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.641+0000 7f2871830700 1 --2- 172.21.15.93:0/3108822701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f286c100540 0x7f286c100910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f285c0099c0 tx=0x7f285c009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:33.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.642+0000 7f2871830700 1 -- 172.21.15.93:0/3108822701 shutdown_connections 2023-11-16T16:47:33.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.642+0000 7f2871830700 1 --2- 172.21.15.93:0/3108822701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f286c100540 0x7f286c100910 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:33.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.642+0000 7f2871830700 1 --2- 172.21.15.93:0/3108822701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f286c1055e0 0x7f286c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:33.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.642+0000 7f2871830700 1 --2- 172.21.15.93:0/3108822701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f286c100ee0 0x7f286c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:33.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.642+0000 7f2871830700 1 -- 172.21.15.93:0/3108822701 >> 172.21.15.93:0/3108822701 conn(0x7f286c0fbdc0 msgr2=0x7f286c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:33.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.642+0000 7f2871830700 1 -- 172.21.15.93:0/3108822701 shutdown_connections 2023-11-16T16:47:33.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.642+0000 7f2871830700 1 -- 172.21.15.93:0/3108822701 wait complete. 2023-11-16T16:47:33.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.643+0000 7f2871830700 1 Processor -- start 2023-11-16T16:47:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.643+0000 7f2871830700 1 -- start start 2023-11-16T16:47:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.644+0000 7f2871830700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f286c100540 0x7f286c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.644+0000 7f2871830700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f286c100ee0 0x7f286c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.644+0000 7f2871830700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f286c1055e0 0x7f286c0738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.644+0000 7f2871830700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c1a4890 con 0x7f286c100ee0 2023-11-16T16:47:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.644+0000 7f2871830700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c1a4a00 con 0x7f286c1055e0 2023-11-16T16:47:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.644+0000 7f2871830700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c1a4bc0 con 0x7f286c100540 2023-11-16T16:47:33.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.644+0000 7f286a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f286c100ee0 0x7f286c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:33.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.644+0000 7f286a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f286c100ee0 0x7f286c073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42620/0 (socket says 172.21.15.93:42620) 2023-11-16T16:47:33.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.644+0000 7f286a7fc700 1 -- 172.21.15.93:0/124261856 learned_addr learned my addr 172.21.15.93:0/124261856 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:33.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.644+0000 7f286affd700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f286c100540 0x7f286c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:33.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.645+0000 7f286b7fe700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f286c1055e0 0x7f286c0738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:33.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.645+0000 7f286a7fc700 1 -- 172.21.15.93:0/124261856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f286c100540 msgr2=0x7f286c072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:33.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.645+0000 7f286a7fc700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f286c100540 0x7f286c072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:33.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.645+0000 7f286a7fc700 1 -- 172.21.15.93:0/124261856 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f286c1055e0 msgr2=0x7f286c0738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:33.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.645+0000 7f286a7fc700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f286c1055e0 0x7f286c0738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:33.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.645+0000 7f286a7fc700 1 -- 172.21.15.93:0/124261856 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c0095f0 con 0x7f286c100ee0 2023-11-16T16:47:33.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.645+0000 7f286a7fc700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f286c100ee0 0x7f286c073330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f286000e970 tx=0x7f286000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.645+0000 7f287082e700 1 -- 172.21.15.93:0/124261856 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860009790 con 0x7f286c100ee0 2023-11-16T16:47:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.645+0000 7f2871830700 1 -- 172.21.15.93:0/124261856 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286c110320 con 0x7f286c100ee0 2023-11-16T16:47:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.645+0000 7f287082e700 1 -- 172.21.15.93:0/124261856 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286000c7e0 con 0x7f286c100ee0 2023-11-16T16:47:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.645+0000 7f287082e700 1 -- 172.21.15.93:0/124261856 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600054e0 con 0x7f286c100ee0 2023-11-16T16:47:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.646+0000 7f2871830700 1 -- 172.21.15.93:0/124261856 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286c110190 con 0x7f286c100ee0 2023-11-16T16:47:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.646+0000 7f2871830700 1 -- 172.21.15.93:0/124261856 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f286c104820 con 0x7f286c100ee0 2023-11-16T16:47:33.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.648+0000 7f287082e700 1 -- 172.21.15.93:0/124261856 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f286000c950 con 0x7f286c100ee0 2023-11-16T16:47:33.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.650+0000 7f287082e700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f28540a11e0 0x7f28540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:33.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.650+0000 7f287082e700 1 -- 172.21.15.93:0/124261856 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f2860014070 con 0x7f286c100ee0 2023-11-16T16:47:33.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.650+0000 7f286affd700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f28540a11e0 0x7f28540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:33.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.650+0000 7f287082e700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600] conn(0x7f28540ac170 0x7f28540ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:33.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.650+0000 7f287082e700 1 -- 172.21.15.93:0/124261856 --> [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28540aecd0 con 0x7f28540ac170 2023-11-16T16:47:33.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.650+0000 7f287082e700 1 -- 172.21.15.93:0/124261856 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28600cf480 con 0x7f286c100ee0 2023-11-16T16:47:33.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.651+0000 7f286b7fe700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600] conn(0x7f28540ac170 0x7f28540ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:33.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.651+0000 7f286affd700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f28540a11e0 0x7f28540a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f285c009fd0 tx=0x7f285c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:33.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.651+0000 7f286b7fe700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600] conn(0x7f28540ac170 0x7f28540ae600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:33.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.652+0000 7f287082e700 1 -- 172.21.15.93:0/124261856 <== osd.3 v2:172.21.15.93:6810/2350765600 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f28540aecd0 con 0x7f28540ac170 2023-11-16T16:47:33.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.672+0000 7f2871830700 1 -- 172.21.15.93:0/124261856 --> [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f286c1104b0 con 0x7f28540ac170 2023-11-16T16:47:33.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.672+0000 7f287082e700 1 -- 172.21.15.93:0/124261856 <== osd.3 v2:172.21.15.93:6810/2350765600 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f286c1104b0 con 0x7f28540ac170 2023-11-16T16:47:33.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.673+0000 7f2871830700 1 -- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600] conn(0x7f28540ac170 msgr2=0x7f28540ae600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:33.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.673+0000 7f2871830700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600] conn(0x7f28540ac170 0x7f28540ae600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:33.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.673+0000 7f2871830700 1 -- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f28540a11e0 msgr2=0x7f28540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:33.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.673+0000 7f2871830700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f28540a11e0 0x7f28540a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f285c009fd0 tx=0x7f285c005fb0 comp rx=0 tx=0).stop 2023-11-16T16:47:33.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.673+0000 7f2871830700 1 -- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f286c100ee0 msgr2=0x7f286c073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:33.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.673+0000 7f2871830700 1 --2- 172.21.15.93:0/124261856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f286c100ee0 0x7f286c073330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f286000e970 tx=0x7f286000ec80 comp rx=0 tx=0).stop 2023-11-16T16:47:33.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.673+0000 7f286b7fe700 1 -- 172.21.15.93:0/124261856 reap_dead start 2023-11-16T16:47:33.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.673+0000 7f2871830700 1 -- 172.21.15.93:0/124261856 shutdown_connections 2023-11-16T16:47:33.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.673+0000 7f2871830700 1 -- 172.21.15.93:0/124261856 >> 172.21.15.93:0/124261856 conn(0x7f286c0fbdc0 msgr2=0x7f286c103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:33.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.673+0000 7f2871830700 1 -- 172.21.15.93:0/124261856 shutdown_connections 2023-11-16T16:47:33.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:33.673+0000 7f2871830700 1 -- 172.21.15.93:0/124261856 wait complete. 2023-11-16T16:47:33.837 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:33 smithi093 ceph-mon[97863]: pgmap v311: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-16T16:47:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:33 smithi159 ceph-mon[131528]: pgmap v311: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-16T16:47:34.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:33 smithi125 ceph-mon[131076]: pgmap v311: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-16T16:47:35.124 INFO:teuthology.orchestra.run.smithi093.stdout:107374182462 2023-11-16T16:47:35.125 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-11-16T16:47:35.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:35 smithi093 ceph-mon[97863]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-11-16T16:47:35.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.779+0000 7fdb250de700 1 -- 172.21.15.93:0/384407614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb20100ee0 msgr2=0x7fdb20104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:35.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.779+0000 7fdb250de700 1 --2- 172.21.15.93:0/384407614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb20100ee0 0x7fdb20104eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdb140099c0 tx=0x7fdb14009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:35.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.779+0000 7fdb250de700 1 -- 172.21.15.93:0/384407614 shutdown_connections 2023-11-16T16:47:35.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.779+0000 7fdb250de700 1 --2- 172.21.15.93:0/384407614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb201055e0 0x7fdb2010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:35.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.779+0000 7fdb250de700 1 --2- 172.21.15.93:0/384407614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb20100540 0x7fdb20100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:35.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.779+0000 7fdb250de700 1 --2- 172.21.15.93:0/384407614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb20100ee0 0x7fdb20104eb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:35.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.779+0000 7fdb250de700 1 -- 172.21.15.93:0/384407614 >> 172.21.15.93:0/384407614 conn(0x7fdb200fbdc0 msgr2=0x7fdb200fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:35.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.779+0000 7fdb250de700 1 -- 172.21.15.93:0/384407614 shutdown_connections 2023-11-16T16:47:35.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.779+0000 7fdb250de700 1 -- 172.21.15.93:0/384407614 wait complete. 2023-11-16T16:47:35.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.780+0000 7fdb250de700 1 Processor -- start 2023-11-16T16:47:35.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.780+0000 7fdb250de700 1 -- start start 2023-11-16T16:47:35.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb250de700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb20100540 0x7fdb20072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:35.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb250de700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb20100ee0 0x7fdb20073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:35.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb250de700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb201055e0 0x7fdb200738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:35.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb250de700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb201a4810 con 0x7fdb201055e0 2023-11-16T16:47:35.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb250de700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb201a4980 con 0x7fdb20100540 2023-11-16T16:47:35.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb250de700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb201a4b40 con 0x7fdb20100ee0 2023-11-16T16:47:35.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb1e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb20100ee0 0x7fdb20073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:35.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb1ed9d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb20100540 0x7fdb20072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:35.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb1e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb20100ee0 0x7fdb20073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:48426/0 (socket says 172.21.15.93:48426) 2023-11-16T16:47:35.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb1e59c700 1 -- 172.21.15.93:0/3489681159 learned_addr learned my addr 172.21.15.93:0/3489681159 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:35.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb1f59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb201055e0 0x7fdb200738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:35.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb1ed9d700 1 -- 172.21.15.93:0/3489681159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb20100ee0 msgr2=0x7fdb20073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:35.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb1ed9d700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb20100ee0 0x7fdb20073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:35.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb1ed9d700 1 -- 172.21.15.93:0/3489681159 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb201055e0 msgr2=0x7fdb200738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:35.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb1ed9d700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb201055e0 0x7fdb200738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:35.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb1ed9d700 1 -- 172.21.15.93:0/3489681159 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb18009c40 con 0x7fdb20100540 2023-11-16T16:47:35.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb1e59c700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb20100ee0 0x7fdb20073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:35.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.781+0000 7fdb1f59e700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb201055e0 0x7fdb200738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:35.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.782+0000 7fdb1ed9d700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb20100540 0x7fdb20072df0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000e9f0 tx=0x7fdb1000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:35.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.782+0000 7fdb07fff700 1 -- 172.21.15.93:0/3489681159 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1000c9c0 con 0x7fdb20100540 2023-11-16T16:47:35.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.782+0000 7fdb07fff700 1 -- 172.21.15.93:0/3489681159 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb10010430 con 0x7fdb20100540 2023-11-16T16:47:35.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.782+0000 7fdb250de700 1 -- 172.21.15.93:0/3489681159 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb140095f0 con 0x7fdb20100540 2023-11-16T16:47:35.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.782+0000 7fdb07fff700 1 -- 172.21.15.93:0/3489681159 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10018680 con 0x7fdb20100540 2023-11-16T16:47:35.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.782+0000 7fdb250de700 1 -- 172.21.15.93:0/3489681159 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb20110310 con 0x7fdb20100540 2023-11-16T16:47:35.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.783+0000 7fdb250de700 1 -- 172.21.15.93:0/3489681159 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdb200fd200 con 0x7fdb20100540 2023-11-16T16:47:35.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.785+0000 7fdb07fff700 1 -- 172.21.15.93:0/3489681159 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdb10010860 con 0x7fdb20100540 2023-11-16T16:47:35.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.786+0000 7fdb07fff700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdb080a11e0 0x7fdb080a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:35.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.786+0000 7fdb07fff700 1 -- 172.21.15.93:0/3489681159 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fdb10014070 con 0x7fdb20100540 2023-11-16T16:47:35.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.786+0000 7fdb1e59c700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdb080a11e0 0x7fdb080a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:35.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.787+0000 7fdb07fff700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909] conn(0x7fdb080ac170 0x7fdb080ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:35.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.787+0000 7fdb07fff700 1 -- 172.21.15.93:0/3489681159 --> [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb080aecd0 con 0x7fdb080ac170 2023-11-16T16:47:35.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.787+0000 7fdb07fff700 1 -- 172.21.15.93:0/3489681159 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb100cd780 con 0x7fdb20100540 2023-11-16T16:47:35.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.787+0000 7fdb1e59c700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdb080a11e0 0x7fdb080a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb200745e0 tx=0x7fdb1401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:35.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.787+0000 7fdb1f59e700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909] conn(0x7fdb080ac170 0x7fdb080ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:35.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.788+0000 7fdb1f59e700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909] conn(0x7fdb080ac170 0x7fdb080ae600 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:35.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.788+0000 7fdb07fff700 1 -- 172.21.15.93:0/3489681159 <== osd.5 v2:172.21.15.159:6808/1380209909 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdb080aecd0 con 0x7fdb080ac170 2023-11-16T16:47:35.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.805+0000 7fdb250de700 1 -- 172.21.15.93:0/3489681159 --> [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdb20110180 con 0x7fdb080ac170 2023-11-16T16:47:35.807 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.806+0000 7fdb07fff700 1 -- 172.21.15.93:0/3489681159 <== osd.5 v2:172.21.15.159:6808/1380209909 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdb20110180 con 0x7fdb080ac170 2023-11-16T16:47:35.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.806+0000 7fdb250de700 1 -- 172.21.15.93:0/3489681159 >> [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909] conn(0x7fdb080ac170 msgr2=0x7fdb080ae600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:35.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.806+0000 7fdb250de700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909] conn(0x7fdb080ac170 0x7fdb080ae600 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:35.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.806+0000 7fdb250de700 1 -- 172.21.15.93:0/3489681159 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdb080a11e0 msgr2=0x7fdb080a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:35.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.806+0000 7fdb250de700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdb080a11e0 0x7fdb080a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb200745e0 tx=0x7fdb1401a040 comp rx=0 tx=0).stop 2023-11-16T16:47:35.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.806+0000 7fdb250de700 1 -- 172.21.15.93:0/3489681159 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb20100540 msgr2=0x7fdb20072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:35.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.806+0000 7fdb250de700 1 --2- 172.21.15.93:0/3489681159 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb20100540 0x7fdb20072df0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000e9f0 tx=0x7fdb1000ed00 comp rx=0 tx=0).stop 2023-11-16T16:47:35.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.806+0000 7fdb1f59e700 1 -- 172.21.15.93:0/3489681159 reap_dead start 2023-11-16T16:47:35.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.806+0000 7fdb250de700 1 -- 172.21.15.93:0/3489681159 shutdown_connections 2023-11-16T16:47:35.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.806+0000 7fdb250de700 1 -- 172.21.15.93:0/3489681159 >> 172.21.15.93:0/3489681159 conn(0x7fdb200fbdc0 msgr2=0x7fdb200fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:35.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.806+0000 7fdb250de700 1 -- 172.21.15.93:0/3489681159 shutdown_connections 2023-11-16T16:47:35.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.806+0000 7fdb250de700 1 -- 172.21.15.93:0/3489681159 wait complete. 2023-11-16T16:47:35.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:35 smithi159 ceph-mon[131528]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-11-16T16:47:36.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:35 smithi125 ceph-mon[131076]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-11-16T16:47:36.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.671+0000 7f30514c1700 1 -- 172.21.15.93:0/805600615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f304c1055e0 msgr2=0x7f304c10d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:36.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.671+0000 7f30514c1700 1 --2- 172.21.15.93:0/805600615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f304c1055e0 0x7f304c10d160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f304400b0d0 tx=0x7f304400b3e0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.671+0000 7f30514c1700 1 -- 172.21.15.93:0/805600615 shutdown_connections 2023-11-16T16:47:36.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.671+0000 7f30514c1700 1 --2- 172.21.15.93:0/805600615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f304c1055e0 0x7f304c10d160 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.671+0000 7f30514c1700 1 --2- 172.21.15.93:0/805600615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f304c100540 0x7f304c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.671+0000 7f30514c1700 1 --2- 172.21.15.93:0/805600615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f304c100ee0 0x7f304c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.671+0000 7f30514c1700 1 -- 172.21.15.93:0/805600615 >> 172.21.15.93:0/805600615 conn(0x7f304c0fbdc0 msgr2=0x7f304c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:36.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.671+0000 7f30514c1700 1 -- 172.21.15.93:0/805600615 shutdown_connections 2023-11-16T16:47:36.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.671+0000 7f30514c1700 1 -- 172.21.15.93:0/805600615 wait complete. 2023-11-16T16:47:36.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.672+0000 7f30514c1700 1 Processor -- start 2023-11-16T16:47:36.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.672+0000 7f30514c1700 1 -- start start 2023-11-16T16:47:36.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.673+0000 7f30514c1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f304c100540 0x7f304c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:36.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.673+0000 7f30514c1700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f304c100ee0 0x7f304c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:36.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.673+0000 7f30514c1700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f304c1055e0 0x7f304c073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:36.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.673+0000 7f30514c1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1a8e60 con 0x7f304c100540 2023-11-16T16:47:36.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.673+0000 7f30514c1700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1a8fa0 con 0x7f304c100ee0 2023-11-16T16:47:36.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.673+0000 7f30514c1700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1a90e0 con 0x7f304c1055e0 2023-11-16T16:47:36.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.673+0000 7f304affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f304c100540 0x7f304c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:36.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.673+0000 7f304affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f304c100540 0x7f304c072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42628/0 (socket says 172.21.15.93:42628) 2023-11-16T16:47:36.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.673+0000 7f304affd700 1 -- 172.21.15.93:0/933559959 learned_addr learned my addr 172.21.15.93:0/933559959 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:36.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.673+0000 7f304b7fe700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f304c1055e0 0x7f304c073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:36.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.673+0000 7f304a7fc700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f304c100ee0 0x7f304c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:36.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.674+0000 7f304affd700 1 -- 172.21.15.93:0/933559959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f304c1055e0 msgr2=0x7f304c073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:36.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.674+0000 7f304affd700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f304c1055e0 0x7f304c073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.674+0000 7f304affd700 1 -- 172.21.15.93:0/933559959 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f304c100ee0 msgr2=0x7f304c073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:36.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.674+0000 7f304affd700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f304c100ee0 0x7f304c073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.674+0000 7f304affd700 1 -- 172.21.15.93:0/933559959 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3044009d00 con 0x7f304c100540 2023-11-16T16:47:36.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.674+0000 7f304a7fc700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f304c100ee0 0x7f304c073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:36.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.674+0000 7f304affd700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f304c100540 0x7f304c072df0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f303c00e9d0 tx=0x7f303c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:36.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.674+0000 7f3033fff700 1 -- 172.21.15.93:0/933559959 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c009790 con 0x7f304c100540 2023-11-16T16:47:36.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.674+0000 7f30514c1700 1 -- 172.21.15.93:0/933559959 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c110370 con 0x7f304c100540 2023-11-16T16:47:36.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.674+0000 7f3033fff700 1 -- 172.21.15.93:0/933559959 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f303c00cbd0 con 0x7f304c100540 2023-11-16T16:47:36.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.674+0000 7f3033fff700 1 -- 172.21.15.93:0/933559959 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c018680 con 0x7f304c100540 2023-11-16T16:47:36.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.674+0000 7f30514c1700 1 -- 172.21.15.93:0/933559959 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c1101e0 con 0x7f304c100540 2023-11-16T16:47:36.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.675+0000 7f30514c1700 1 -- 172.21.15.93:0/933559959 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f304c0fd110 con 0x7f304c100540 2023-11-16T16:47:36.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.678+0000 7f3033fff700 1 -- 172.21.15.93:0/933559959 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f303c010430 con 0x7f304c100540 2023-11-16T16:47:36.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.680+0000 7f3033fff700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f30340a11e0 0x7f30340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:36.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.680+0000 7f304a7fc700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f30340a11e0 0x7f30340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:36.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.680+0000 7f3033fff700 1 -- 172.21.15.93:0/933559959 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f303c014070 con 0x7f304c100540 2023-11-16T16:47:36.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.681+0000 7f3033fff700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748] conn(0x7f30340ac170 0x7f30340ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:36.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.681+0000 7f3033fff700 1 -- 172.21.15.93:0/933559959 --> [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30340aecd0 con 0x7f30340ac170 2023-11-16T16:47:36.264 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.681+0000 7f3033fff700 1 -- 172.21.15.93:0/933559959 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f303c0cd740 con 0x7f304c100540 2023-11-16T16:47:36.264 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.681+0000 7f304a7fc700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f30340a11e0 0x7f30340a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f304c074690 tx=0x7f3040009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:36.264 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.681+0000 7f304b7fe700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748] conn(0x7f30340ac170 0x7f30340ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:36.264 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.682+0000 7f304b7fe700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748] conn(0x7f30340ac170 0x7f30340ae600 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:36.264 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.682+0000 7f3033fff700 1 -- 172.21.15.93:0/933559959 <== osd.0 v2:172.21.15.93:6802/1213182748 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f30340aecd0 con 0x7f30340ac170 2023-11-16T16:47:36.264 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.706+0000 7f30514c1700 1 -- 172.21.15.93:0/933559959 --> [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f304c110500 con 0x7f30340ac170 2023-11-16T16:47:36.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.707+0000 7f3033fff700 1 -- 172.21.15.93:0/933559959 <== osd.0 v2:172.21.15.93:6802/1213182748 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f304c110500 con 0x7f30340ac170 2023-11-16T16:47:36.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.707+0000 7f30514c1700 1 -- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748] conn(0x7f30340ac170 msgr2=0x7f30340ae600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:36.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.707+0000 7f30514c1700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748] conn(0x7f30340ac170 0x7f30340ae600 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.708+0000 7f30514c1700 1 -- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f30340a11e0 msgr2=0x7f30340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:36.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.708+0000 7f30514c1700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f30340a11e0 0x7f30340a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f304c074690 tx=0x7f3040009260 comp rx=0 tx=0).stop 2023-11-16T16:47:36.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.708+0000 7f30514c1700 1 -- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f304c100540 msgr2=0x7f304c072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:36.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.708+0000 7f30514c1700 1 --2- 172.21.15.93:0/933559959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f304c100540 0x7f304c072df0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f303c00e9d0 tx=0x7f303c00ed90 comp rx=0 tx=0).stop 2023-11-16T16:47:36.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.708+0000 7f304b7fe700 1 -- 172.21.15.93:0/933559959 reap_dead start 2023-11-16T16:47:36.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.708+0000 7f30514c1700 1 -- 172.21.15.93:0/933559959 shutdown_connections 2023-11-16T16:47:36.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.708+0000 7f30514c1700 1 -- 172.21.15.93:0/933559959 >> 172.21.15.93:0/933559959 conn(0x7f304c0fbdc0 msgr2=0x7f304c0fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:36.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.708+0000 7f30514c1700 1 -- 172.21.15.93:0/933559959 shutdown_connections 2023-11-16T16:47:36.266 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:35.708+0000 7f30514c1700 1 -- 172.21.15.93:0/933559959 wait complete. 2023-11-16T16:47:36.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.661+0000 7f03dbd89700 1 -- 172.21.15.93:0/89556978 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03d4100ee0 msgr2=0x7f03d4104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:36.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.661+0000 7f03dbd89700 1 --2- 172.21.15.93:0/89556978 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03d4100ee0 0x7f03d4104eb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f03c40099c0 tx=0x7f03c4009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.661+0000 7f03dbd89700 1 -- 172.21.15.93:0/89556978 shutdown_connections 2023-11-16T16:47:36.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.661+0000 7f03dbd89700 1 --2- 172.21.15.93:0/89556978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03d41055e0 0x7f03d410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.661+0000 7f03dbd89700 1 --2- 172.21.15.93:0/89556978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03d4100540 0x7f03d4100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.661+0000 7f03dbd89700 1 --2- 172.21.15.93:0/89556978 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03d4100ee0 0x7f03d4104eb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.661+0000 7f03dbd89700 1 -- 172.21.15.93:0/89556978 >> 172.21.15.93:0/89556978 conn(0x7f03d40fbdc0 msgr2=0x7f03d40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:36.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.661+0000 7f03dbd89700 1 -- 172.21.15.93:0/89556978 shutdown_connections 2023-11-16T16:47:36.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.661+0000 7f03dbd89700 1 -- 172.21.15.93:0/89556978 wait complete. 2023-11-16T16:47:36.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.662+0000 7f03dbd89700 1 Processor -- start 2023-11-16T16:47:36.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.662+0000 7f03dbd89700 1 -- start start 2023-11-16T16:47:36.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.663+0000 7f03dbd89700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03d4100540 0x7f03d41963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:36.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.663+0000 7f03dbd89700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03d4100ee0 0x7f03d4196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:36.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.663+0000 7f03dbd89700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03d41055e0 0x7f03d419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:36.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.663+0000 7f03dbd89700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d419b3b0 con 0x7f03d4100540 2023-11-16T16:47:36.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.663+0000 7f03dbd89700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d419b520 con 0x7f03d41055e0 2023-11-16T16:47:36.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.663+0000 7f03dbd89700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d419b6e0 con 0x7f03d4100ee0 2023-11-16T16:47:36.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.663+0000 7f03d9b25700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03d4100540 0x7f03d41963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:36.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.663+0000 7f03d9b25700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03d4100540 0x7f03d41963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42676/0 (socket says 172.21.15.93:42676) 2023-11-16T16:47:36.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.663+0000 7f03d9b25700 1 -- 172.21.15.93:0/3040956264 learned_addr learned my addr 172.21.15.93:0/3040956264 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:36.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.663+0000 7f03d9324700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03d4100ee0 0x7f03d4196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:36.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.663+0000 7f03da326700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03d41055e0 0x7f03d419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:36.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.664+0000 7f03d9324700 1 -- 172.21.15.93:0/3040956264 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03d41055e0 msgr2=0x7f03d419ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:36.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.664+0000 7f03d9324700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03d41055e0 0x7f03d419ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.664+0000 7f03d9324700 1 -- 172.21.15.93:0/3040956264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03d4100540 msgr2=0x7f03d41963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:36.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.664+0000 7f03d9324700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03d4100540 0x7f03d41963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.664+0000 7f03d9324700 1 -- 172.21.15.93:0/3040956264 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c40095f0 con 0x7f03d4100ee0 2023-11-16T16:47:36.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.664+0000 7f03d9b25700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03d4100540 0x7f03d41963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:36.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.664+0000 7f03d9324700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03d4100ee0 0x7f03d4196930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03c40058e0 tx=0x7f03c400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:36.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.664+0000 7f03caffd700 1 -- 172.21.15.93:0/3040956264 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c401d070 con 0x7f03d4100ee0 2023-11-16T16:47:36.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.664+0000 7f03dbd89700 1 -- 172.21.15.93:0/3040956264 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d4110380 con 0x7f03d4100ee0 2023-11-16T16:47:36.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.664+0000 7f03caffd700 1 -- 172.21.15.93:0/3040956264 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03c4005020 con 0x7f03d4100ee0 2023-11-16T16:47:36.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.665+0000 7f03caffd700 1 -- 172.21.15.93:0/3040956264 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4004970 con 0x7f03d4100ee0 2023-11-16T16:47:36.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.665+0000 7f03dbd89700 1 -- 172.21.15.93:0/3040956264 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d41101f0 con 0x7f03d4100ee0 2023-11-16T16:47:36.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.667+0000 7f03dbd89700 1 -- 172.21.15.93:0/3040956264 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f03d40fd1e0 con 0x7f03d4100ee0 2023-11-16T16:47:36.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.668+0000 7f03caffd700 1 -- 172.21.15.93:0/3040956264 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f03c4005190 con 0x7f03d4100ee0 2023-11-16T16:47:36.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.670+0000 7f03caffd700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f03c00a11e0 0x7f03c00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:36.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.670+0000 7f03caffd700 1 -- 172.21.15.93:0/3040956264 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f03c40cd910 con 0x7f03d4100ee0 2023-11-16T16:47:36.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.671+0000 7f03caffd700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100] conn(0x7f03c00ac170 0x7f03c00ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:36.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.671+0000 7f03caffd700 1 -- 172.21.15.93:0/3040956264 --> [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03c00aecd0 con 0x7f03c00ac170 2023-11-16T16:47:36.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.671+0000 7f03caffd700 1 -- 172.21.15.93:0/3040956264 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03c40d22a0 con 0x7f03d4100ee0 2023-11-16T16:47:36.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.671+0000 7f03d9b25700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f03c00a11e0 0x7f03c00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:36.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.672+0000 7f03da326700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100] conn(0x7f03c00ac170 0x7f03c00ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:36.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.672+0000 7f03d9b25700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f03c00a11e0 0x7f03c00a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f03d0011550 tx=0x7f03d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:36.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.673+0000 7f03da326700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100] conn(0x7f03c00ac170 0x7f03c00ae600 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:36.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.674+0000 7f03caffd700 1 -- 172.21.15.93:0/3040956264 <== osd.10 v2:172.21.15.125:6824/1757421100 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f03c00aecd0 con 0x7f03c00ac170 2023-11-16T16:47:36.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.692+0000 7f03dbd89700 1 -- 172.21.15.93:0/3040956264 --> [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f03d4110510 con 0x7f03c00ac170 2023-11-16T16:47:36.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.693+0000 7f03caffd700 1 -- 172.21.15.93:0/3040956264 <== osd.10 v2:172.21.15.125:6824/1757421100 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f03d4110510 con 0x7f03c00ac170 2023-11-16T16:47:36.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.693+0000 7f03dbd89700 1 -- 172.21.15.93:0/3040956264 >> [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100] conn(0x7f03c00ac170 msgr2=0x7f03c00ae600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:36.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.693+0000 7f03dbd89700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100] conn(0x7f03c00ac170 0x7f03c00ae600 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:36.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.693+0000 7f03dbd89700 1 -- 172.21.15.93:0/3040956264 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f03c00a11e0 msgr2=0x7f03c00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:36.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.693+0000 7f03dbd89700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f03c00a11e0 0x7f03c00a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f03d0011550 tx=0x7f03d0009260 comp rx=0 tx=0).stop 2023-11-16T16:47:36.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.693+0000 7f03dbd89700 1 -- 172.21.15.93:0/3040956264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03d4100ee0 msgr2=0x7f03d4196930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:36.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.693+0000 7f03dbd89700 1 --2- 172.21.15.93:0/3040956264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03d4100ee0 0x7f03d4196930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03c40058e0 tx=0x7f03c400f690 comp rx=0 tx=0).stop 2023-11-16T16:47:36.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.693+0000 7f03da326700 1 -- 172.21.15.93:0/3040956264 reap_dead start 2023-11-16T16:47:36.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.693+0000 7f03dbd89700 1 -- 172.21.15.93:0/3040956264 shutdown_connections 2023-11-16T16:47:36.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.693+0000 7f03dbd89700 1 -- 172.21.15.93:0/3040956264 >> 172.21.15.93:0/3040956264 conn(0x7f03d40fbdc0 msgr2=0x7f03d40fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:36.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.694+0000 7f03dbd89700 1 -- 172.21.15.93:0/3040956264 shutdown_connections 2023-11-16T16:47:36.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:36.694+0000 7f03dbd89700 1 -- 172.21.15.93:0/3040956264 wait complete. 2023-11-16T16:47:37.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.125+0000 7f4d3db6a700 1 -- 172.21.15.93:0/3132257481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d38100ee0 msgr2=0x7f4d38104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.125+0000 7f4d3db6a700 1 --2- 172.21.15.93:0/3132257481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d38100ee0 0x7f4d38104eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0099c0 tx=0x7f4d2c009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.125+0000 7f4d3db6a700 1 -- 172.21.15.93:0/3132257481 shutdown_connections 2023-11-16T16:47:37.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.125+0000 7f4d3db6a700 1 --2- 172.21.15.93:0/3132257481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d38100540 0x7f4d38100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.125+0000 7f4d3db6a700 1 --2- 172.21.15.93:0/3132257481 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d381055e0 0x7f4d3810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.125+0000 7f4d3db6a700 1 --2- 172.21.15.93:0/3132257481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d38100ee0 0x7f4d38104eb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.125+0000 7f4d3db6a700 1 -- 172.21.15.93:0/3132257481 >> 172.21.15.93:0/3132257481 conn(0x7f4d380fbdc0 msgr2=0x7f4d380fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:37.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.125+0000 7f4d3db6a700 1 -- 172.21.15.93:0/3132257481 shutdown_connections 2023-11-16T16:47:37.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.125+0000 7f4d3db6a700 1 -- 172.21.15.93:0/3132257481 wait complete. 2023-11-16T16:47:37.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.126+0000 7f4d3db6a700 1 Processor -- start 2023-11-16T16:47:37.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.127+0000 7f4d3db6a700 1 -- start start 2023-11-16T16:47:37.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.127+0000 7f4d3db6a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d38100540 0x7f4d38196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.127+0000 7f4d3db6a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d38100ee0 0x7f4d38196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.127+0000 7f4d3db6a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d381055e0 0x7f4d3819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.127+0000 7f4d3db6a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3819b400 con 0x7f4d381055e0 2023-11-16T16:47:37.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.127+0000 7f4d3db6a700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3819b570 con 0x7f4d38100540 2023-11-16T16:47:37.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.127+0000 7f4d3db6a700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3819b730 con 0x7f4d38100ee0 2023-11-16T16:47:37.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.128+0000 7f4d36ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d38100ee0 0x7f4d38196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.128+0000 7f4d36ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d38100ee0 0x7f4d38196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:44336/0 (socket says 172.21.15.93:44336) 2023-11-16T16:47:37.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.128+0000 7f4d377fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d38100540 0x7f4d38196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.128+0000 7f4d36ffd700 1 -- 172.21.15.93:0/84016037 learned_addr learned my addr 172.21.15.93:0/84016037 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:37.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.128+0000 7f4d37fff700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d381055e0 0x7f4d3819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.128+0000 7f4d36ffd700 1 -- 172.21.15.93:0/84016037 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d38100540 msgr2=0x7f4d38196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.128+0000 7f4d36ffd700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d38100540 0x7f4d38196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.128+0000 7f4d36ffd700 1 -- 172.21.15.93:0/84016037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d381055e0 msgr2=0x7f4d3819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.128+0000 7f4d36ffd700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d381055e0 0x7f4d3819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.128+0000 7f4d36ffd700 1 -- 172.21.15.93:0/84016037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d30009c40 con 0x7f4d38100ee0 2023-11-16T16:47:37.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.128+0000 7f4d377fe700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d38100540 0x7f4d38196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:37.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.128+0000 7f4d36ffd700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d38100ee0 0x7f4d38196980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00b580 tx=0x7f4d2c00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:37.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.129+0000 7f4d34ff9700 1 -- 172.21.15.93:0/84016037 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c01d070 con 0x7f4d38100ee0 2023-11-16T16:47:37.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.129+0000 7f4d34ff9700 1 -- 172.21.15.93:0/84016037 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d2c005050 con 0x7f4d38100ee0 2023-11-16T16:47:37.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.129+0000 7f4d3db6a700 1 -- 172.21.15.93:0/84016037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2c0095f0 con 0x7f4d38100ee0 2023-11-16T16:47:37.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.129+0000 7f4d34ff9700 1 -- 172.21.15.93:0/84016037 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c004950 con 0x7f4d38100ee0 2023-11-16T16:47:37.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.129+0000 7f4d3db6a700 1 -- 172.21.15.93:0/84016037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d38110310 con 0x7f4d38100ee0 2023-11-16T16:47:37.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.130+0000 7f4d3db6a700 1 -- 172.21.15.93:0/84016037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4d380fd1e0 con 0x7f4d38100ee0 2023-11-16T16:47:37.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.133+0000 7f4d34ff9700 1 -- 172.21.15.93:0/84016037 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4d2c0051c0 con 0x7f4d38100ee0 2023-11-16T16:47:37.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.134+0000 7f4d34ff9700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4d200a1260 0x7f4d200a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.135+0000 7f4d34ff9700 1 -- 172.21.15.93:0/84016037 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4d2c0cdbc0 con 0x7f4d38100ee0 2023-11-16T16:47:37.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.135+0000 7f4d34ff9700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922] conn(0x7f4d200ac1f0 0x7f4d200ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.135+0000 7f4d377fe700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4d200a1260 0x7f4d200a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.135+0000 7f4d34ff9700 1 -- 172.21.15.93:0/84016037 --> [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d200aed50 con 0x7f4d200ac1f0 2023-11-16T16:47:37.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.135+0000 7f4d34ff9700 1 -- 172.21.15.93:0/84016037 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d2c0cdfd0 con 0x7f4d38100ee0 2023-11-16T16:47:37.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.135+0000 7f4d37fff700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922] conn(0x7f4d200ac1f0 0x7f4d200ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.135+0000 7f4d377fe700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4d200a1260 0x7f4d200a3710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d28011490 tx=0x7f4d28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:37.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.136+0000 7f4d37fff700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922] conn(0x7f4d200ac1f0 0x7f4d200ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:37.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.137+0000 7f4d34ff9700 1 -- 172.21.15.93:0/84016037 <== osd.7 v2:172.21.15.125:6816/3103492922 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4d200aed50 con 0x7f4d200ac1f0 2023-11-16T16:47:37.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.153+0000 7f4d3db6a700 1 -- 172.21.15.93:0/84016037 --> [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4d38110180 con 0x7f4d200ac1f0 2023-11-16T16:47:37.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.154+0000 7f4d34ff9700 1 -- 172.21.15.93:0/84016037 <== osd.7 v2:172.21.15.125:6816/3103492922 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4d38110180 con 0x7f4d200ac1f0 2023-11-16T16:47:37.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.155+0000 7f4d3db6a700 1 -- 172.21.15.93:0/84016037 >> [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922] conn(0x7f4d200ac1f0 msgr2=0x7f4d200ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.155+0000 7f4d3db6a700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922] conn(0x7f4d200ac1f0 0x7f4d200ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.155+0000 7f4d3db6a700 1 -- 172.21.15.93:0/84016037 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4d200a1260 msgr2=0x7f4d200a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.155+0000 7f4d3db6a700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4d200a1260 0x7f4d200a3710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d28011490 tx=0x7f4d28009260 comp rx=0 tx=0).stop 2023-11-16T16:47:37.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.155+0000 7f4d3db6a700 1 -- 172.21.15.93:0/84016037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d38100ee0 msgr2=0x7f4d38196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.155+0000 7f4d3db6a700 1 --2- 172.21.15.93:0/84016037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d38100ee0 0x7f4d38196980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00b580 tx=0x7f4d2c00bfd0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.155+0000 7f4d37fff700 1 -- 172.21.15.93:0/84016037 reap_dead start 2023-11-16T16:47:37.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.155+0000 7f4d3db6a700 1 -- 172.21.15.93:0/84016037 shutdown_connections 2023-11-16T16:47:37.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.155+0000 7f4d3db6a700 1 -- 172.21.15.93:0/84016037 >> 172.21.15.93:0/84016037 conn(0x7f4d380fbdc0 msgr2=0x7f4d380fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:37.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.155+0000 7f4d3db6a700 1 -- 172.21.15.93:0/84016037 shutdown_connections 2023-11-16T16:47:37.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.155+0000 7f4d3db6a700 1 -- 172.21.15.93:0/84016037 wait complete. 2023-11-16T16:47:37.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.400+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3822394221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa858102da0 msgr2=0x7fa85810f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.400+0000 7fa85ee7c700 1 --2- 172.21.15.93:0/3822394221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa858102da0 0x7fa85810f4b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa850009f90 tx=0x7fa85000b330 comp rx=0 tx=0).stop 2023-11-16T16:47:37.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.401+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3822394221 shutdown_connections 2023-11-16T16:47:37.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.401+0000 7fa85ee7c700 1 --2- 172.21.15.93:0/3822394221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa858102da0 0x7fa85810f4b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.401+0000 7fa85ee7c700 1 --2- 172.21.15.93:0/3822394221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa858108670 0x7fa858108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.401+0000 7fa85ee7c700 1 --2- 172.21.15.93:0/3822394221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa858108f80 0x7fa858102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.401+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3822394221 >> 172.21.15.93:0/3822394221 conn(0x7fa8580fe170 msgr2=0x7fa858100580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:37.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.401+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3822394221 shutdown_connections 2023-11-16T16:47:37.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.401+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3822394221 wait complete. 2023-11-16T16:47:37.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.402+0000 7fa85ee7c700 1 Processor -- start 2023-11-16T16:47:37.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.402+0000 7fa85ee7c700 1 -- start start 2023-11-16T16:47:37.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.403+0000 7fa85ee7c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa858102da0 0x7fa858075000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.403+0000 7fa85ee7c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa858108670 0x7fa858075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.403+0000 7fa85ee7c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa858108f80 0x7fa858075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.403+0000 7fa85ee7c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581a8e30 con 0x7fa858102da0 2023-11-16T16:47:37.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.403+0000 7fa85ee7c700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581a8fa0 con 0x7fa858108670 2023-11-16T16:47:37.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.403+0000 7fa85ee7c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa858198500 con 0x7fa858108f80 2023-11-16T16:47:37.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.403+0000 7fa85cc18700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa858102da0 0x7fa858075000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.403+0000 7fa85cc18700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa858102da0 0x7fa858075000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46284/0 (socket says 172.21.15.93:46284) 2023-11-16T16:47:37.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.403+0000 7fa85cc18700 1 -- 172.21.15.93:0/3491595360 learned_addr learned my addr 172.21.15.93:0/3491595360 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:37.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.403+0000 7fa857fff700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa858108670 0x7fa858075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.403+0000 7fa85d419700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa858108f80 0x7fa858075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.404+0000 7fa85cc18700 1 -- 172.21.15.93:0/3491595360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa858108f80 msgr2=0x7fa858075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.404+0000 7fa85cc18700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa858108f80 0x7fa858075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.404+0000 7fa85cc18700 1 -- 172.21.15.93:0/3491595360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa858108670 msgr2=0x7fa858075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.404+0000 7fa85cc18700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa858108670 0x7fa858075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.404+0000 7fa85cc18700 1 -- 172.21.15.93:0/3491595360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa850009c40 con 0x7fa858102da0 2023-11-16T16:47:37.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.404+0000 7fa85cc18700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa858102da0 0x7fa858075000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00e970 tx=0x7fa84c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:37.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.404+0000 7fa855ffb700 1 -- 172.21.15.93:0/3491595360 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c009790 con 0x7fa858102da0 2023-11-16T16:47:37.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.404+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3491595360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8581126e0 con 0x7fa858102da0 2023-11-16T16:47:37.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.404+0000 7fa855ffb700 1 -- 172.21.15.93:0/3491595360 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa84c00c7c0 con 0x7fa858102da0 2023-11-16T16:47:37.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.404+0000 7fa855ffb700 1 -- 172.21.15.93:0/3491595360 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c003c20 con 0x7fa858102da0 2023-11-16T16:47:37.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.404+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3491595360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa858112550 con 0x7fa858102da0 2023-11-16T16:47:37.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.405+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3491595360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa858109ec0 con 0x7fa858102da0 2023-11-16T16:47:37.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.407+0000 7fa855ffb700 1 -- 172.21.15.93:0/3491595360 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa84c004220 con 0x7fa858102da0 2023-11-16T16:47:37.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.409+0000 7fa855ffb700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa8480a1260 0x7fa8480a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.409+0000 7fa855ffb700 1 -- 172.21.15.93:0/3491595360 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa84c014070 con 0x7fa858102da0 2023-11-16T16:47:37.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.409+0000 7fa857fff700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa8480a1260 0x7fa8480a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.409+0000 7fa855ffb700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195] conn(0x7fa8480ac1f0 0x7fa8480ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.409+0000 7fa855ffb700 1 -- 172.21.15.93:0/3491595360 --> [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8480aed50 con 0x7fa8480ac1f0 2023-11-16T16:47:37.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.409+0000 7fa855ffb700 1 -- 172.21.15.93:0/3491595360 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa84c0cd380 con 0x7fa858102da0 2023-11-16T16:47:37.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.410+0000 7fa85d419700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195] conn(0x7fa8480ac1f0 0x7fa8480ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.410+0000 7fa857fff700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa8480a1260 0x7fa8480a3710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa8580767f0 tx=0x7fa844009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:37.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.410+0000 7fa85d419700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195] conn(0x7fa8480ac1f0 0x7fa8480ae680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:37.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.411+0000 7fa855ffb700 1 -- 172.21.15.93:0/3491595360 <== osd.6 v2:172.21.15.93:6818/1437083195 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa8480aed50 con 0x7fa8480ac1f0 2023-11-16T16:47:37.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.428+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3491595360 --> [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa858112870 con 0x7fa8480ac1f0 2023-11-16T16:47:37.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.428+0000 7fa855ffb700 1 -- 172.21.15.93:0/3491595360 <== osd.6 v2:172.21.15.93:6818/1437083195 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa858112870 con 0x7fa8480ac1f0 2023-11-16T16:47:37.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.428+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195] conn(0x7fa8480ac1f0 msgr2=0x7fa8480ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.428+0000 7fa85ee7c700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195] conn(0x7fa8480ac1f0 0x7fa8480ae680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.428+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa8480a1260 msgr2=0x7fa8480a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.428+0000 7fa85ee7c700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa8480a1260 0x7fa8480a3710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa8580767f0 tx=0x7fa844009310 comp rx=0 tx=0).stop 2023-11-16T16:47:37.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.428+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa858102da0 msgr2=0x7fa858075000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.429+0000 7fa85ee7c700 1 --2- 172.21.15.93:0/3491595360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa858102da0 0x7fa858075000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00e970 tx=0x7fa84c00ec80 comp rx=0 tx=0).stop 2023-11-16T16:47:37.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.429+0000 7fa85d419700 1 -- 172.21.15.93:0/3491595360 reap_dead start 2023-11-16T16:47:37.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.429+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3491595360 shutdown_connections 2023-11-16T16:47:37.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.429+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3491595360 >> 172.21.15.93:0/3491595360 conn(0x7fa8580fe170 msgr2=0x7fa8581094f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:37.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.429+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3491595360 shutdown_connections 2023-11-16T16:47:37.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.429+0000 7fa85ee7c700 1 -- 172.21.15.93:0/3491595360 wait complete. 2023-11-16T16:47:37.852 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:37 smithi093 ceph-mon[97863]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:47:37.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.948+0000 7fcb5846a700 1 -- 172.21.15.93:0/483490678 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb501055e0 msgr2=0x7fcb5010d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.948+0000 7fcb5846a700 1 --2- 172.21.15.93:0/483490678 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb501055e0 0x7fcb5010d160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009f90 tx=0x7fcb4800b330 comp rx=0 tx=0).stop 2023-11-16T16:47:37.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.948+0000 7fcb5846a700 1 -- 172.21.15.93:0/483490678 shutdown_connections 2023-11-16T16:47:37.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.948+0000 7fcb5846a700 1 --2- 172.21.15.93:0/483490678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb50100540 0x7fcb50100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.948+0000 7fcb5846a700 1 --2- 172.21.15.93:0/483490678 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb501055e0 0x7fcb5010d160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.948+0000 7fcb5846a700 1 --2- 172.21.15.93:0/483490678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb50100ee0 0x7fcb50104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.948+0000 7fcb5846a700 1 -- 172.21.15.93:0/483490678 >> 172.21.15.93:0/483490678 conn(0x7fcb500fbdc0 msgr2=0x7fcb500fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:37.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.948+0000 7fcb5846a700 1 -- 172.21.15.93:0/483490678 shutdown_connections 2023-11-16T16:47:37.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.948+0000 7fcb5846a700 1 -- 172.21.15.93:0/483490678 wait complete. 2023-11-16T16:47:37.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.949+0000 7fcb5846a700 1 Processor -- start 2023-11-16T16:47:37.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.949+0000 7fcb5846a700 1 -- start start 2023-11-16T16:47:37.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb5846a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb50100540 0x7fcb501964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb5846a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb50100ee0 0x7fcb50196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb5846a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb501055e0 0x7fcb5019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb5846a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019b480 con 0x7fcb50100540 2023-11-16T16:47:37.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb5846a700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019b5f0 con 0x7fcb501055e0 2023-11-16T16:47:37.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb5846a700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019b7b0 con 0x7fcb50100ee0 2023-11-16T16:47:37.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb56206700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb50100540 0x7fcb501964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb55a05700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb50100ee0 0x7fcb50196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb56206700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb50100540 0x7fcb501964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46294/0 (socket says 172.21.15.93:46294) 2023-11-16T16:47:37.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb56206700 1 -- 172.21.15.93:0/2613115057 learned_addr learned my addr 172.21.15.93:0/2613115057 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:37.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb56a07700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb501055e0 0x7fcb5019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb55a05700 1 -- 172.21.15.93:0/2613115057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb501055e0 msgr2=0x7fcb5019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb55a05700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb501055e0 0x7fcb5019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb55a05700 1 -- 172.21.15.93:0/2613115057 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb50100540 msgr2=0x7fcb501964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb55a05700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb50100540 0x7fcb501964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.950+0000 7fcb55a05700 1 -- 172.21.15.93:0/2613115057 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb48009c40 con 0x7fcb50100ee0 2023-11-16T16:47:37.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.951+0000 7fcb55a05700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb50100ee0 0x7fcb50196a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000e9c0 tx=0x7fcb4000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:37.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.951+0000 7fcb477fe700 1 -- 172.21.15.93:0/2613115057 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000c9c0 con 0x7fcb50100ee0 2023-11-16T16:47:37.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.951+0000 7fcb5846a700 1 -- 172.21.15.93:0/2613115057 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb50110370 con 0x7fcb50100ee0 2023-11-16T16:47:37.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.951+0000 7fcb477fe700 1 -- 172.21.15.93:0/2613115057 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb40010430 con 0x7fcb50100ee0 2023-11-16T16:47:37.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.951+0000 7fcb5846a700 1 -- 172.21.15.93:0/2613115057 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb501101e0 con 0x7fcb50100ee0 2023-11-16T16:47:37.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.951+0000 7fcb477fe700 1 -- 172.21.15.93:0/2613115057 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40018680 con 0x7fcb50100ee0 2023-11-16T16:47:37.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.952+0000 7fcb5846a700 1 -- 172.21.15.93:0/2613115057 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcb34000ff0 con 0x7fcb50100ee0 2023-11-16T16:47:37.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.954+0000 7fcb477fe700 1 -- 172.21.15.93:0/2613115057 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fcb4000cb20 con 0x7fcb50100ee0 2023-11-16T16:47:37.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.955+0000 7fcb477fe700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcb3c0a11e0 0x7fcb3c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.955+0000 7fcb477fe700 1 -- 172.21.15.93:0/2613115057 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcb40014070 con 0x7fcb50100ee0 2023-11-16T16:47:37.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.956+0000 7fcb56206700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcb3c0a11e0 0x7fcb3c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.956+0000 7fcb477fe700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260] conn(0x7fcb3c0ac170 0x7fcb3c0ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:37.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.956+0000 7fcb477fe700 1 -- 172.21.15.93:0/2613115057 --> [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb3c0aecd0 con 0x7fcb3c0ac170 2023-11-16T16:47:37.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.956+0000 7fcb477fe700 1 -- 172.21.15.93:0/2613115057 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb400cd200 con 0x7fcb50100ee0 2023-11-16T16:47:37.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.956+0000 7fcb56a07700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260] conn(0x7fcb3c0ac170 0x7fcb3c0ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:37.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.956+0000 7fcb56206700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcb3c0a11e0 0x7fcb3c0a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c005950 tx=0x7fcb4c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:37.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.957+0000 7fcb56a07700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260] conn(0x7fcb3c0ac170 0x7fcb3c0ae600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:37.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.957+0000 7fcb477fe700 1 -- 172.21.15.93:0/2613115057 <== osd.9 v2:172.21.15.93:6826/594522260 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcb3c0aecd0 con 0x7fcb3c0ac170 2023-11-16T16:47:37.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.974+0000 7fcb5846a700 1 -- 172.21.15.93:0/2613115057 --> [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcb34002d70 con 0x7fcb3c0ac170 2023-11-16T16:47:37.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.975+0000 7fcb477fe700 1 -- 172.21.15.93:0/2613115057 <== osd.9 v2:172.21.15.93:6826/594522260 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcb34002d70 con 0x7fcb3c0ac170 2023-11-16T16:47:37.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.975+0000 7fcb5846a700 1 -- 172.21.15.93:0/2613115057 >> [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260] conn(0x7fcb3c0ac170 msgr2=0x7fcb3c0ae600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.975+0000 7fcb5846a700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260] conn(0x7fcb3c0ac170 0x7fcb3c0ae600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.975+0000 7fcb5846a700 1 -- 172.21.15.93:0/2613115057 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcb3c0a11e0 msgr2=0x7fcb3c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.975+0000 7fcb5846a700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcb3c0a11e0 0x7fcb3c0a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c005950 tx=0x7fcb4c009310 comp rx=0 tx=0).stop 2023-11-16T16:47:37.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.975+0000 7fcb5846a700 1 -- 172.21.15.93:0/2613115057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb50100ee0 msgr2=0x7fcb50196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:37.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.975+0000 7fcb5846a700 1 --2- 172.21.15.93:0/2613115057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb50100ee0 0x7fcb50196a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000e9c0 tx=0x7fcb4000ecd0 comp rx=0 tx=0).stop 2023-11-16T16:47:37.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.975+0000 7fcb56a07700 1 -- 172.21.15.93:0/2613115057 reap_dead start 2023-11-16T16:47:37.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.975+0000 7fcb5846a700 1 -- 172.21.15.93:0/2613115057 shutdown_connections 2023-11-16T16:47:37.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.975+0000 7fcb5846a700 1 -- 172.21.15.93:0/2613115057 >> 172.21.15.93:0/2613115057 conn(0x7fcb500fbdc0 msgr2=0x7fcb500fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:37.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.976+0000 7fcb5846a700 1 -- 172.21.15.93:0/2613115057 shutdown_connections 2023-11-16T16:47:37.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:37.976+0000 7fcb5846a700 1 -- 172.21.15.93:0/2613115057 wait complete. 2023-11-16T16:47:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:37 smithi159 ceph-mon[131528]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:47:38.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:37 smithi125 ceph-mon[131076]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:47:38.417 INFO:teuthology.orchestra.run.smithi093.stdout:309237645341 2023-11-16T16:47:38.418 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-11-16T16:47:38.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.722+0000 7f8f44ff3700 1 -- 172.21.15.93:0/3654704951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f400ffc40 msgr2=0x7f8f40104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:38.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.722+0000 7f8f44ff3700 1 --2- 172.21.15.93:0/3654704951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f400ffc40 0x7f8f40104120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009990 tx=0x7f8f34009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.722+0000 7f8f44ff3700 1 -- 172.21.15.93:0/3654704951 shutdown_connections 2023-11-16T16:47:38.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.722+0000 7f8f44ff3700 1 --2- 172.21.15.93:0/3654704951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f40104660 0x7f8f40104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.722+0000 7f8f44ff3700 1 --2- 172.21.15.93:0/3654704951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f400ff330 0x7f8f400ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.722+0000 7f8f44ff3700 1 --2- 172.21.15.93:0/3654704951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f400ffc40 0x7f8f40104120 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.722+0000 7f8f44ff3700 1 -- 172.21.15.93:0/3654704951 >> 172.21.15.93:0/3654704951 conn(0x7f8f400770a0 msgr2=0x7f8f400774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:38.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.722+0000 7f8f44ff3700 1 -- 172.21.15.93:0/3654704951 shutdown_connections 2023-11-16T16:47:38.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.723+0000 7f8f44ff3700 1 -- 172.21.15.93:0/3654704951 wait complete. 2023-11-16T16:47:38.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.724+0000 7f8f44ff3700 1 Processor -- start 2023-11-16T16:47:38.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.724+0000 7f8f44ff3700 1 -- start start 2023-11-16T16:47:38.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f44ff3700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f400ff330 0x7f8f401a2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:38.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f44ff3700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f400ffc40 0x7f8f401a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:38.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f44ff3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f40104660 0x7f8f4019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:38.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f44ff3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4019cfb0 con 0x7f8f40104660 2023-11-16T16:47:38.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f44ff3700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4019d120 con 0x7f8f400ff330 2023-11-16T16:47:38.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f44ff3700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4019d2e0 con 0x7f8f400ffc40 2023-11-16T16:47:38.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f3e59c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f400ff330 0x7f8f401a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:38.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f3e59c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f400ff330 0x7f8f401a2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:55980/0 (socket says 172.21.15.93:55980) 2023-11-16T16:47:38.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f3e59c700 1 -- 172.21.15.93:0/2015312417 learned_addr learned my addr 172.21.15.93:0/2015312417 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:38.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f3dd9b700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f400ffc40 0x7f8f401a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:38.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f3ed9d700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f40104660 0x7f8f4019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:38.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f3e59c700 1 -- 172.21.15.93:0/2015312417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f400ffc40 msgr2=0x7f8f401a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:38.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f3e59c700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f400ffc40 0x7f8f401a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f3e59c700 1 -- 172.21.15.93:0/2015312417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f40104660 msgr2=0x7f8f4019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:38.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f3e59c700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f40104660 0x7f8f4019c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.725+0000 7f8f3e59c700 1 -- 172.21.15.93:0/2015312417 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f340095f0 con 0x7f8f400ff330 2023-11-16T16:47:38.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.726+0000 7f8f3e59c700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f400ff330 0x7f8f401a2810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000e9a0 tx=0x7f8f3000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:38.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.726+0000 7f8f2f7fe700 1 -- 172.21.15.93:0/2015312417 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30009790 con 0x7f8f400ff330 2023-11-16T16:47:38.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.726+0000 7f8f2f7fe700 1 -- 172.21.15.93:0/2015312417 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f3000cbd0 con 0x7f8f400ff330 2023-11-16T16:47:38.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.726+0000 7f8f44ff3700 1 -- 172.21.15.93:0/2015312417 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f40116970 con 0x7f8f400ff330 2023-11-16T16:47:38.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.726+0000 7f8f2f7fe700 1 -- 172.21.15.93:0/2015312417 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30018680 con 0x7f8f400ff330 2023-11-16T16:47:38.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.726+0000 7f8f44ff3700 1 -- 172.21.15.93:0/2015312417 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f401167e0 con 0x7f8f400ff330 2023-11-16T16:47:38.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.727+0000 7f8f44ff3700 1 -- 172.21.15.93:0/2015312417 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f40103a80 con 0x7f8f400ff330 2023-11-16T16:47:38.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.730+0000 7f8f2f7fe700 1 -- 172.21.15.93:0/2015312417 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8f3000cd40 con 0x7f8f400ff330 2023-11-16T16:47:38.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.732+0000 7f8f2f7fe700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f280a1260 0x7f8f280a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:38.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.732+0000 7f8f2f7fe700 1 -- 172.21.15.93:0/2015312417 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f8f30014070 con 0x7f8f400ff330 2023-11-16T16:47:38.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.732+0000 7f8f3dd9b700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f280a1260 0x7f8f280a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:38.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.732+0000 7f8f2f7fe700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422] conn(0x7f8f280ac1f0 0x7f8f280ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:38.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.732+0000 7f8f2f7fe700 1 -- 172.21.15.93:0/2015312417 --> [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f280aed50 con 0x7f8f280ac1f0 2023-11-16T16:47:38.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.732+0000 7f8f2f7fe700 1 -- 172.21.15.93:0/2015312417 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f300cd290 con 0x7f8f400ff330 2023-11-16T16:47:38.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.733+0000 7f8f3ed9d700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422] conn(0x7f8f280ac1f0 0x7f8f280ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:38.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.733+0000 7f8f3dd9b700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f280a1260 0x7f8f280a3710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f4019e030 tx=0x7f8f34005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:38.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.733+0000 7f8f3ed9d700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422] conn(0x7f8f280ac1f0 0x7f8f280ae680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:38.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.734+0000 7f8f2f7fe700 1 -- 172.21.15.93:0/2015312417 <== osd.11 v2:172.21.15.159:6824/2996534422 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8f280aed50 con 0x7f8f280ac1f0 2023-11-16T16:47:38.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.751+0000 7f8f44ff3700 1 -- 172.21.15.93:0/2015312417 --> [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f40116b00 con 0x7f8f280ac1f0 2023-11-16T16:47:38.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.752+0000 7f8f2f7fe700 1 -- 172.21.15.93:0/2015312417 <== osd.11 v2:172.21.15.159:6824/2996534422 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8f40116b00 con 0x7f8f280ac1f0 2023-11-16T16:47:38.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.753+0000 7f8f44ff3700 1 -- 172.21.15.93:0/2015312417 >> [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422] conn(0x7f8f280ac1f0 msgr2=0x7f8f280ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:38.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.753+0000 7f8f44ff3700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422] conn(0x7f8f280ac1f0 0x7f8f280ae680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.753+0000 7f8f44ff3700 1 -- 172.21.15.93:0/2015312417 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f280a1260 msgr2=0x7f8f280a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:38.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.753+0000 7f8f44ff3700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f280a1260 0x7f8f280a3710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f4019e030 tx=0x7f8f34005fd0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.753+0000 7f8f44ff3700 1 -- 172.21.15.93:0/2015312417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f400ff330 msgr2=0x7f8f401a2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:38.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.753+0000 7f8f44ff3700 1 --2- 172.21.15.93:0/2015312417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f400ff330 0x7f8f401a2810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000e9a0 tx=0x7f8f3000ecb0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.753+0000 7f8f3ed9d700 1 -- 172.21.15.93:0/2015312417 reap_dead start 2023-11-16T16:47:38.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.753+0000 7f8f44ff3700 1 -- 172.21.15.93:0/2015312417 shutdown_connections 2023-11-16T16:47:38.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.753+0000 7f8f44ff3700 1 -- 172.21.15.93:0/2015312417 >> 172.21.15.93:0/2015312417 conn(0x7f8f400770a0 msgr2=0x7f8f40102060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:38.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.753+0000 7f8f44ff3700 1 -- 172.21.15.93:0/2015312417 shutdown_connections 2023-11-16T16:47:38.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.753+0000 7f8f44ff3700 1 -- 172.21.15.93:0/2015312417 wait complete. 2023-11-16T16:47:38.777 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:38 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:47:38.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.792+0000 7fe023775700 1 -- 172.21.15.93:0/2382269580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c102530 msgr2=0x7fe01c1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:38.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.792+0000 7fe023775700 1 --2- 172.21.15.93:0/2382269580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c102530 0x7fe01c1029a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009990 tx=0x7fe00c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.792+0000 7fe023775700 1 -- 172.21.15.93:0/2382269580 shutdown_connections 2023-11-16T16:47:38.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.792+0000 7fe023775700 1 --2- 172.21.15.93:0/2382269580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe01c108550 0x7fe01c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.792+0000 7fe023775700 1 --2- 172.21.15.93:0/2382269580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe01c102ee0 0x7fe01c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.792+0000 7fe023775700 1 --2- 172.21.15.93:0/2382269580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c102530 0x7fe01c1029a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.792+0000 7fe023775700 1 -- 172.21.15.93:0/2382269580 >> 172.21.15.93:0/2382269580 conn(0x7fe01c0fdfd0 msgr2=0x7fe01c1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:38.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.793+0000 7fe023775700 1 -- 172.21.15.93:0/2382269580 shutdown_connections 2023-11-16T16:47:38.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.793+0000 7fe023775700 1 -- 172.21.15.93:0/2382269580 wait complete. 2023-11-16T16:47:38.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe023775700 1 Processor -- start 2023-11-16T16:47:38.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe023775700 1 -- start start 2023-11-16T16:47:38.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe023775700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe01c102530 0x7fe01c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:38.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe023775700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c102ee0 0x7fe01c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:38.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe023775700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe01c108550 0x7fe01c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:38.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe023775700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c19d600 con 0x7fe01c108550 2023-11-16T16:47:38.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe023775700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c19d770 con 0x7fe01c102530 2023-11-16T16:47:38.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe023775700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c19d930 con 0x7fe01c102ee0 2023-11-16T16:47:38.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe021d12700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe01c108550 0x7fe01c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:38.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe020d10700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c102ee0 0x7fe01c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:38.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe021511700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe01c102530 0x7fe01c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:38.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe020d10700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c102ee0 0x7fe01c198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:44412/0 (socket says 172.21.15.93:44412) 2023-11-16T16:47:38.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.794+0000 7fe020d10700 1 -- 172.21.15.93:0/2662166640 learned_addr learned my addr 172.21.15.93:0/2662166640 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:38.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.795+0000 7fe020d10700 1 -- 172.21.15.93:0/2662166640 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe01c102530 msgr2=0x7fe01c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:38.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.795+0000 7fe020d10700 1 --2- 172.21.15.93:0/2662166640 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe01c102530 0x7fe01c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.795+0000 7fe020d10700 1 -- 172.21.15.93:0/2662166640 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe01c108550 msgr2=0x7fe01c19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:38.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.795+0000 7fe020d10700 1 --2- 172.21.15.93:0/2662166640 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe01c108550 0x7fe01c19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.795+0000 7fe020d10700 1 -- 172.21.15.93:0/2662166640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe014009c40 con 0x7fe01c102ee0 2023-11-16T16:47:38.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.795+0000 7fe020d10700 1 --2- 172.21.15.93:0/2662166640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c102ee0 0x7fe01c198b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00b680 tx=0x7fe00c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:38.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.795+0000 7fe0127fc700 1 -- 172.21.15.93:0/2662166640 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c01d070 con 0x7fe01c102ee0 2023-11-16T16:47:38.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.796+0000 7fe0127fc700 1 -- 172.21.15.93:0/2662166640 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe00c021410 con 0x7fe01c102ee0 2023-11-16T16:47:38.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.796+0000 7fe023775700 1 -- 172.21.15.93:0/2662166640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00c0095f0 con 0x7fe01c102ee0 2023-11-16T16:47:38.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.796+0000 7fe0127fc700 1 -- 172.21.15.93:0/2662166640 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c017650 con 0x7fe01c102ee0 2023-11-16T16:47:38.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.796+0000 7fe023775700 1 -- 172.21.15.93:0/2662166640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c112560 con 0x7fe01c102ee0 2023-11-16T16:47:38.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.796+0000 7fe023775700 1 -- 172.21.15.93:0/2662166640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe01c0ffea0 con 0x7fe01c102ee0 2023-11-16T16:47:38.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.799+0000 7fe0127fc700 1 -- 172.21.15.93:0/2662166640 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe00c021580 con 0x7fe01c102ee0 2023-11-16T16:47:38.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.800+0000 7fe0127fc700 1 --2- 172.21.15.93:0/2662166640 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe0080a1260 0x7fe0080a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:38.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.800+0000 7fe0127fc700 1 -- 172.21.15.93:0/2662166640 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe00c0ceba0 con 0x7fe01c102ee0 2023-11-16T16:47:38.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.800+0000 7fe021511700 1 --2- 172.21.15.93:0/2662166640 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe0080a1260 0x7fe0080a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:38.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.800+0000 7fe0127fc700 1 --2- 172.21.15.93:0/2662166640 >> [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992] conn(0x7fe0080ac1f0 0x7fe0080ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:38.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.800+0000 7fe0127fc700 1 -- 172.21.15.93:0/2662166640 --> [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0080aed50 con 0x7fe0080ac1f0 2023-11-16T16:47:38.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.800+0000 7fe0127fc700 1 -- 172.21.15.93:0/2662166640 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe00c0cef10 con 0x7fe01c102ee0 2023-11-16T16:47:38.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.801+0000 7fe021d12700 1 --2- 172.21.15.93:0/2662166640 >> [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992] conn(0x7fe0080ac1f0 0x7fe0080ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:38.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.801+0000 7fe021511700 1 --2- 172.21.15.93:0/2662166640 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe0080a1260 0x7fe0080a3710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe018007b60 tx=0x7fe0180073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:38.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.801+0000 7fe021d12700 1 --2- 172.21.15.93:0/2662166640 >> [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992] conn(0x7fe0080ac1f0 0x7fe0080ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:38.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.802+0000 7fe0127fc700 1 -- 172.21.15.93:0/2662166640 <== osd.2 v2:172.21.15.159:6800/189817992 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe0080aed50 con 0x7fe0080ac1f0 2023-11-16T16:47:38.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.819+0000 7fe023775700 1 -- 172.21.15.93:0/2662166640 --> [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe01c1123d0 con 0x7fe0080ac1f0 2023-11-16T16:47:38.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.820+0000 7fe0127fc700 1 -- 172.21.15.93:0/2662166640 <== osd.2 v2:172.21.15.159:6800/189817992 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe01c1123d0 con 0x7fe0080ac1f0 2023-11-16T16:47:38.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.820+0000 7fe023775700 1 -- 172.21.15.93:0/2662166640 >> [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992] conn(0x7fe0080ac1f0 msgr2=0x7fe0080ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:38.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.820+0000 7fe023775700 1 --2- 172.21.15.93:0/2662166640 >> [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992] conn(0x7fe0080ac1f0 0x7fe0080ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.820+0000 7fe023775700 1 -- 172.21.15.93:0/2662166640 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe0080a1260 msgr2=0x7fe0080a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:38.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.820+0000 7fe023775700 1 --2- 172.21.15.93:0/2662166640 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe0080a1260 0x7fe0080a3710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe018007b60 tx=0x7fe0180073d0 comp rx=0 tx=0).stop 2023-11-16T16:47:38.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.820+0000 7fe023775700 1 -- 172.21.15.93:0/2662166640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c102ee0 msgr2=0x7fe01c198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:38.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.820+0000 7fe023775700 1 --2- 172.21.15.93:0/2662166640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c102ee0 0x7fe01c198b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00b680 tx=0x7fe00c005740 comp rx=0 tx=0).stop 2023-11-16T16:47:38.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.820+0000 7fe021d12700 1 -- 172.21.15.93:0/2662166640 reap_dead start 2023-11-16T16:47:38.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.820+0000 7fe023775700 1 -- 172.21.15.93:0/2662166640 shutdown_connections 2023-11-16T16:47:38.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.820+0000 7fe023775700 1 -- 172.21.15.93:0/2662166640 >> 172.21.15.93:0/2662166640 conn(0x7fe01c0fdfd0 msgr2=0x7fe01c0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:38.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.820+0000 7fe023775700 1 -- 172.21.15.93:0/2662166640 shutdown_connections 2023-11-16T16:47:38.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:38.820+0000 7fe023775700 1 -- 172.21.15.93:0/2662166640 wait complete. 2023-11-16T16:47:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:38 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:47:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:38 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:47:39.157 INFO:teuthology.orchestra.run.smithi093.stdout:163208757302 2023-11-16T16:47:39.157 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-11-16T16:47:39.474 INFO:teuthology.orchestra.run.smithi093.stdout:34359738444 2023-11-16T16:47:39.475 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-11-16T16:47:39.511 INFO:teuthology.orchestra.run.smithi093.stdout:197568495665 2023-11-16T16:47:39.512 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-11-16T16:47:39.633 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:39 smithi093 ceph-mon[97863]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:47:39.955 INFO:teuthology.orchestra.run.smithi093.stdout:219043332140 2023-11-16T16:47:39.956 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-11-16T16:47:39.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:39 smithi159 ceph-mon[131528]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:47:39.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.995+0000 7f661deb1700 1 -- 172.21.15.93:0/1374842767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6618100540 msgr2=0x7f6618100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:39.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.995+0000 7f661deb1700 1 --2- 172.21.15.93:0/1374842767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6618100540 0x7f6618100910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66080098f0 tx=0x7f6608009c00 comp rx=0 tx=0).stop 2023-11-16T16:47:39.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.995+0000 7f661deb1700 1 -- 172.21.15.93:0/1374842767 shutdown_connections 2023-11-16T16:47:39.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.995+0000 7f661deb1700 1 --2- 172.21.15.93:0/1374842767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66181055e0 0x7f661810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:39.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.995+0000 7f661deb1700 1 --2- 172.21.15.93:0/1374842767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6618100540 0x7f6618100910 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:39.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.995+0000 7f661deb1700 1 --2- 172.21.15.93:0/1374842767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6618100ee0 0x7f6618104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:39.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.995+0000 7f661deb1700 1 -- 172.21.15.93:0/1374842767 >> 172.21.15.93:0/1374842767 conn(0x7f66180fbdc0 msgr2=0x7f66180fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:39.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.995+0000 7f661deb1700 1 -- 172.21.15.93:0/1374842767 shutdown_connections 2023-11-16T16:47:39.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.995+0000 7f661deb1700 1 -- 172.21.15.93:0/1374842767 wait complete. 2023-11-16T16:47:39.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.996+0000 7f661deb1700 1 Processor -- start 2023-11-16T16:47:40.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.997+0000 7f661deb1700 1 -- start start 2023-11-16T16:47:40.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.997+0000 7f661deb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6618100540 0x7f661810c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:40.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.997+0000 7f661deb1700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6618100ee0 0x7f661810ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:40.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.997+0000 7f661deb1700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66181055e0 0x7f66181089d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:40.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.997+0000 7f661deb1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6618109150 con 0x7f6618100540 2023-11-16T16:47:40.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.997+0000 7f661deb1700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66181092c0 con 0x7f66181055e0 2023-11-16T16:47:40.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.997+0000 7f661deb1700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6618109480 con 0x7f6618100ee0 2023-11-16T16:47:40.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.997+0000 7f6616ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6618100ee0 0x7f661810ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:40.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.998+0000 7f66177fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6618100540 0x7f661810c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.998+0000 7f6616ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6618100ee0 0x7f661810ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:44426/0 (socket says 172.21.15.93:44426) 2023-11-16T16:47:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.998+0000 7f6616ffd700 1 -- 172.21.15.93:0/1845083792 learned_addr learned my addr 172.21.15.93:0/1845083792 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.998+0000 7f6617fff700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66181055e0 0x7f66181089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.998+0000 7f6616ffd700 1 -- 172.21.15.93:0/1845083792 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66181055e0 msgr2=0x7f66181089d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.998+0000 7f6616ffd700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66181055e0 0x7f66181089d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.998+0000 7f6616ffd700 1 -- 172.21.15.93:0/1845083792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6618100540 msgr2=0x7f661810c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.998+0000 7f6616ffd700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6618100540 0x7f661810c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.998+0000 7f6616ffd700 1 -- 172.21.15.93:0/1845083792 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0095f0 con 0x7f6618100ee0 2023-11-16T16:47:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.998+0000 7f6616ffd700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6618100ee0 0x7f661810ce50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f660c00ec70 tx=0x7f660c00ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.999+0000 7f6614ff9700 1 -- 172.21.15.93:0/1845083792 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c0099d0 con 0x7f6618100ee0 2023-11-16T16:47:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.999+0000 7f661deb1700 1 -- 172.21.15.93:0/1845083792 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6608009520 con 0x7f6618100ee0 2023-11-16T16:47:40.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.999+0000 7f6614ff9700 1 -- 172.21.15.93:0/1845083792 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660c0044e0 con 0x7f6618100ee0 2023-11-16T16:47:40.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.999+0000 7f661deb1700 1 -- 172.21.15.93:0/1845083792 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6618073490 con 0x7f6618100ee0 2023-11-16T16:47:40.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:39.999+0000 7f6614ff9700 1 -- 172.21.15.93:0/1845083792 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c010430 con 0x7f6618100ee0 2023-11-16T16:47:40.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.000+0000 7f661deb1700 1 -- 172.21.15.93:0/1845083792 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f66180fd120 con 0x7f6618100ee0 2023-11-16T16:47:40.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.003+0000 7f6614ff9700 1 -- 172.21.15.93:0/1845083792 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f660c003680 con 0x7f6618100ee0 2023-11-16T16:47:40.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.005+0000 7f6614ff9700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f66000a1240 0x7f66000a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:40.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.005+0000 7f6614ff9700 1 -- 172.21.15.93:0/1845083792 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f660c014070 con 0x7f6618100ee0 2023-11-16T16:47:40.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.005+0000 7f6614ff9700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088] conn(0x7f66000ac1d0 0x7f66000ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:40.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.005+0000 7f6614ff9700 1 -- 172.21.15.93:0/1845083792 --> [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66000aed30 con 0x7f66000ac1d0 2023-11-16T16:47:40.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.005+0000 7f6614ff9700 1 -- 172.21.15.93:0/1845083792 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f660c010430 con 0x7f6618100ee0 2023-11-16T16:47:40.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.006+0000 7f66177fe700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f66000a1240 0x7f66000a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:40.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.006+0000 7f6617fff700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088] conn(0x7f66000ac1d0 0x7f66000ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:40.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.006+0000 7f6617fff700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088] conn(0x7f66000ac1d0 0x7f66000ae660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:40.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.007+0000 7f66177fe700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f66000a1240 0x7f66000a36f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6608009fd0 tx=0x7f6608019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:40.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.007+0000 7f6614ff9700 1 -- 172.21.15.93:0/1845083792 <== osd.4 v2:172.21.15.125:6808/3405048088 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f66000aed30 con 0x7f66000ac1d0 2023-11-16T16:47:40.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.026+0000 7f661deb1700 1 -- 172.21.15.93:0/1845083792 --> [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6618073300 con 0x7f66000ac1d0 2023-11-16T16:47:40.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.027+0000 7f6614ff9700 1 -- 172.21.15.93:0/1845083792 <== osd.4 v2:172.21.15.125:6808/3405048088 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6618073300 con 0x7f66000ac1d0 2023-11-16T16:47:40.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.028+0000 7f661deb1700 1 -- 172.21.15.93:0/1845083792 >> [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088] conn(0x7f66000ac1d0 msgr2=0x7f66000ae660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:40.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.028+0000 7f661deb1700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088] conn(0x7f66000ac1d0 0x7f66000ae660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:40.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.028+0000 7f661deb1700 1 -- 172.21.15.93:0/1845083792 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f66000a1240 msgr2=0x7f66000a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:40.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.028+0000 7f661deb1700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f66000a1240 0x7f66000a36f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6608009fd0 tx=0x7f6608019040 comp rx=0 tx=0).stop 2023-11-16T16:47:40.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.028+0000 7f661deb1700 1 -- 172.21.15.93:0/1845083792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6618100ee0 msgr2=0x7f661810ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:40.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.028+0000 7f661deb1700 1 --2- 172.21.15.93:0/1845083792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6618100ee0 0x7f661810ce50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f660c00ec70 tx=0x7f660c00ef80 comp rx=0 tx=0).stop 2023-11-16T16:47:40.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.028+0000 7f6617fff700 1 -- 172.21.15.93:0/1845083792 reap_dead start 2023-11-16T16:47:40.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.028+0000 7f661deb1700 1 -- 172.21.15.93:0/1845083792 shutdown_connections 2023-11-16T16:47:40.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.028+0000 7f661deb1700 1 -- 172.21.15.93:0/1845083792 >> 172.21.15.93:0/1845083792 conn(0x7f66180fbdc0 msgr2=0x7f66180fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:40.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.029+0000 7f661deb1700 1 -- 172.21.15.93:0/1845083792 shutdown_connections 2023-11-16T16:47:40.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.029+0000 7f661deb1700 1 -- 172.21.15.93:0/1845083792 wait complete. 2023-11-16T16:47:40.058 INFO:teuthology.orchestra.run.smithi093.stdout:287762808867 2023-11-16T16:47:40.058 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-11-16T16:47:40.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:39 smithi125 ceph-mon[131076]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:47:40.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.157+0000 7f15e8409700 1 -- 172.21.15.93:0/3547562079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e00ff330 msgr2=0x7f15e00ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:40.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.157+0000 7f15e8409700 1 --2- 172.21.15.93:0/3547562079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e00ff330 0x7f15e00ff700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009990 tx=0x7f15dc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:40.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.157+0000 7f15e8409700 1 -- 172.21.15.93:0/3547562079 shutdown_connections 2023-11-16T16:47:40.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.157+0000 7f15e8409700 1 --2- 172.21.15.93:0/3547562079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e0104660 0x7f15e0104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:40.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.157+0000 7f15e8409700 1 --2- 172.21.15.93:0/3547562079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e00ff330 0x7f15e00ff700 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009990 tx=0x7f15dc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:40.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.157+0000 7f15e8409700 1 --2- 172.21.15.93:0/3547562079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15e00ffc40 0x7f15e0104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:40.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.157+0000 7f15e8409700 1 -- 172.21.15.93:0/3547562079 >> 172.21.15.93:0/3547562079 conn(0x7f15e00770a0 msgr2=0x7f15e00774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:40.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.158+0000 7f15e8409700 1 -- 172.21.15.93:0/3547562079 shutdown_connections 2023-11-16T16:47:40.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.158+0000 7f15e8409700 1 -- 172.21.15.93:0/3547562079 wait complete. 2023-11-16T16:47:40.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.159+0000 7f15e8409700 1 Processor -- start 2023-11-16T16:47:40.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.160+0000 7f15e8409700 1 -- start start 2023-11-16T16:47:40.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.160+0000 7f15e8409700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15e00ffc40 0x7f15e01132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.161+0000 7f15e8409700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e0104660 0x7f15e0113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.161+0000 7f15e8409700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e010c2e0 0x7f15e010c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.161+0000 7f15e59a4700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e0104660 0x7f15e0113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.161+0000 7f15e59a4700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e0104660 0x7f15e0113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:56018/0 (socket says 172.21.15.93:56018) 2023-11-16T16:47:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.161+0000 7f15e59a4700 1 -- 172.21.15.93:0/3478905829 learned_addr learned my addr 172.21.15.93:0/3478905829 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.161+0000 7f15e69a6700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e010c2e0 0x7f15e010c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.161+0000 7f15e8409700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e010ce90 con 0x7f15e00ffc40 2023-11-16T16:47:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.162+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e010d000 con 0x7f15e0104660 2023-11-16T16:47:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.162+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e0110e60 con 0x7f15e010c2e0 2023-11-16T16:47:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.162+0000 7f15e69a6700 1 -- 172.21.15.93:0/3478905829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e0104660 msgr2=0x7f15e0113840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:40.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.162+0000 7f15e69a6700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e0104660 0x7f15e0113840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:40.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.162+0000 7f15e69a6700 1 -- 172.21.15.93:0/3478905829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15e00ffc40 msgr2=0x7f15e01132c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.161+0000 7f15e61a5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15e00ffc40 0x7f15e01132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.163+0000 7f15e69a6700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15e00ffc40 0x7f15e01132c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.163+0000 7f15e69a6700 1 -- 172.21.15.93:0/3478905829 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d8009c40 con 0x7f15e010c2e0 2023-11-16T16:47:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.163+0000 7f15e61a5700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15e00ffc40 0x7f15e01132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.166+0000 7f15e69a6700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e010c2e0 0x7f15e010c780 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15d800efb0 tx=0x7f15d800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.166+0000 7f15d37fe700 1 -- 172.21.15.93:0/3478905829 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d800ca50 con 0x7f15e010c2e0 2023-11-16T16:47:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.166+0000 7f15d37fe700 1 -- 172.21.15.93:0/3478905829 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15d800f040 con 0x7f15e010c2e0 2023-11-16T16:47:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.166+0000 7f15d37fe700 1 -- 172.21.15.93:0/3478905829 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d8004660 con 0x7f15e010c2e0 2023-11-16T16:47:40.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.167+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15dc0095f0 con 0x7f15e010c2e0 2023-11-16T16:47:40.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.167+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e0116900 con 0x7f15e010c2e0 2023-11-16T16:47:40.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.168+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15e0103cd0 con 0x7f15e010c2e0 2023-11-16T16:47:40.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.172+0000 7f15d37fe700 1 -- 172.21.15.93:0/3478905829 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f15d80047c0 con 0x7f15e010c2e0 2023-11-16T16:47:40.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.173+0000 7f15d37fe700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15cc0a1260 0x7f15cc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:40.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.174+0000 7f15d37fe700 1 -- 172.21.15.93:0/3478905829 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f15d80ccf20 con 0x7f15e010c2e0 2023-11-16T16:47:40.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.175+0000 7f15d37fe700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083] conn(0x7f15cc0ac1f0 0x7f15cc0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:40.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.175+0000 7f15e61a5700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15cc0a1260 0x7f15cc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:40.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.175+0000 7f15d37fe700 1 -- 172.21.15.93:0/3478905829 --> [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15cc0aed50 con 0x7f15cc0ac1f0 2023-11-16T16:47:40.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.175+0000 7f15d37fe700 1 -- 172.21.15.93:0/3478905829 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15d80cd330 con 0x7f15e010c2e0 2023-11-16T16:47:40.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.175+0000 7f15e59a4700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083] conn(0x7f15cc0ac1f0 0x7f15cc0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:40.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.176+0000 7f15e61a5700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15cc0a1260 0x7f15cc0a3710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b2e0 tx=0x7f15dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:40.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.176+0000 7f15e59a4700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083] conn(0x7f15cc0ac1f0 0x7f15cc0ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:40.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.177+0000 7f15d37fe700 1 -- 172.21.15.93:0/3478905829 <== osd.8 v2:172.21.15.159:6816/1960504083 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f15cc0aed50 con 0x7f15cc0ac1f0 2023-11-16T16:47:40.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.208+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 --> [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f15e0116770 con 0x7f15cc0ac1f0 2023-11-16T16:47:40.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.209+0000 7f15d37fe700 1 -- 172.21.15.93:0/3478905829 <== osd.8 v2:172.21.15.159:6816/1960504083 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f15e0116770 con 0x7f15cc0ac1f0 2023-11-16T16:47:40.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.211+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 >> [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083] conn(0x7f15cc0ac1f0 msgr2=0x7f15cc0ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:40.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.211+0000 7f15e8409700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083] conn(0x7f15cc0ac1f0 0x7f15cc0ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:40.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.212+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15cc0a1260 msgr2=0x7f15cc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:40.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.212+0000 7f15e8409700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15cc0a1260 0x7f15cc0a3710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00b2e0 tx=0x7f15dc005fb0 comp rx=0 tx=0).stop 2023-11-16T16:47:40.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.212+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e010c2e0 msgr2=0x7f15e010c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:40.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.212+0000 7f15e8409700 1 --2- 172.21.15.93:0/3478905829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e010c2e0 0x7f15e010c780 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15d800efb0 tx=0x7f15d800c3a0 comp rx=0 tx=0).stop 2023-11-16T16:47:40.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.212+0000 7f15e69a6700 1 -- 172.21.15.93:0/3478905829 reap_dead start 2023-11-16T16:47:40.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.212+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 shutdown_connections 2023-11-16T16:47:40.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.212+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 >> 172.21.15.93:0/3478905829 conn(0x7f15e00770a0 msgr2=0x7f15e01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:40.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.212+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 shutdown_connections 2023-11-16T16:47:40.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:40.213+0000 7f15e8409700 1 -- 172.21.15.93:0/3478905829 wait complete. 2023-11-16T16:47:40.811 INFO:teuthology.orchestra.run.smithi093.stdout:141733920825 2023-11-16T16:47:40.811 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-11-16T16:47:41.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.341+0000 7f6e88ebd700 1 -- 172.21.15.93:0/1112622699 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e84100d10 msgr2=0x7f6e84104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:41.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.341+0000 7f6e88ebd700 1 --2- 172.21.15.93:0/1112622699 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e84100d10 0x7f6e84104ce0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e780099c0 tx=0x7f6e78009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:41.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.341+0000 7f6e88ebd700 1 -- 172.21.15.93:0/1112622699 shutdown_connections 2023-11-16T16:47:41.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.341+0000 7f6e88ebd700 1 --2- 172.21.15.93:0/1112622699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e84105410 0x7f6e8410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:41.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.341+0000 7f6e88ebd700 1 --2- 172.21.15.93:0/1112622699 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e84100370 0x7f6e84100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:41.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.341+0000 7f6e88ebd700 1 --2- 172.21.15.93:0/1112622699 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e84100d10 0x7f6e84104ce0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:41.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.341+0000 7f6e88ebd700 1 -- 172.21.15.93:0/1112622699 >> 172.21.15.93:0/1112622699 conn(0x7f6e840fbbd0 msgr2=0x7f6e840fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:41.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.341+0000 7f6e88ebd700 1 -- 172.21.15.93:0/1112622699 shutdown_connections 2023-11-16T16:47:41.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.342+0000 7f6e88ebd700 1 -- 172.21.15.93:0/1112622699 wait complete. 2023-11-16T16:47:41.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.342+0000 7f6e88ebd700 1 Processor -- start 2023-11-16T16:47:41.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.343+0000 7f6e88ebd700 1 -- start start 2023-11-16T16:47:41.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.343+0000 7f6e88ebd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e84100370 0x7f6e84196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:41.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.343+0000 7f6e88ebd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e84100d10 0x7f6e841967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:41.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.343+0000 7f6e88ebd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e84105410 0x7f6e8419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:41.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.343+0000 7f6e88ebd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8419b1b0 con 0x7f6e84100370 2023-11-16T16:47:41.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.343+0000 7f6e88ebd700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8419b320 con 0x7f6e84100d10 2023-11-16T16:47:41.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.343+0000 7f6e88ebd700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8419b4e0 con 0x7f6e84105410 2023-11-16T16:47:41.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.343+0000 7f6e837fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e84100370 0x7f6e84196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:41.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.343+0000 7f6e837fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e84100370 0x7f6e84196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46382/0 (socket says 172.21.15.93:46382) 2023-11-16T16:47:41.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.343+0000 7f6e837fe700 1 -- 172.21.15.93:0/2965520220 learned_addr learned my addr 172.21.15.93:0/2965520220 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:41.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.343+0000 7f6e82ffd700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e84100d10 0x7f6e841967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:41.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.343+0000 7f6e83fff700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e84105410 0x7f6e8419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:41.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.344+0000 7f6e837fe700 1 -- 172.21.15.93:0/2965520220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e84105410 msgr2=0x7f6e8419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:41.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.344+0000 7f6e837fe700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e84105410 0x7f6e8419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:41.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.344+0000 7f6e837fe700 1 -- 172.21.15.93:0/2965520220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e84100d10 msgr2=0x7f6e841967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:41.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.344+0000 7f6e837fe700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e84100d10 0x7f6e841967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:41.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.344+0000 7f6e837fe700 1 -- 172.21.15.93:0/2965520220 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e780095f0 con 0x7f6e84100370 2023-11-16T16:47:41.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.344+0000 7f6e83fff700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e84105410 0x7f6e8419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:41.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.344+0000 7f6e837fe700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e84100370 0x7f6e84196290 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400e9a0 tx=0x7f6e7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:41.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.344+0000 7f6e80ff9700 1 -- 172.21.15.93:0/2965520220 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74009790 con 0x7f6e84100370 2023-11-16T16:47:41.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.344+0000 7f6e80ff9700 1 -- 172.21.15.93:0/2965520220 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e7400cbd0 con 0x7f6e84100370 2023-11-16T16:47:41.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.344+0000 7f6e80ff9700 1 -- 172.21.15.93:0/2965520220 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74018680 con 0x7f6e84100370 2023-11-16T16:47:41.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.344+0000 7f6e88ebd700 1 -- 172.21.15.93:0/2965520220 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e84110140 con 0x7f6e84100370 2023-11-16T16:47:41.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.344+0000 7f6e88ebd700 1 -- 172.21.15.93:0/2965520220 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8410ffb0 con 0x7f6e84100370 2023-11-16T16:47:41.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.345+0000 7f6e88ebd700 1 -- 172.21.15.93:0/2965520220 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e840fd0f0 con 0x7f6e84100370 2023-11-16T16:47:41.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.348+0000 7f6e80ff9700 1 -- 172.21.15.93:0/2965520220 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6e7400cd40 con 0x7f6e84100370 2023-11-16T16:47:41.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.349+0000 7f6e80ff9700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6e6c0a1260 0x7f6e6c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:41.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.349+0000 7f6e80ff9700 1 -- 172.21.15.93:0/2965520220 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6e74014070 con 0x7f6e84100370 2023-11-16T16:47:41.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.349+0000 7f6e82ffd700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6e6c0a1260 0x7f6e6c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:41.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.349+0000 7f6e80ff9700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115] conn(0x7f6e6c0ac1f0 0x7f6e6c0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:41.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.349+0000 7f6e80ff9700 1 -- 172.21.15.93:0/2965520220 --> [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e6c0aed50 con 0x7f6e6c0ac1f0 2023-11-16T16:47:41.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.349+0000 7f6e80ff9700 1 -- 172.21.15.93:0/2965520220 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e740cd720 con 0x7f6e84100370 2023-11-16T16:47:41.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.350+0000 7f6e83fff700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115] conn(0x7f6e6c0ac1f0 0x7f6e6c0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:41.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.350+0000 7f6e82ffd700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6e6c0a1260 0x7f6e6c0a3710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009990 tx=0x7f6e78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:41.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.350+0000 7f6e83fff700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115] conn(0x7f6e6c0ac1f0 0x7f6e6c0ae680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:41.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.351+0000 7f6e80ff9700 1 -- 172.21.15.93:0/2965520220 <== osd.1 v2:172.21.15.125:6800/2084419115 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6e6c0aed50 con 0x7f6e6c0ac1f0 2023-11-16T16:47:41.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.367+0000 7f6e88ebd700 1 -- 172.21.15.93:0/2965520220 --> [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6e841102d0 con 0x7f6e6c0ac1f0 2023-11-16T16:47:41.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.368+0000 7f6e80ff9700 1 -- 172.21.15.93:0/2965520220 <== osd.1 v2:172.21.15.125:6800/2084419115 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6e841102d0 con 0x7f6e6c0ac1f0 2023-11-16T16:47:41.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.368+0000 7f6e88ebd700 1 -- 172.21.15.93:0/2965520220 >> [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115] conn(0x7f6e6c0ac1f0 msgr2=0x7f6e6c0ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:41.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.368+0000 7f6e88ebd700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115] conn(0x7f6e6c0ac1f0 0x7f6e6c0ae680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:41.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.368+0000 7f6e88ebd700 1 -- 172.21.15.93:0/2965520220 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6e6c0a1260 msgr2=0x7f6e6c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:41.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.368+0000 7f6e88ebd700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6e6c0a1260 0x7f6e6c0a3710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009990 tx=0x7f6e78005fb0 comp rx=0 tx=0).stop 2023-11-16T16:47:41.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.368+0000 7f6e88ebd700 1 -- 172.21.15.93:0/2965520220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e84100370 msgr2=0x7f6e84196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:41.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.368+0000 7f6e88ebd700 1 --2- 172.21.15.93:0/2965520220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e84100370 0x7f6e84196290 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400e9a0 tx=0x7f6e7400ecb0 comp rx=0 tx=0).stop 2023-11-16T16:47:41.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.368+0000 7f6e83fff700 1 -- 172.21.15.93:0/2965520220 reap_dead start 2023-11-16T16:47:41.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.368+0000 7f6e88ebd700 1 -- 172.21.15.93:0/2965520220 shutdown_connections 2023-11-16T16:47:41.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.368+0000 7f6e88ebd700 1 -- 172.21.15.93:0/2965520220 >> 172.21.15.93:0/2965520220 conn(0x7f6e840fbbd0 msgr2=0x7f6e840fc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:41.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.368+0000 7f6e88ebd700 1 -- 172.21.15.93:0/2965520220 shutdown_connections 2023-11-16T16:47:41.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:41.369+0000 7f6e88ebd700 1 -- 172.21.15.93:0/2965520220 wait complete. 2023-11-16T16:47:41.421 INFO:teuthology.orchestra.run.smithi093.stdout:339302416408 2023-11-16T16:47:41.422 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-11-16T16:47:41.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:41 smithi093 ceph-mon[97863]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-16T16:47:41.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:41 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:41.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:41 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:41.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:41 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:41.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:41 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:41.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:41 smithi125 ceph-mon[131076]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-16T16:47:41.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:41 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:41.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:41 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:41.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:41 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:41.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:41 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:41.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:41 smithi159 ceph-mon[131528]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-16T16:47:41.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:41 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:41.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:41 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:41.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:41 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:41.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:41 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:42.086 INFO:teuthology.orchestra.run.smithi093.stdout:253403070504 2023-11-16T16:47:42.086 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-11-16T16:47:42.221 INFO:teuthology.orchestra.run.smithi093.stdout:55834574920 2023-11-16T16:47:42.222 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-11-16T16:47:42.342 INFO:teuthology.orchestra.run.smithi093.stdout:77309411397 2023-11-16T16:47:42.342 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-11-16T16:47:42.384 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:42.696 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:42.697 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:42.702 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:42.734 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:42.842 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:43.004 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:43.006 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:43.208 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:43.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:43 smithi125 ceph-mon[131076]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:47:43.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:43 smithi093 ceph-mon[97863]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:47:43.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:43 smithi159 ceph-mon[131528]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:47:45.669 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:45 smithi093 ceph-mon[97863]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-16T16:47:45.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:45 smithi125 ceph-mon[131076]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-16T16:47:45.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:45 smithi159 ceph-mon[131528]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-16T16:47:46.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.116+0000 7f3aeb8de700 1 -- 172.21.15.93:0/1202629554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ae410a890 msgr2=0x7f3ae410ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:46.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.116+0000 7f3aeb8de700 1 --2- 172.21.15.93:0/1202629554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ae410a890 0x7f3ae410ccd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009f90 tx=0x7f3adc00b330 comp rx=0 tx=0).stop 2023-11-16T16:47:46.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.117+0000 7f3aeb8de700 1 -- 172.21.15.93:0/1202629554 shutdown_connections 2023-11-16T16:47:46.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.117+0000 7f3aeb8de700 1 --2- 172.21.15.93:0/1202629554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ae410a890 0x7f3ae410ccd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:46.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.117+0000 7f3aeb8de700 1 --2- 172.21.15.93:0/1202629554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae4101540 0x7f3ae4101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:46.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.117+0000 7f3aeb8de700 1 --2- 172.21.15.93:0/1202629554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4101e50 0x7f3ae410a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:46.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.117+0000 7f3aeb8de700 1 -- 172.21.15.93:0/1202629554 >> 172.21.15.93:0/1202629554 conn(0x7f3ae40770a0 msgr2=0x7f3ae40774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:46.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.117+0000 7f3aeb8de700 1 -- 172.21.15.93:0/1202629554 shutdown_connections 2023-11-16T16:47:46.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.117+0000 7f3aeb8de700 1 -- 172.21.15.93:0/1202629554 wait complete. 2023-11-16T16:47:46.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.119+0000 7f3aeb8de700 1 Processor -- start 2023-11-16T16:47:46.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.119+0000 7f3aeb8de700 1 -- start start 2023-11-16T16:47:46.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.119+0000 7f3aeb8de700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4101540 0x7f3ae4199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:46.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.119+0000 7f3aeb8de700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae4101e50 0x7f3ae41995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:46.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.119+0000 7f3aeb8de700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ae410a890 0x7f3ae4109080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:46.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.119+0000 7f3aeb8de700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae4109850 con 0x7f3ae410a890 2023-11-16T16:47:46.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.119+0000 7f3aeb8de700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41099c0 con 0x7f3ae4101540 2023-11-16T16:47:46.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.119+0000 7f3aeb8de700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae4109b80 con 0x7f3ae4101e50 2023-11-16T16:47:46.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.120+0000 7f3ae967a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4101540 0x7f3ae4199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:46.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.120+0000 7f3ae9e7b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ae410a890 0x7f3ae4109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:46.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.120+0000 7f3ae9e7b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ae410a890 0x7f3ae4109080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46396/0 (socket says 172.21.15.93:46396) 2023-11-16T16:47:46.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.120+0000 7f3ae9e7b700 1 -- 172.21.15.93:0/2357522976 learned_addr learned my addr 172.21.15.93:0/2357522976 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:46.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.120+0000 7f3ae8e79700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae4101e50 0x7f3ae41995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:46.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.121+0000 7f3ae9e7b700 1 -- 172.21.15.93:0/2357522976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae4101e50 msgr2=0x7f3ae41995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:46.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.121+0000 7f3ae9e7b700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae4101e50 0x7f3ae41995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:46.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.121+0000 7f3ae9e7b700 1 -- 172.21.15.93:0/2357522976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4101540 msgr2=0x7f3ae4199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:46.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.121+0000 7f3ae9e7b700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4101540 0x7f3ae4199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:46.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.121+0000 7f3ae9e7b700 1 -- 172.21.15.93:0/2357522976 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0009520 con 0x7f3ae410a890 2023-11-16T16:47:46.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.121+0000 7f3ae9e7b700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ae410a890 0x7f3ae4109080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009010 tx=0x7f3adc004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:46.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.122+0000 7f3ada7fc700 1 -- 172.21.15.93:0/2357522976 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc00e040 con 0x7f3ae410a890 2023-11-16T16:47:46.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.122+0000 7f3ada7fc700 1 -- 172.21.15.93:0/2357522976 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3adc00bbf0 con 0x7f3ae410a890 2023-11-16T16:47:46.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.122+0000 7f3aeb8de700 1 -- 172.21.15.93:0/2357522976 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad40095f0 con 0x7f3ae410a890 2023-11-16T16:47:46.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.122+0000 7f3ada7fc700 1 -- 172.21.15.93:0/2357522976 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc0052a0 con 0x7f3ae410a890 2023-11-16T16:47:46.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.122+0000 7f3aeb8de700 1 -- 172.21.15.93:0/2357522976 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3adc009c40 con 0x7f3ae410a890 2023-11-16T16:47:46.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.124+0000 7f3aeb8de700 1 -- 172.21.15.93:0/2357522976 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae4066ea0 con 0x7f3ae410a890 2023-11-16T16:47:46.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.126+0000 7f3ada7fc700 1 -- 172.21.15.93:0/2357522976 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3adc01d470 con 0x7f3ae410a890 2023-11-16T16:47:46.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.128+0000 7f3ada7fc700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3ad00a1260 0x7f3ad00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:46.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.129+0000 7f3ada7fc700 1 -- 172.21.15.93:0/2357522976 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3adc0cd660 con 0x7f3ae410a890 2023-11-16T16:47:46.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.129+0000 7f3ae967a700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3ad00a1260 0x7f3ad00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:46.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.130+0000 7f3ae967a700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3ad00a1260 0x7f3ad00a3710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009d80 tx=0x7f3ae0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:46.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.132+0000 7f3ada7fc700 1 -- 172.21.15.93:0/2357522976 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3adc0969f0 con 0x7f3ae410a890 2023-11-16T16:47:46.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.382+0000 7f3aeb8de700 1 -- 172.21.15.93:0/2357522976 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f3ae419d0d0 con 0x7f3ae410a890 2023-11-16T16:47:46.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.382+0000 7f3ada7fc700 1 -- 172.21.15.93:0/2357522976 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f3adc0232c0 con 0x7f3ae410a890 2023-11-16T16:47:46.384 INFO:teuthology.orchestra.run.smithi093.stdout:309237645343 2023-11-16T16:47:46.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.385+0000 7f3aeb8de700 1 -- 172.21.15.93:0/2357522976 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3ad00a1260 msgr2=0x7f3ad00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:46.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.385+0000 7f3aeb8de700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3ad00a1260 0x7f3ad00a3710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009d80 tx=0x7f3ae0009260 comp rx=0 tx=0).stop 2023-11-16T16:47:46.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.385+0000 7f3aeb8de700 1 -- 172.21.15.93:0/2357522976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ae410a890 msgr2=0x7f3ae4109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:46.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.385+0000 7f3aeb8de700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ae410a890 0x7f3ae4109080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009010 tx=0x7f3adc004b00 comp rx=0 tx=0).stop 2023-11-16T16:47:46.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.385+0000 7f3aeb8de700 1 -- 172.21.15.93:0/2357522976 shutdown_connections 2023-11-16T16:47:46.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.385+0000 7f3aeb8de700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3ad00a1260 0x7f3ad00a3710 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:46.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.385+0000 7f3aeb8de700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ae410a890 0x7f3ae4109080 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:46.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.385+0000 7f3aeb8de700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4101540 0x7f3ae4199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:46.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.385+0000 7f3aeb8de700 1 --2- 172.21.15.93:0/2357522976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae4101e50 0x7f3ae41995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:46.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.385+0000 7f3aeb8de700 1 -- 172.21.15.93:0/2357522976 >> 172.21.15.93:0/2357522976 conn(0x7f3ae40770a0 msgr2=0x7f3ae40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:46.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.385+0000 7f3aeb8de700 1 -- 172.21.15.93:0/2357522976 shutdown_connections 2023-11-16T16:47:46.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:46.385+0000 7f3aeb8de700 1 -- 172.21.15.93:0/2357522976 wait complete. 2023-11-16T16:47:46.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:46 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2357522976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-16T16:47:46.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:46 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2357522976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-16T16:47:46.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:46 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2357522976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-16T16:47:47.716 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:47 smithi093 ceph-mon[97863]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-16T16:47:47.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:47 smithi125 ceph-mon[131076]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-16T16:47:47.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:47 smithi159 ceph-mon[131528]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-16T16:47:48.166 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.998+0000 7f78ceacd700 1 -- 172.21.15.93:0/1430074360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78c8100ee0 msgr2=0x7f78c8104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:48.166 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.998+0000 7f78ceacd700 1 --2- 172.21.15.93:0/1430074360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78c8100ee0 0x7f78c8104eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009920 tx=0x7f78b4009c30 comp rx=0 tx=0).stop 2023-11-16T16:47:48.166 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.998+0000 7f78ceacd700 1 -- 172.21.15.93:0/1430074360 shutdown_connections 2023-11-16T16:47:48.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.998+0000 7f78ceacd700 1 --2- 172.21.15.93:0/1430074360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78c8100540 0x7f78c8100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.998+0000 7f78ceacd700 1 --2- 172.21.15.93:0/1430074360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78c81055e0 0x7f78c810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.998+0000 7f78ceacd700 1 --2- 172.21.15.93:0/1430074360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78c8100ee0 0x7f78c8104eb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.998+0000 7f78ceacd700 1 -- 172.21.15.93:0/1430074360 >> 172.21.15.93:0/1430074360 conn(0x7f78c80fbdc0 msgr2=0x7f78c80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:48.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.998+0000 7f78ceacd700 1 -- 172.21.15.93:0/1430074360 shutdown_connections 2023-11-16T16:47:48.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.998+0000 7f78ceacd700 1 -- 172.21.15.93:0/1430074360 wait complete. 2023-11-16T16:47:48.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.999+0000 7f78ceacd700 1 Processor -- start 2023-11-16T16:47:48.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.999+0000 7f78ceacd700 1 -- start start 2023-11-16T16:47:48.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.999+0000 7f78ceacd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78c8100540 0x7f78c8196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:48.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.999+0000 7f78ceacd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78c8100ee0 0x7f78c8196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:48.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.999+0000 7f78ceacd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78c81055e0 0x7f78c819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:48.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:47.999+0000 7f78ceacd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c819b400 con 0x7f78c81055e0 2023-11-16T16:47:48.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78ceacd700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c819b570 con 0x7f78c8100540 2023-11-16T16:47:48.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78ceacd700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c819b730 con 0x7f78c8100ee0 2023-11-16T16:47:48.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78cc869700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78c8100540 0x7f78c8196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:48.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78cd06a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78c81055e0 0x7f78c819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:48.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78cc869700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78c8100540 0x7f78c8196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:47746/0 (socket says 172.21.15.93:47746) 2023-11-16T16:47:48.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78cc869700 1 -- 172.21.15.93:0/609541701 learned_addr learned my addr 172.21.15.93:0/609541701 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:48.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78c7fff700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78c8100ee0 0x7f78c8196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:48.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78cd06a700 1 -- 172.21.15.93:0/609541701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78c8100ee0 msgr2=0x7f78c8196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:48.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78cd06a700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78c8100ee0 0x7f78c8196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78cd06a700 1 -- 172.21.15.93:0/609541701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78c8100540 msgr2=0x7f78c8196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:48.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78cd06a700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78c8100540 0x7f78c8196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78cd06a700 1 -- 172.21.15.93:0/609541701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc0095f0 con 0x7f78c81055e0 2023-11-16T16:47:48.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.000+0000 7f78cd06a700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78c81055e0 0x7f78c819adf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f78c000efc0 tx=0x7f78c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:48.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.001+0000 7f78c5ffb700 1 -- 172.21.15.93:0/609541701 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0018070 con 0x7f78c81055e0 2023-11-16T16:47:48.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.001+0000 7f78c5ffb700 1 -- 172.21.15.93:0/609541701 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78c000f040 con 0x7f78c81055e0 2023-11-16T16:47:48.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.001+0000 7f78ceacd700 1 -- 172.21.15.93:0/609541701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b4009520 con 0x7f78c81055e0 2023-11-16T16:47:48.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.001+0000 7f78c5ffb700 1 -- 172.21.15.93:0/609541701 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c000cd40 con 0x7f78c81055e0 2023-11-16T16:47:48.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.001+0000 7f78ceacd700 1 -- 172.21.15.93:0/609541701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c8110360 con 0x7f78c81055e0 2023-11-16T16:47:48.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.002+0000 7f78ceacd700 1 -- 172.21.15.93:0/609541701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c81101d0 con 0x7f78c81055e0 2023-11-16T16:47:48.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.004+0000 7f78c5ffb700 1 -- 172.21.15.93:0/609541701 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f78c0003a00 con 0x7f78c81055e0 2023-11-16T16:47:48.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.005+0000 7f78c5ffb700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f78b80a1260 0x7f78b80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:48.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.005+0000 7f78c5ffb700 1 -- 172.21.15.93:0/609541701 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f78c00cc8f0 con 0x7f78c81055e0 2023-11-16T16:47:48.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.005+0000 7f78cc869700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f78b80a1260 0x7f78b80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:48.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.006+0000 7f78cc869700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f78b80a1260 0x7f78b80a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f78bc005d10 tx=0x7f78bc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:48.170 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.009+0000 7f78c5ffb700 1 -- 172.21.15.93:0/609541701 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f78c0095c10 con 0x7f78c81055e0 2023-11-16T16:47:48.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.207+0000 7fbb222fc700 1 -- 172.21.15.93:0/3093669857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c100ef0 msgr2=0x7fbb1c104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:48.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.207+0000 7fbb222fc700 1 --2- 172.21.15.93:0/3093669857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c104ec0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb100099c0 tx=0x7fbb10009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.207+0000 7fbb222fc700 1 -- 172.21.15.93:0/3093669857 shutdown_connections 2023-11-16T16:47:48.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.207+0000 7fbb222fc700 1 --2- 172.21.15.93:0/3093669857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c1055f0 0x7fbb1c10d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.207+0000 7fbb222fc700 1 --2- 172.21.15.93:0/3093669857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb1c100550 0x7fbb1c100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.207+0000 7fbb222fc700 1 --2- 172.21.15.93:0/3093669857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c104ec0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.207+0000 7fbb222fc700 1 -- 172.21.15.93:0/3093669857 >> 172.21.15.93:0/3093669857 conn(0x7fbb1c0fbdf0 msgr2=0x7fbb1c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:48.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.207+0000 7fbb222fc700 1 -- 172.21.15.93:0/3093669857 shutdown_connections 2023-11-16T16:47:48.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.207+0000 7fbb222fc700 1 -- 172.21.15.93:0/3093669857 wait complete. 2023-11-16T16:47:48.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.209+0000 7fbb222fc700 1 Processor -- start 2023-11-16T16:47:48.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.209+0000 7fbb222fc700 1 -- start start 2023-11-16T16:47:48.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.209+0000 7fbb222fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c100550 0x7fbb1c1071f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:48.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.209+0000 7fbb222fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c107730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:48.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.209+0000 7fbb222fc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb1c1055f0 0x7fbb1c107c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:48.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.209+0000 7fbb222fc700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1a8d00 con 0x7fbb1c100550 2023-11-16T16:47:48.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.210+0000 7fbb222fc700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1a8e70 con 0x7fbb1c1055f0 2023-11-16T16:47:48.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.210+0000 7fbb222fc700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1a9030 con 0x7fbb1c100ef0 2023-11-16T16:47:48.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.210+0000 7fbb20899700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb1c1055f0 0x7fbb1c107c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:48.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.210+0000 7fbb20899700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb1c1055f0 0x7fbb1c107c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:47768/0 (socket says 172.21.15.93:47768) 2023-11-16T16:47:48.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.210+0000 7fbb20899700 1 -- 172.21.15.93:0/1041686043 learned_addr learned my addr 172.21.15.93:0/1041686043 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:48.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.210+0000 7fbb1b7fe700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:48.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.210+0000 7fbb1bfff700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c100550 0x7fbb1c1071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:48.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.211+0000 7fbb1bfff700 1 -- 172.21.15.93:0/1041686043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c100ef0 msgr2=0x7fbb1c107730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:48.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.211+0000 7fbb1bfff700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c107730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.211+0000 7fbb1bfff700 1 -- 172.21.15.93:0/1041686043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb1c1055f0 msgr2=0x7fbb1c107c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:48.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.211+0000 7fbb1bfff700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb1c1055f0 0x7fbb1c107c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.211+0000 7fbb1bfff700 1 -- 172.21.15.93:0/1041686043 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb14009c40 con 0x7fbb1c100550 2023-11-16T16:47:48.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.211+0000 7fbb20899700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb1c1055f0 0x7fbb1c107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:48.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.212+0000 7fbb1bfff700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c100550 0x7fbb1c1071f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00e9f0 tx=0x7fbb0c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:48.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.212+0000 7fbb197fa700 1 -- 172.21.15.93:0/1041686043 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00c9c0 con 0x7fbb1c100550 2023-11-16T16:47:48.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.212+0000 7fbb197fa700 1 -- 172.21.15.93:0/1041686043 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb0c010430 con 0x7fbb1c100550 2023-11-16T16:47:48.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.212+0000 7fbb222fc700 1 -- 172.21.15.93:0/1041686043 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb100095f0 con 0x7fbb1c100550 2023-11-16T16:47:48.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.212+0000 7fbb197fa700 1 -- 172.21.15.93:0/1041686043 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c018680 con 0x7fbb1c100550 2023-11-16T16:47:48.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.212+0000 7fbb222fc700 1 -- 172.21.15.93:0/1041686043 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c110370 con 0x7fbb1c100550 2023-11-16T16:47:48.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.214+0000 7fbb222fc700 1 -- 172.21.15.93:0/1041686043 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c1101e0 con 0x7fbb1c100550 2023-11-16T16:47:48.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.217+0000 7fbb197fa700 1 -- 172.21.15.93:0/1041686043 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbb0c010860 con 0x7fbb1c100550 2023-11-16T16:47:48.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.218+0000 7fbb197fa700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbb040a1260 0x7fbb040a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:48.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.220+0000 7fbb197fa700 1 -- 172.21.15.93:0/1041686043 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbb0c014070 con 0x7fbb1c100550 2023-11-16T16:47:48.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.220+0000 7fbb1b7fe700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbb040a1260 0x7fbb040a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:48.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.221+0000 7fbb1b7fe700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbb040a1260 0x7fbb040a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb100098e0 tx=0x7fbb10000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:48.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.227+0000 7fbb197fa700 1 -- 172.21.15.93:0/1041686043 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbb0c051730 con 0x7fbb1c100550 2023-11-16T16:47:48.269 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.268+0000 7f78ceacd700 1 -- 172.21.15.93:0/609541701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f78c81104f0 con 0x7f78c81055e0 2023-11-16T16:47:48.271 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.269+0000 7f78c5ffb700 1 -- 172.21.15.93:0/609541701 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f78c0099230 con 0x7f78c81055e0 2023-11-16T16:47:48.272 INFO:teuthology.orchestra.run.smithi093.stdout:34359738446 2023-11-16T16:47:48.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.273+0000 7f78ceacd700 1 -- 172.21.15.93:0/609541701 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f78b80a1260 msgr2=0x7f78b80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:48.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.273+0000 7f78ceacd700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f78b80a1260 0x7f78b80a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f78bc005d10 tx=0x7f78bc009310 comp rx=0 tx=0).stop 2023-11-16T16:47:48.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.273+0000 7f78ceacd700 1 -- 172.21.15.93:0/609541701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78c81055e0 msgr2=0x7f78c819adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:48.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.273+0000 7f78ceacd700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78c81055e0 0x7f78c819adf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f78c000efc0 tx=0x7f78c000c3a0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.273+0000 7f78ceacd700 1 -- 172.21.15.93:0/609541701 shutdown_connections 2023-11-16T16:47:48.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.273+0000 7f78ceacd700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f78b80a1260 0x7f78b80a3710 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.273+0000 7f78ceacd700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78c81055e0 0x7f78c819adf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.273+0000 7f78ceacd700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78c8100540 0x7f78c8196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.273+0000 7f78ceacd700 1 --2- 172.21.15.93:0/609541701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78c8100ee0 0x7f78c8196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.273+0000 7f78ceacd700 1 -- 172.21.15.93:0/609541701 >> 172.21.15.93:0/609541701 conn(0x7f78c80fbdc0 msgr2=0x7f78c80fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:48.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.277+0000 7f78ceacd700 1 -- 172.21.15.93:0/609541701 shutdown_connections 2023-11-16T16:47:48.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.277+0000 7f78ceacd700 1 -- 172.21.15.93:0/609541701 wait complete. 2023-11-16T16:47:48.289 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:48.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.501+0000 7fbb222fc700 1 -- 172.21.15.93:0/1041686043 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbb1c110500 con 0x7fbb1c100550 2023-11-16T16:47:48.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.502+0000 7fbb197fa700 1 -- 172.21.15.93:0/1041686043 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb0c096150 con 0x7fbb1c100550 2023-11-16T16:47:48.503 INFO:teuthology.orchestra.run.smithi093.stdout:197568495667 2023-11-16T16:47:48.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.504+0000 7fbb222fc700 1 -- 172.21.15.93:0/1041686043 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbb040a1260 msgr2=0x7fbb040a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:48.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.504+0000 7fbb222fc700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbb040a1260 0x7fbb040a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb100098e0 tx=0x7fbb10000bc0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.504+0000 7fbb222fc700 1 -- 172.21.15.93:0/1041686043 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c100550 msgr2=0x7fbb1c1071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:48.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.504+0000 7fbb222fc700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c100550 0x7fbb1c1071f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00e9f0 tx=0x7fbb0c00ed00 comp rx=0 tx=0).stop 2023-11-16T16:47:48.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.504+0000 7fbb222fc700 1 -- 172.21.15.93:0/1041686043 shutdown_connections 2023-11-16T16:47:48.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.504+0000 7fbb222fc700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbb040a1260 0x7fbb040a3710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.504+0000 7fbb222fc700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c100550 0x7fbb1c1071f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.504+0000 7fbb222fc700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb1c1055f0 0x7fbb1c107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.504+0000 7fbb222fc700 1 --2- 172.21.15.93:0/1041686043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:48.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.504+0000 7fbb222fc700 1 -- 172.21.15.93:0/1041686043 >> 172.21.15.93:0/1041686043 conn(0x7fbb1c0fbdf0 msgr2=0x7fbb1c0fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:48.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.504+0000 7fbb222fc700 1 -- 172.21.15.93:0/1041686043 shutdown_connections 2023-11-16T16:47:48.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:48.505+0000 7fbb222fc700 1 -- 172.21.15.93:0/1041686043 wait complete. 2023-11-16T16:47:48.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:48 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/609541701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-16T16:47:48.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:48 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1041686043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-16T16:47:48.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:48 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/609541701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-16T16:47:48.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:48 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1041686043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-16T16:47:49.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:48 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/609541701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-16T16:47:49.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:48 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1041686043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-16T16:47:49.158 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645341 got 309237645343 for osd.10 2023-11-16T16:47:49.158 DEBUG:teuthology.parallel:result is None 2023-11-16T16:47:49.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:49 smithi093 ceph-mon[97863]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-16T16:47:49.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.699+0000 7f7a16c71700 1 -- 172.21.15.93:0/4171652502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a10100540 msgr2=0x7f7a10100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:49.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.699+0000 7f7a16c71700 1 --2- 172.21.15.93:0/4171652502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a10100540 0x7f7a10100910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7a040099c0 tx=0x7f7a04009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:49.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.700+0000 7f7a16c71700 1 -- 172.21.15.93:0/4171652502 shutdown_connections 2023-11-16T16:47:49.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.700+0000 7f7a16c71700 1 --2- 172.21.15.93:0/4171652502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a10100540 0x7f7a10100910 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:49.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.700+0000 7f7a16c71700 1 --2- 172.21.15.93:0/4171652502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a101055e0 0x7f7a1010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:49.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.700+0000 7f7a16c71700 1 --2- 172.21.15.93:0/4171652502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a10100ee0 0x7f7a10104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:49.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.700+0000 7f7a16c71700 1 -- 172.21.15.93:0/4171652502 >> 172.21.15.93:0/4171652502 conn(0x7f7a100fbdc0 msgr2=0x7f7a100fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:49.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.700+0000 7f7a16c71700 1 -- 172.21.15.93:0/4171652502 shutdown_connections 2023-11-16T16:47:49.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.700+0000 7f7a16c71700 1 -- 172.21.15.93:0/4171652502 wait complete. 2023-11-16T16:47:49.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.701+0000 7f7a16c71700 1 Processor -- start 2023-11-16T16:47:49.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.701+0000 7f7a16c71700 1 -- start start 2023-11-16T16:47:49.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.701+0000 7f7a16c71700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a10100540 0x7f7a10072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:49.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.701+0000 7f7a16c71700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a10100ee0 0x7f7a10073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:49.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.701+0000 7f7a16c71700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a101055e0 0x7f7a100738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:49.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.701+0000 7f7a16c71700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a101a4840 con 0x7f7a10100ee0 2023-11-16T16:47:49.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.701+0000 7f7a16c71700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a101a49b0 con 0x7f7a101055e0 2023-11-16T16:47:49.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.701+0000 7f7a16c71700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a101a4b70 con 0x7f7a10100540 2023-11-16T16:47:49.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a0ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a10100ee0 0x7f7a10073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:49.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a0ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a10100ee0 0x7f7a10073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42436/0 (socket says 172.21.15.93:42436) 2023-11-16T16:47:49.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a0ffff700 1 -- 172.21.15.93:0/3896789299 learned_addr learned my addr 172.21.15.93:0/3896789299 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:49.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a14a0d700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a10100540 0x7f7a10072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:49.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a1520e700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a101055e0 0x7f7a100738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:49.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a0ffff700 1 -- 172.21.15.93:0/3896789299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a10100540 msgr2=0x7f7a10072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:49.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a0ffff700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a10100540 0x7f7a10072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:49.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a0ffff700 1 -- 172.21.15.93:0/3896789299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a101055e0 msgr2=0x7f7a100738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:49.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a0ffff700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a101055e0 0x7f7a100738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:49.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a0ffff700 1 -- 172.21.15.93:0/3896789299 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a040095f0 con 0x7f7a10100ee0 2023-11-16T16:47:49.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a1520e700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a101055e0 0x7f7a100738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:49.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a0ffff700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a10100ee0 0x7f7a10073330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00d980 tx=0x7f79fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:49.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.702+0000 7f7a0dffb700 1 -- 172.21.15.93:0/3896789299 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc009930 con 0x7f7a10100ee0 2023-11-16T16:47:49.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.703+0000 7f7a16c71700 1 -- 172.21.15.93:0/3896789299 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a10110370 con 0x7f7a10100ee0 2023-11-16T16:47:49.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.703+0000 7f7a0dffb700 1 -- 172.21.15.93:0/3896789299 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79fc010460 con 0x7f7a10100ee0 2023-11-16T16:47:49.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.703+0000 7f7a0dffb700 1 -- 172.21.15.93:0/3896789299 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc00f620 con 0x7f7a10100ee0 2023-11-16T16:47:49.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.703+0000 7f7a16c71700 1 -- 172.21.15.93:0/3896789299 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a101101e0 con 0x7f7a10100ee0 2023-11-16T16:47:49.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.704+0000 7f7a16c71700 1 -- 172.21.15.93:0/3896789299 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a10110500 con 0x7f7a10100ee0 2023-11-16T16:47:49.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.706+0000 7f7a0dffb700 1 -- 172.21.15.93:0/3896789299 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f79fc0108d0 con 0x7f7a10100ee0 2023-11-16T16:47:49.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.707+0000 7f7a0dffb700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7a000a11e0 0x7f7a000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:49.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.707+0000 7f7a0dffb700 1 -- 172.21.15.93:0/3896789299 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f79fc0cc8a0 con 0x7f7a10100ee0 2023-11-16T16:47:49.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.707+0000 7f7a14a0d700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7a000a11e0 0x7f7a000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:49.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.708+0000 7f7a14a0d700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7a000a11e0 0x7f7a000a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7a04009990 tx=0x7f7a04005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:49.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.711+0000 7f7a0dffb700 1 -- 172.21.15.93:0/3896789299 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f79fc095b30 con 0x7f7a10100ee0 2023-11-16T16:47:49.893 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738446 for osd.0 2023-11-16T16:47:49.893 DEBUG:teuthology.parallel:result is None 2023-11-16T16:47:49.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.935+0000 7f7a16c71700 1 -- 172.21.15.93:0/3896789299 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7a101a5670 con 0x7f7a10100ee0 2023-11-16T16:47:49.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.936+0000 7f7a0dffb700 1 -- 172.21.15.93:0/3896789299 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f79fc099150 con 0x7f7a10100ee0 2023-11-16T16:47:49.937 INFO:teuthology.orchestra.run.smithi093.stdout:163208757304 2023-11-16T16:47:49.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.938+0000 7f7a16c71700 1 -- 172.21.15.93:0/3896789299 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7a000a11e0 msgr2=0x7f7a000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:49.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.938+0000 7f7a16c71700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7a000a11e0 0x7f7a000a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7a04009990 tx=0x7f7a04005fb0 comp rx=0 tx=0).stop 2023-11-16T16:47:49.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.938+0000 7f7a16c71700 1 -- 172.21.15.93:0/3896789299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a10100ee0 msgr2=0x7f7a10073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:49.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.938+0000 7f7a16c71700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a10100ee0 0x7f7a10073330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00d980 tx=0x7f79fc00dc90 comp rx=0 tx=0).stop 2023-11-16T16:47:49.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.938+0000 7f7a16c71700 1 -- 172.21.15.93:0/3896789299 shutdown_connections 2023-11-16T16:47:49.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.938+0000 7f7a16c71700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7a000a11e0 0x7f7a000a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:49.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.938+0000 7f7a16c71700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a101055e0 0x7f7a100738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:49.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.938+0000 7f7a16c71700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a10100540 0x7f7a10072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:49.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.938+0000 7f7a16c71700 1 --2- 172.21.15.93:0/3896789299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a10100ee0 0x7f7a10073330 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:49.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.938+0000 7f7a16c71700 1 -- 172.21.15.93:0/3896789299 >> 172.21.15.93:0/3896789299 conn(0x7f7a100fbdc0 msgr2=0x7f7a100fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:49.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.939+0000 7f7a16c71700 1 -- 172.21.15.93:0/3896789299 shutdown_connections 2023-11-16T16:47:49.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.939+0000 7f7a16c71700 1 -- 172.21.15.93:0/3896789299 wait complete. 2023-11-16T16:47:49.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:49 smithi159 ceph-mon[131528]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-16T16:47:50.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:49 smithi125 ceph-mon[131076]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-16T16:47:50.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.989+0000 7f55a221b700 1 -- 172.21.15.93:0/1385821022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c1055f0 msgr2=0x7f559c10d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.989+0000 7f55a221b700 1 --2- 172.21.15.93:0/1385821022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c1055f0 0x7f559c10d170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f559400b0d0 tx=0x7f559400b3e0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.989+0000 7f55a221b700 1 -- 172.21.15.93:0/1385821022 shutdown_connections 2023-11-16T16:47:50.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.989+0000 7f55a221b700 1 --2- 172.21.15.93:0/1385821022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f559c100550 0x7f559c100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.989+0000 7f55a221b700 1 --2- 172.21.15.93:0/1385821022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c1055f0 0x7f559c10d170 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.989+0000 7f55a221b700 1 --2- 172.21.15.93:0/1385821022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f559c100ef0 0x7f559c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.989+0000 7f55a221b700 1 -- 172.21.15.93:0/1385821022 >> 172.21.15.93:0/1385821022 conn(0x7f559c0fbdf0 msgr2=0x7f559c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:50.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.989+0000 7f55a221b700 1 -- 172.21.15.93:0/1385821022 shutdown_connections 2023-11-16T16:47:50.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.989+0000 7f55a221b700 1 -- 172.21.15.93:0/1385821022 wait complete. 2023-11-16T16:47:50.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.991+0000 7f55a221b700 1 Processor -- start 2023-11-16T16:47:50.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.991+0000 7f55a221b700 1 -- start start 2023-11-16T16:47:50.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.992+0000 7f55a221b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f559c100550 0x7f559c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:50.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.992+0000 7f55a221b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c100ef0 0x7f559c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:50.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.992+0000 7f55a221b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f559c1055f0 0x7f559c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:50.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.992+0000 7f55a221b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c19d620 con 0x7f559c100550 2023-11-16T16:47:50.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.992+0000 7f55a221b700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c19d790 con 0x7f559c1055f0 2023-11-16T16:47:50.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.992+0000 7f55a221b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c19d950 con 0x7f559c100ef0 2023-11-16T16:47:50.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.992+0000 7f559b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f559c100550 0x7f559c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:50.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.992+0000 7f559b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f559c100550 0x7f559c198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42450/0 (socket says 172.21.15.93:42450) 2023-11-16T16:47:50.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.992+0000 7f559b7fe700 1 -- 172.21.15.93:0/2693987576 learned_addr learned my addr 172.21.15.93:0/2693987576 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:50.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.992+0000 7f559bfff700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f559c1055f0 0x7f559c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:50.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.992+0000 7f559affd700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c100ef0 0x7f559c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:50.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.993+0000 7f559bfff700 1 -- 172.21.15.93:0/2693987576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c100ef0 msgr2=0x7f559c198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.993+0000 7f559bfff700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c100ef0 0x7f559c198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.993+0000 7f559bfff700 1 -- 172.21.15.93:0/2693987576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f559c100550 msgr2=0x7f559c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.993+0000 7f559bfff700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f559c100550 0x7f559c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.993+0000 7f559bfff700 1 -- 172.21.15.93:0/2693987576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5594009d00 con 0x7f559c1055f0 2023-11-16T16:47:50.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.993+0000 7f559b7fe700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f559c100550 0x7f559c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:50.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.993+0000 7f559bfff700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f559c1055f0 0x7f559c19d010 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f559400b770 tx=0x7f5594007870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:50.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.993+0000 7f5598ff9700 1 -- 172.21.15.93:0/2693987576 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559400e040 con 0x7f559c1055f0 2023-11-16T16:47:50.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.993+0000 7f55a221b700 1 -- 172.21.15.93:0/2693987576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559c110330 con 0x7f559c1055f0 2023-11-16T16:47:50.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.993+0000 7f55a221b700 1 -- 172.21.15.93:0/2693987576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c1101a0 con 0x7f559c1055f0 2023-11-16T16:47:50.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.994+0000 7f5598ff9700 1 -- 172.21.15.93:0/2693987576 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5594009b80 con 0x7f559c1055f0 2023-11-16T16:47:50.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.994+0000 7f5598ff9700 1 -- 172.21.15.93:0/2693987576 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559401b680 con 0x7f559c1055f0 2023-11-16T16:47:50.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.995+0000 7f55a221b700 1 -- 172.21.15.93:0/2693987576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5588005320 con 0x7f559c1055f0 2023-11-16T16:47:50.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.998+0000 7f5598ff9700 1 -- 172.21.15.93:0/2693987576 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f559401d470 con 0x7f559c1055f0 2023-11-16T16:47:50.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.999+0000 7f5598ff9700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f55840a11e0 0x7f55840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:50.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:49.999+0000 7f5598ff9700 1 -- 172.21.15.93:0/2693987576 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f55940cddf0 con 0x7f559c1055f0 2023-11-16T16:47:50.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.000+0000 7f559b7fe700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f55840a11e0 0x7f55840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:50.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.001+0000 7f559b7fe700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f55840a11e0 0x7f55840a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f558c005950 tx=0x7f558c00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:50.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.004+0000 7f5598ff9700 1 -- 172.21.15.93:0/2693987576 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5594097170 con 0x7f559c1055f0 2023-11-16T16:47:50.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.276+0000 7f55a221b700 1 -- 172.21.15.93:0/2693987576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5588005cc0 con 0x7f559c1055f0 2023-11-16T16:47:50.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.276+0000 7f5598ff9700 1 -- 172.21.15.93:0/2693987576 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f559409a790 con 0x7f559c1055f0 2023-11-16T16:47:50.278 INFO:teuthology.orchestra.run.smithi093.stdout:107374182465 2023-11-16T16:47:50.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.280+0000 7f55a221b700 1 -- 172.21.15.93:0/2693987576 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f55840a11e0 msgr2=0x7f55840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.280+0000 7f55a221b700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f55840a11e0 0x7f55840a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f558c005950 tx=0x7f558c00b500 comp rx=0 tx=0).stop 2023-11-16T16:47:50.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.280+0000 7f55a221b700 1 -- 172.21.15.93:0/2693987576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f559c1055f0 msgr2=0x7f559c19d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.280+0000 7f55a221b700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f559c1055f0 0x7f559c19d010 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f559400b770 tx=0x7f5594007870 comp rx=0 tx=0).stop 2023-11-16T16:47:50.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.280+0000 7f55a221b700 1 -- 172.21.15.93:0/2693987576 shutdown_connections 2023-11-16T16:47:50.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.280+0000 7f55a221b700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f55840a11e0 0x7f55840a3690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.280+0000 7f55a221b700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f559c100550 0x7f559c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.280+0000 7f55a221b700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f559c1055f0 0x7f559c19d010 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.280+0000 7f55a221b700 1 --2- 172.21.15.93:0/2693987576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c100ef0 0x7f559c198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.280+0000 7f55a221b700 1 -- 172.21.15.93:0/2693987576 >> 172.21.15.93:0/2693987576 conn(0x7f559c0fbdf0 msgr2=0x7f559c10b610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:50.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.280+0000 7f55a221b700 1 -- 172.21.15.93:0/2693987576 shutdown_connections 2023-11-16T16:47:50.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.280+0000 7f55a221b700 1 -- 172.21.15.93:0/2693987576 wait complete. 2023-11-16T16:47:50.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.285+0000 7f1637a98700 1 -- 172.21.15.93:0/4235864227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16301055e0 msgr2=0x7f163010d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.285+0000 7f1637a98700 1 --2- 172.21.15.93:0/4235864227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16301055e0 0x7f163010d160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1628009fc0 tx=0x7f162800b330 comp rx=0 tx=0).stop 2023-11-16T16:47:50.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.285+0000 7f1637a98700 1 -- 172.21.15.93:0/4235864227 shutdown_connections 2023-11-16T16:47:50.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.285+0000 7f1637a98700 1 --2- 172.21.15.93:0/4235864227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16301055e0 0x7f163010d160 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.285+0000 7f1637a98700 1 --2- 172.21.15.93:0/4235864227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1630100540 0x7f1630100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.285+0000 7f1637a98700 1 --2- 172.21.15.93:0/4235864227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1630100ee0 0x7f1630104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.285+0000 7f1637a98700 1 -- 172.21.15.93:0/4235864227 >> 172.21.15.93:0/4235864227 conn(0x7f16300fbdc0 msgr2=0x7f16300fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:50.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.285+0000 7f1637a98700 1 -- 172.21.15.93:0/4235864227 shutdown_connections 2023-11-16T16:47:50.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.285+0000 7f1637a98700 1 -- 172.21.15.93:0/4235864227 wait complete. 2023-11-16T16:47:50.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.286+0000 7f1637a98700 1 Processor -- start 2023-11-16T16:47:50.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.287+0000 7f1637a98700 1 -- start start 2023-11-16T16:47:50.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.287+0000 7f1637a98700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1630100540 0x7f16301965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:50.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.287+0000 7f1637a98700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630100ee0 0x7f1630196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:50.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.287+0000 7f1637a98700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16301055e0 0x7f163019af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:50.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.287+0000 7f1637a98700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163019b590 con 0x7f1630100540 2023-11-16T16:47:50.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.287+0000 7f1637a98700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163019b700 con 0x7f16301055e0 2023-11-16T16:47:50.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.287+0000 7f1637a98700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163019b8c0 con 0x7f1630100ee0 2023-11-16T16:47:50.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.288+0000 7f1635033700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630100ee0 0x7f1630196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:50.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.288+0000 7f1635033700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630100ee0 0x7f1630196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:40974/0 (socket says 172.21.15.93:40974) 2023-11-16T16:47:50.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.288+0000 7f1635033700 1 -- 172.21.15.93:0/1610471866 learned_addr learned my addr 172.21.15.93:0/1610471866 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:50.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.288+0000 7f1635834700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1630100540 0x7f16301965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:50.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.288+0000 7f1636035700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16301055e0 0x7f163019af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:50.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.288+0000 7f1635033700 1 -- 172.21.15.93:0/1610471866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16301055e0 msgr2=0x7f163019af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.288+0000 7f1635033700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16301055e0 0x7f163019af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.288+0000 7f1635033700 1 -- 172.21.15.93:0/1610471866 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1630100540 msgr2=0x7f16301965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.288+0000 7f1635033700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1630100540 0x7f16301965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.288+0000 7f1635033700 1 -- 172.21.15.93:0/1610471866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1628009c40 con 0x7f1630100ee0 2023-11-16T16:47:50.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.289+0000 7f1635033700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630100ee0 0x7f1630196b10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f162000e9f0 tx=0x7f162000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:50.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.289+0000 7f1626ffd700 1 -- 172.21.15.93:0/1610471866 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620009690 con 0x7f1630100ee0 2023-11-16T16:47:50.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.289+0000 7f1626ffd700 1 -- 172.21.15.93:0/1610471866 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f162000ce80 con 0x7f1630100ee0 2023-11-16T16:47:50.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.289+0000 7f1626ffd700 1 -- 172.21.15.93:0/1610471866 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620018420 con 0x7f1630100ee0 2023-11-16T16:47:50.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.289+0000 7f1637a98700 1 -- 172.21.15.93:0/1610471866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1630110370 con 0x7f1630100ee0 2023-11-16T16:47:50.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.290+0000 7f1637a98700 1 -- 172.21.15.93:0/1610471866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16301101e0 con 0x7f1630100ee0 2023-11-16T16:47:50.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.292+0000 7f1637a98700 1 -- 172.21.15.93:0/1610471866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1630110500 con 0x7f1630100ee0 2023-11-16T16:47:50.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.295+0000 7f1626ffd700 1 -- 172.21.15.93:0/1610471866 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f162000c9c0 con 0x7f1630100ee0 2023-11-16T16:47:50.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.297+0000 7f1626ffd700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f161c0a1260 0x7f161c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:50.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.298+0000 7f1626ffd700 1 -- 172.21.15.93:0/1610471866 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1620014070 con 0x7f1630100ee0 2023-11-16T16:47:50.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.298+0000 7f1635834700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f161c0a1260 0x7f161c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:50.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.298+0000 7f1635834700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f161c0a1260 0x7f161c0a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f162c009c60 tx=0x7f162c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:50.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.300+0000 7f1626ffd700 1 -- 172.21.15.93:0/1610471866 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1620051730 con 0x7f1630100ee0 2023-11-16T16:47:50.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.433+0000 7f839d0e3700 1 -- 172.21.15.93:0/612541530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83980737a0 msgr2=0x7f8398110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.433+0000 7f839d0e3700 1 --2- 172.21.15.93:0/612541530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83980737a0 0x7f8398110db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f838c0099c0 tx=0x7f838c009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.434+0000 7f839d0e3700 1 -- 172.21.15.93:0/612541530 shutdown_connections 2023-11-16T16:47:50.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.434+0000 7f839d0e3700 1 --2- 172.21.15.93:0/612541530 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f83981112f0 0x7f83981136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.434+0000 7f839d0e3700 1 --2- 172.21.15.93:0/612541530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8398072e90 0x7f8398073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.434+0000 7f839d0e3700 1 --2- 172.21.15.93:0/612541530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83980737a0 0x7f8398110db0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.434+0000 7f839d0e3700 1 -- 172.21.15.93:0/612541530 >> 172.21.15.93:0/612541530 conn(0x7f83980fbdc0 msgr2=0x7f83980fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:50.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.434+0000 7f839d0e3700 1 -- 172.21.15.93:0/612541530 shutdown_connections 2023-11-16T16:47:50.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.434+0000 7f839d0e3700 1 -- 172.21.15.93:0/612541530 wait complete. 2023-11-16T16:47:50.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.435+0000 7f839d0e3700 1 Processor -- start 2023-11-16T16:47:50.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.435+0000 7f839d0e3700 1 -- start start 2023-11-16T16:47:50.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.435+0000 7f839d0e3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8398072e90 0x7f83981a27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:50.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.435+0000 7f839d0e3700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83980737a0 0x7f83981a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:50.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.435+0000 7f839d0e3700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83981112f0 0x7f839819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:50.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.435+0000 7f839d0e3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839819cea0 con 0x7f8398072e90 2023-11-16T16:47:50.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.435+0000 7f839d0e3700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839819d010 con 0x7f83981112f0 2023-11-16T16:47:50.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.435+0000 7f839d0e3700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839819d1d0 con 0x7f83980737a0 2023-11-16T16:47:50.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.436+0000 7f8396d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8398072e90 0x7f83981a27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:50.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.436+0000 7f8396d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8398072e90 0x7f83981a27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42502/0 (socket says 172.21.15.93:42502) 2023-11-16T16:47:50.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.436+0000 7f8396d9d700 1 -- 172.21.15.93:0/1288182417 learned_addr learned my addr 172.21.15.93:0/1288182417 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:50.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.436+0000 7f839659c700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83980737a0 0x7f83981a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:50.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.436+0000 7f839759e700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83981112f0 0x7f839819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:50.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.436+0000 7f8396d9d700 1 -- 172.21.15.93:0/1288182417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83980737a0 msgr2=0x7f83981a2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.436+0000 7f8396d9d700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83980737a0 0x7f83981a2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.436+0000 7f8396d9d700 1 -- 172.21.15.93:0/1288182417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83981112f0 msgr2=0x7f839819c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.436+0000 7f8396d9d700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83981112f0 0x7f839819c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.436+0000 7f8396d9d700 1 -- 172.21.15.93:0/1288182417 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c0095f0 con 0x7f8398072e90 2023-11-16T16:47:50.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.436+0000 7f839659c700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83980737a0 0x7f83981a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:50.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.436+0000 7f8396d9d700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8398072e90 0x7f83981a27b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f838800e9a0 tx=0x7f838800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:50.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.437+0000 7f837ffff700 1 -- 172.21.15.93:0/1288182417 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388009790 con 0x7f8398072e90 2023-11-16T16:47:50.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.437+0000 7f837ffff700 1 -- 172.21.15.93:0/1288182417 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8388010430 con 0x7f8398072e90 2023-11-16T16:47:50.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.437+0000 7f837ffff700 1 -- 172.21.15.93:0/1288182417 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388018680 con 0x7f8398072e90 2023-11-16T16:47:50.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.437+0000 7f839d0e3700 1 -- 172.21.15.93:0/1288182417 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8398116980 con 0x7f8398072e90 2023-11-16T16:47:50.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.437+0000 7f839d0e3700 1 -- 172.21.15.93:0/1288182417 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83981167f0 con 0x7f8398072e90 2023-11-16T16:47:50.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.438+0000 7f839d0e3700 1 -- 172.21.15.93:0/1288182417 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8398116b10 con 0x7f8398072e90 2023-11-16T16:47:50.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.440+0000 7f837ffff700 1 -- 172.21.15.93:0/1288182417 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f83880105a0 con 0x7f8398072e90 2023-11-16T16:47:50.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.442+0000 7f837ffff700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f83800a1260 0x7f83800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:50.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.442+0000 7f837ffff700 1 -- 172.21.15.93:0/1288182417 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f8388014070 con 0x7f8398072e90 2023-11-16T16:47:50.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.442+0000 7f839659c700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f83800a1260 0x7f83800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:50.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.443+0000 7f839659c700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f83800a1260 0x7f83800a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f838c006010 tx=0x7f838c003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:50.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.446+0000 7f837ffff700 1 -- 172.21.15.93:0/1288182417 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8388051730 con 0x7f8398072e90 2023-11-16T16:47:50.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.552+0000 7f1637a98700 1 -- 172.21.15.93:0/1610471866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f1630066ea0 con 0x7f1630100ee0 2023-11-16T16:47:50.554 INFO:teuthology.orchestra.run.smithi093.stdout:339302416410 2023-11-16T16:47:50.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.552+0000 7f1626ffd700 1 -- 172.21.15.93:0/1610471866 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f1620099b40 con 0x7f1630100ee0 2023-11-16T16:47:50.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.556+0000 7f1637a98700 1 -- 172.21.15.93:0/1610471866 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f161c0a1260 msgr2=0x7f161c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.556+0000 7f1637a98700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f161c0a1260 0x7f161c0a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f162c009c60 tx=0x7f162c009260 comp rx=0 tx=0).stop 2023-11-16T16:47:50.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.556+0000 7f1637a98700 1 -- 172.21.15.93:0/1610471866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630100ee0 msgr2=0x7f1630196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.556+0000 7f1637a98700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630100ee0 0x7f1630196b10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f162000e9f0 tx=0x7f162000ed00 comp rx=0 tx=0).stop 2023-11-16T16:47:50.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.558+0000 7f1637a98700 1 -- 172.21.15.93:0/1610471866 shutdown_connections 2023-11-16T16:47:50.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.558+0000 7f1637a98700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f161c0a1260 0x7f161c0a3710 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.558+0000 7f1637a98700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1630100540 0x7f16301965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.558+0000 7f1637a98700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16301055e0 0x7f163019af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.558+0000 7f1637a98700 1 --2- 172.21.15.93:0/1610471866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630100ee0 0x7f1630196b10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.558+0000 7f1637a98700 1 -- 172.21.15.93:0/1610471866 >> 172.21.15.93:0/1610471866 conn(0x7f16300fbdc0 msgr2=0x7f16300fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:50.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.558+0000 7f1637a98700 1 -- 172.21.15.93:0/1610471866 shutdown_connections 2023-11-16T16:47:50.561 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.559+0000 7f1637a98700 1 -- 172.21.15.93:0/1610471866 wait complete. 2023-11-16T16:47:50.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.697+0000 7f839d0e3700 1 -- 172.21.15.93:0/1288182417 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8398066ea0 con 0x7f8398072e90 2023-11-16T16:47:50.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.697+0000 7f837ffff700 1 -- 172.21.15.93:0/1288182417 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f83880961d0 con 0x7f8398072e90 2023-11-16T16:47:50.700 INFO:teuthology.orchestra.run.smithi093.stdout:219043332142 2023-11-16T16:47:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.700+0000 7f839d0e3700 1 -- 172.21.15.93:0/1288182417 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f83800a1260 msgr2=0x7f83800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.700+0000 7f839d0e3700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f83800a1260 0x7f83800a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f838c006010 tx=0x7f838c003680 comp rx=0 tx=0).stop 2023-11-16T16:47:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.700+0000 7f839d0e3700 1 -- 172.21.15.93:0/1288182417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8398072e90 msgr2=0x7f83981a27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.700+0000 7f839d0e3700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8398072e90 0x7f83981a27b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f838800e9a0 tx=0x7f838800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.700+0000 7f839d0e3700 1 -- 172.21.15.93:0/1288182417 shutdown_connections 2023-11-16T16:47:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.700+0000 7f839d0e3700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f83800a1260 0x7f83800a3710 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.700+0000 7f839d0e3700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8398072e90 0x7f83981a27b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.700+0000 7f839d0e3700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83981112f0 0x7f839819c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.700+0000 7f839d0e3700 1 --2- 172.21.15.93:0/1288182417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83980737a0 0x7f83981a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:50.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.700+0000 7f839d0e3700 1 -- 172.21.15.93:0/1288182417 >> 172.21.15.93:0/1288182417 conn(0x7f83980fbdc0 msgr2=0x7f83980fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:50.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.700+0000 7f839d0e3700 1 -- 172.21.15.93:0/1288182417 shutdown_connections 2023-11-16T16:47:50.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.700+0000 7f839d0e3700 1 -- 172.21.15.93:0/1288182417 wait complete. 2023-11-16T16:47:50.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:50 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3896789299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-16T16:47:50.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:50 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2693987576' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-16T16:47:50.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:50 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2797255761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-16T16:47:50.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:50 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1610471866' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-16T16:47:50.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:50 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3896789299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-16T16:47:50.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:50 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2693987576' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-16T16:47:50.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:50 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2797255761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-16T16:47:50.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:50 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1610471866' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-16T16:47:51.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:50 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3896789299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-16T16:47:51.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:50 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2693987576' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-16T16:47:51.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:50 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2797255761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-16T16:47:51.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:50 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1610471866' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-16T16:47:51.343 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757302 got 163208757304 for osd.5 2023-11-16T16:47:51.343 DEBUG:teuthology.parallel:result is None 2023-11-16T16:47:51.599 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:51 smithi093 ceph-mon[97863]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-16T16:47:51.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:51 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1288182417' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-16T16:47:51.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [1, 2]}]: dispatch 2023-11-16T16:47:51.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]: dispatch 2023-11-16T16:47:51.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2023-11-16T16:47:51.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2023-11-16T16:47:51.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]: dispatch 2023-11-16T16:47:51.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-11-16T16:47:51.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:51 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2023-11-16T16:47:51.676 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182465 for osd.3 2023-11-16T16:47:51.676 DEBUG:teuthology.parallel:result is None 2023-11-16T16:47:51.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:51 smithi159 ceph-mon[131528]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-16T16:47:51.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:51 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1288182417' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-16T16:47:51.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [1, 2]}]: dispatch 2023-11-16T16:47:51.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]: dispatch 2023-11-16T16:47:51.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2023-11-16T16:47:51.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2023-11-16T16:47:51.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]: dispatch 2023-11-16T16:47:51.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-11-16T16:47:51.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:51 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2023-11-16T16:47:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:51 smithi125 ceph-mon[131076]: pgmap v320: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-16T16:47:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:51 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1288182417' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-16T16:47:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [1, 2]}]: dispatch 2023-11-16T16:47:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]: dispatch 2023-11-16T16:47:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2023-11-16T16:47:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2023-11-16T16:47:52.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]: dispatch 2023-11-16T16:47:52.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-11-16T16:47:52.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:51 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2023-11-16T16:47:52.360 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495665 got 197568495667 for osd.6 2023-11-16T16:47:52.360 DEBUG:teuthology.parallel:result is None 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.090+0000 7f733c6d8700 1 -- 172.21.15.93:0/2213984965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7334100ee0 msgr2=0x7f7334104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.090+0000 7f733c6d8700 1 --2- 172.21.15.93:0/2213984965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7334100ee0 0x7f7334104eb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7330009990 tx=0x7f7330009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.090+0000 7f733c6d8700 1 -- 172.21.15.93:0/2213984965 shutdown_connections 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.090+0000 7f733c6d8700 1 --2- 172.21.15.93:0/2213984965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73341055e0 0x7f733410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.090+0000 7f733c6d8700 1 --2- 172.21.15.93:0/2213984965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7334100540 0x7f7334100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.090+0000 7f733c6d8700 1 --2- 172.21.15.93:0/2213984965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7334100ee0 0x7f7334104eb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.090+0000 7f733c6d8700 1 -- 172.21.15.93:0/2213984965 >> 172.21.15.93:0/2213984965 conn(0x7f73340fbdc0 msgr2=0x7f73340fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.091+0000 7f733c6d8700 1 -- 172.21.15.93:0/2213984965 shutdown_connections 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.091+0000 7f733c6d8700 1 -- 172.21.15.93:0/2213984965 wait complete. 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.094+0000 7f733c6d8700 1 Processor -- start 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.095+0000 7f733c6d8700 1 -- start start 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.095+0000 7f733c6d8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7334100540 0x7f7334196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.095+0000 7f733c6d8700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7334100ee0 0x7f73341969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.095+0000 7f733c6d8700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73341055e0 0x7f733419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.095+0000 7f733c6d8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733419b430 con 0x7f7334100540 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.095+0000 7f733c6d8700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733419b5a0 con 0x7f73341055e0 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.095+0000 7f733c6d8700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733419b760 con 0x7f7334100ee0 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.096+0000 7f7339c73700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7334100ee0 0x7f73341969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.096+0000 7f733ac75700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73341055e0 0x7f733419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.096+0000 7f733a474700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7334100540 0x7f7334196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.096+0000 7f7339c73700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7334100ee0 0x7f73341969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:40960/0 (socket says 172.21.15.93:40960) 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.096+0000 7f7339c73700 1 -- 172.21.15.93:0/2797255761 learned_addr learned my addr 172.21.15.93:0/2797255761 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.096+0000 7f733ac75700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73341055e0 0x7f733419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:47798/0 (socket says 172.21.15.93:47798) 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.097+0000 7f7339c73700 1 -- 172.21.15.93:0/2797255761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73341055e0 msgr2=0x7f733419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.097+0000 7f7339c73700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73341055e0 0x7f733419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.097+0000 7f7339c73700 1 -- 172.21.15.93:0/2797255761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7334100540 msgr2=0x7f7334196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.097+0000 7f7339c73700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7334100540 0x7f7334196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.097+0000 7f7339c73700 1 -- 172.21.15.93:0/2797255761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73300095f0 con 0x7f7334100ee0 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.097+0000 7f733a474700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7334100540 0x7f7334196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.097+0000 7f7339c73700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7334100ee0 0x7f73341969b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7330005b40 tx=0x7f7330005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.098+0000 7f732b7fe700 1 -- 172.21.15.93:0/2797255761 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733001d070 con 0x7f7334100ee0 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.098+0000 7f732b7fe700 1 -- 172.21.15.93:0/2797255761 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7330021410 con 0x7f7334100ee0 2023-11-16T16:47:52.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.098+0000 7f732b7fe700 1 -- 172.21.15.93:0/2797255761 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330017650 con 0x7f7334100ee0 2023-11-16T16:47:52.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.099+0000 7f733c6d8700 1 -- 172.21.15.93:0/2797255761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73341103e0 con 0x7f7334100ee0 2023-11-16T16:47:52.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.099+0000 7f733c6d8700 1 -- 172.21.15.93:0/2797255761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7334110250 con 0x7f7334100ee0 2023-11-16T16:47:52.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.102+0000 7f733c6d8700 1 -- 172.21.15.93:0/2797255761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733419bca0 con 0x7f7334100ee0 2023-11-16T16:47:52.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.104+0000 7f732b7fe700 1 -- 172.21.15.93:0/2797255761 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f73300177b0 con 0x7f7334100ee0 2023-11-16T16:47:52.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.106+0000 7f732b7fe700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f73200a1330 0x7f73200a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.106+0000 7f732b7fe700 1 -- 172.21.15.93:0/2797255761 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f73300d5350 con 0x7f7334100ee0 2023-11-16T16:47:52.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.106+0000 7f733a474700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f73200a1330 0x7f73200a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.107+0000 7f733a474700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f73200a1330 0x7f73200a37e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73240099b0 tx=0x7f7324008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.110+0000 7f732b7fe700 1 -- 172.21.15.93:0/2797255761 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f73300d2060 con 0x7f7334100ee0 2023-11-16T16:47:52.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.394+0000 7f733c6d8700 1 -- 172.21.15.93:0/2797255761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7334110570 con 0x7f7334100ee0 2023-11-16T16:47:52.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.395+0000 7f732b7fe700 1 -- 172.21.15.93:0/2797255761 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7330097140 con 0x7f7334100ee0 2023-11-16T16:47:52.470 INFO:teuthology.orchestra.run.smithi093.stdout:141733920827 2023-11-16T16:47:52.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.398+0000 7f733c6d8700 1 -- 172.21.15.93:0/2797255761 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f73200a1330 msgr2=0x7f73200a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.398+0000 7f733c6d8700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f73200a1330 0x7f73200a37e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73240099b0 tx=0x7f7324008040 comp rx=0 tx=0).stop 2023-11-16T16:47:52.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.398+0000 7f733c6d8700 1 -- 172.21.15.93:0/2797255761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7334100ee0 msgr2=0x7f73341969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.398+0000 7f733c6d8700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7334100ee0 0x7f73341969b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7330005b40 tx=0x7f7330005720 comp rx=0 tx=0).stop 2023-11-16T16:47:52.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.398+0000 7f733c6d8700 1 -- 172.21.15.93:0/2797255761 shutdown_connections 2023-11-16T16:47:52.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.398+0000 7f733c6d8700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f73200a1330 0x7f73200a37e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.398+0000 7f733c6d8700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7334100540 0x7f7334196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.398+0000 7f733c6d8700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73341055e0 0x7f733419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.398+0000 7f733c6d8700 1 --2- 172.21.15.93:0/2797255761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7334100ee0 0x7f73341969b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.398+0000 7f733c6d8700 1 -- 172.21.15.93:0/2797255761 >> 172.21.15.93:0/2797255761 conn(0x7f73340fbdc0 msgr2=0x7f73340fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.399+0000 7f733c6d8700 1 -- 172.21.15.93:0/2797255761 shutdown_connections 2023-11-16T16:47:52.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:50.399+0000 7f733c6d8700 1 -- 172.21.15.93:0/2797255761 wait complete. 2023-11-16T16:47:52.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416408 got 339302416410 for osd.11 2023-11-16T16:47:52.532 DEBUG:teuthology.parallel:result is None 2023-11-16T16:47:52.780 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332140 got 219043332142 for osd.7 2023-11-16T16:47:52.781 DEBUG:teuthology.parallel:result is None 2023-11-16T16:47:52.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.949+0000 7f116bc26700 1 -- 172.21.15.93:0/1602361849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1164100540 msgr2=0x7f1164100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.949+0000 7f116bc26700 1 --2- 172.21.15.93:0/1602361849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1164100540 0x7f1164100910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1160009990 tx=0x7f1160009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.949+0000 7f116bc26700 1 -- 172.21.15.93:0/1602361849 shutdown_connections 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.949+0000 7f116bc26700 1 --2- 172.21.15.93:0/1602361849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11641055e0 0x7f116410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.949+0000 7f116bc26700 1 --2- 172.21.15.93:0/1602361849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1164100540 0x7f1164100910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.949+0000 7f116bc26700 1 --2- 172.21.15.93:0/1602361849 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1164100ee0 0x7f1164104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.949+0000 7f116bc26700 1 -- 172.21.15.93:0/1602361849 >> 172.21.15.93:0/1602361849 conn(0x7f11640fbdc0 msgr2=0x7f11640fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.949+0000 7f116bc26700 1 -- 172.21.15.93:0/1602361849 shutdown_connections 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.949+0000 7f116bc26700 1 -- 172.21.15.93:0/1602361849 wait complete. 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.950+0000 7f116bc26700 1 Processor -- start 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.951+0000 7f116bc26700 1 -- start start 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.951+0000 7f116bc26700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1164100540 0x7f1164194250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.951+0000 7f116bc26700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1164100ee0 0x7f1164194790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.951+0000 7f116bc26700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11641055e0 0x7f1164198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.951+0000 7f116bc26700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1164199210 con 0x7f1164100ee0 2023-11-16T16:47:52.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.951+0000 7f116bc26700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1164199380 con 0x7f11641055e0 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.951+0000 7f116bc26700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1164199540 con 0x7f1164100540 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.952+0000 7f11699c2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1164100540 0x7f1164194250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.952+0000 7f116a1c3700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11641055e0 0x7f1164198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.952+0000 7f116a1c3700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11641055e0 0x7f1164198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:47846/0 (socket says 172.21.15.93:47846) 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.952+0000 7f116a1c3700 1 -- 172.21.15.93:0/3510871389 learned_addr learned my addr 172.21.15.93:0/3510871389 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.952+0000 7f11691c1700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1164100ee0 0x7f1164194790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.952+0000 7f116a1c3700 1 -- 172.21.15.93:0/3510871389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1164100540 msgr2=0x7f1164194250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.952+0000 7f116a1c3700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1164100540 0x7f1164194250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.952+0000 7f116a1c3700 1 -- 172.21.15.93:0/3510871389 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1164100ee0 msgr2=0x7f1164194790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.952+0000 7f116a1c3700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1164100ee0 0x7f1164194790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.952+0000 7f116a1c3700 1 -- 172.21.15.93:0/3510871389 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1154009520 con 0x7f11641055e0 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.952+0000 7f11691c1700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1164100ee0 0x7f1164194790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.952+0000 7f116a1c3700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11641055e0 0x7f1164198c00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f115c0049d0 tx=0x7f115c00f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.953+0000 7f115affd700 1 -- 172.21.15.93:0/3510871389 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c014040 con 0x7f11641055e0 2023-11-16T16:47:52.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.953+0000 7f116bc26700 1 -- 172.21.15.93:0/3510871389 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11600095f0 con 0x7f11641055e0 2023-11-16T16:47:52.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.953+0000 7f115affd700 1 -- 172.21.15.93:0/3510871389 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115c004db0 con 0x7f11641055e0 2023-11-16T16:47:52.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.953+0000 7f115affd700 1 -- 172.21.15.93:0/3510871389 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c007e90 con 0x7f11641055e0 2023-11-16T16:47:52.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.953+0000 7f116bc26700 1 -- 172.21.15.93:0/3510871389 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116410e220 con 0x7f11641055e0 2023-11-16T16:47:52.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.954+0000 7f116bc26700 1 -- 172.21.15.93:0/3510871389 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116410e090 con 0x7f11641055e0 2023-11-16T16:47:52.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.956+0000 7f115affd700 1 -- 172.21.15.93:0/3510871389 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f115c007500 con 0x7f11641055e0 2023-11-16T16:47:52.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.957+0000 7f115affd700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f11500a1260 0x7f11500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.957+0000 7f115affd700 1 -- 172.21.15.93:0/3510871389 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f115c0d0b20 con 0x7f11641055e0 2023-11-16T16:47:52.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.957+0000 7f11699c2700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f11500a1260 0x7f11500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.958+0000 7f11699c2700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f11500a1260 0x7f11500a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f116000b2e0 tx=0x7f1160005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:51.961+0000 7f115affd700 1 -- 172.21.15.93:0/3510871389 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f115c052730 con 0x7f11641055e0 2023-11-16T16:47:52.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.197+0000 7f116bc26700 1 -- 172.21.15.93:0/3510871389 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f116410e3b0 con 0x7f11641055e0 2023-11-16T16:47:52.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.198+0000 7f115affd700 1 -- 172.21.15.93:0/3510871389 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f115c099db0 con 0x7f11641055e0 2023-11-16T16:47:52.785 INFO:teuthology.orchestra.run.smithi093.stdout:287762808869 2023-11-16T16:47:52.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.200+0000 7f116bc26700 1 -- 172.21.15.93:0/3510871389 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f11500a1260 msgr2=0x7f11500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.200+0000 7f116bc26700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f11500a1260 0x7f11500a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f116000b2e0 tx=0x7f1160005fb0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.200+0000 7f116bc26700 1 -- 172.21.15.93:0/3510871389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11641055e0 msgr2=0x7f1164198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.200+0000 7f116bc26700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11641055e0 0x7f1164198c00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f115c0049d0 tx=0x7f115c00f740 comp rx=0 tx=0).stop 2023-11-16T16:47:52.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.200+0000 7f116bc26700 1 -- 172.21.15.93:0/3510871389 shutdown_connections 2023-11-16T16:47:52.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.200+0000 7f116bc26700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f11500a1260 0x7f11500a3710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.200+0000 7f116bc26700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11641055e0 0x7f1164198c00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.200+0000 7f116bc26700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1164100540 0x7f1164194250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.200+0000 7f116bc26700 1 --2- 172.21.15.93:0/3510871389 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1164100ee0 0x7f1164194790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.200+0000 7f116bc26700 1 -- 172.21.15.93:0/3510871389 >> 172.21.15.93:0/3510871389 conn(0x7f11640fbdc0 msgr2=0x7f11640fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.200+0000 7f116bc26700 1 -- 172.21.15.93:0/3510871389 shutdown_connections 2023-11-16T16:47:52.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:52.200+0000 7f116bc26700 1 -- 172.21.15.93:0/3510871389 wait complete. 2023-11-16T16:47:52.883 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:52 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3510871389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-16T16:47:52.883 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [1, 2]}]': finished 2023-11-16T16:47:52.883 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]': finished 2023-11-16T16:47:52.883 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2023-11-16T16:47:52.883 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2023-11-16T16:47:52.883 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]': finished 2023-11-16T16:47:52.883 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-11-16T16:47:52.883 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:52 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2023-11-16T16:47:52.883 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:52 smithi093 ceph-mon[97863]: osdmap e91: 12 total, 12 up, 12 in 2023-11-16T16:47:52.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:52 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3510871389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-16T16:47:52.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [1, 2]}]': finished 2023-11-16T16:47:52.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]': finished 2023-11-16T16:47:52.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2023-11-16T16:47:52.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2023-11-16T16:47:52.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]': finished 2023-11-16T16:47:52.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-11-16T16:47:52.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:52 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2023-11-16T16:47:52.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:52 smithi159 ceph-mon[131528]: osdmap e91: 12 total, 12 up, 12 in 2023-11-16T16:47:53.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:52 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3510871389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-16T16:47:53.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [1, 2]}]': finished 2023-11-16T16:47:53.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]': finished 2023-11-16T16:47:53.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2023-11-16T16:47:53.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2023-11-16T16:47:53.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]': finished 2023-11-16T16:47:53.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-11-16T16:47:53.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:52 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2023-11-16T16:47:53.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:52 smithi125 ceph-mon[131076]: osdmap e91: 12 total, 12 up, 12 in 2023-11-16T16:47:53.335 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808867 got 287762808869 for osd.9 2023-11-16T16:47:53.335 DEBUG:teuthology.parallel:result is None 2023-11-16T16:47:53.597 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:53.648 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:53.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.653+0000 7f9c201fb700 1 -- 172.21.15.93:0/1324800963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c18100540 msgr2=0x7f9c18100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.653+0000 7f9c201fb700 1 --2- 172.21.15.93:0/1324800963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c18100540 0x7f9c18100910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9c140099c0 tx=0x7f9c14009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.653+0000 7f9c201fb700 1 -- 172.21.15.93:0/1324800963 shutdown_connections 2023-11-16T16:47:53.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.653+0000 7f9c201fb700 1 --2- 172.21.15.93:0/1324800963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c18100540 0x7f9c18100910 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.653+0000 7f9c201fb700 1 --2- 172.21.15.93:0/1324800963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c181055e0 0x7f9c1810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.653+0000 7f9c201fb700 1 --2- 172.21.15.93:0/1324800963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c18100ee0 0x7f9c18104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.653+0000 7f9c201fb700 1 -- 172.21.15.93:0/1324800963 >> 172.21.15.93:0/1324800963 conn(0x7f9c180fbdc0 msgr2=0x7f9c180fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:53.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.655+0000 7f9c201fb700 1 -- 172.21.15.93:0/1324800963 shutdown_connections 2023-11-16T16:47:53.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.655+0000 7f9c201fb700 1 -- 172.21.15.93:0/1324800963 wait complete. 2023-11-16T16:47:53.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.656+0000 7f9c201fb700 1 Processor -- start 2023-11-16T16:47:53.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.657+0000 7f9c201fb700 1 -- start start 2023-11-16T16:47:53.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.657+0000 7f9c201fb700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c18100540 0x7f9c1810ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.657+0000 7f9c201fb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c18100ee0 0x7f9c18109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.657+0000 7f9c201fb700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c181055e0 0x7f9c1810a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.657+0000 7f9c201fb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1810a750 con 0x7f9c18100ee0 2023-11-16T16:47:53.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.657+0000 7f9c201fb700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1810a8c0 con 0x7f9c181055e0 2023-11-16T16:47:53.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.657+0000 7f9c201fb700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1810aa80 con 0x7f9c18100540 2023-11-16T16:47:53.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.657+0000 7f9c1df97700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c18100540 0x7f9c1810ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.657+0000 7f9c1df97700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c18100540 0x7f9c1810ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:41036/0 (socket says 172.21.15.93:41036) 2023-11-16T16:47:53.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.658+0000 7f9c1df97700 1 -- 172.21.15.93:0/3680526458 learned_addr learned my addr 172.21.15.93:0/3680526458 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:53.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.658+0000 7f9c1e798700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c181055e0 0x7f9c1810a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.658+0000 7f9c1d796700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c18100ee0 0x7f9c18109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.658+0000 7f9c1df97700 1 -- 172.21.15.93:0/3680526458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c181055e0 msgr2=0x7f9c1810a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.658+0000 7f9c1df97700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c181055e0 0x7f9c1810a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.658+0000 7f9c1df97700 1 -- 172.21.15.93:0/3680526458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c18100ee0 msgr2=0x7f9c18109cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.658+0000 7f9c1df97700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c18100ee0 0x7f9c18109cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.658+0000 7f9c1df97700 1 -- 172.21.15.93:0/3680526458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c140095f0 con 0x7f9c18100540 2023-11-16T16:47:53.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.659+0000 7f9c1df97700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c18100540 0x7f9c1810ccb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9c14000c00 tx=0x7f9c1400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:53.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.659+0000 7f9c0effd700 1 -- 172.21.15.93:0/3680526458 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1401d070 con 0x7f9c18100540 2023-11-16T16:47:53.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.659+0000 7f9c201fb700 1 -- 172.21.15.93:0/3680526458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c180747f0 con 0x7f9c18100540 2023-11-16T16:47:53.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.659+0000 7f9c0effd700 1 -- 172.21.15.93:0/3680526458 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c14005050 con 0x7f9c18100540 2023-11-16T16:47:53.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.659+0000 7f9c0effd700 1 -- 172.21.15.93:0/3680526458 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14004b70 con 0x7f9c18100540 2023-11-16T16:47:53.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.659+0000 7f9c201fb700 1 -- 172.21.15.93:0/3680526458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c18074660 con 0x7f9c18100540 2023-11-16T16:47:53.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.660+0000 7f9c201fb700 1 -- 172.21.15.93:0/3680526458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c18074980 con 0x7f9c18100540 2023-11-16T16:47:53.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.662+0000 7f9c0effd700 1 -- 172.21.15.93:0/3680526458 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9c14004cd0 con 0x7f9c18100540 2023-11-16T16:47:53.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.665+0000 7f9c0effd700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9c040a11e0 0x7f9c040a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.665+0000 7f9c0effd700 1 -- 172.21.15.93:0/3680526458 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f9c140cdc20 con 0x7f9c18100540 2023-11-16T16:47:53.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.665+0000 7f9c1d796700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9c040a11e0 0x7f9c040a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.666+0000 7f9c1d796700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9c040a11e0 0x7f9c040a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c1810b4c0 tx=0x7f9c080073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:53.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.668+0000 7f9c0effd700 1 -- 172.21.15.93:0/3680526458 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9c14096e70 con 0x7f9c18100540 2023-11-16T16:47:53.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:53 smithi093 ceph-mon[97863]: pgmap v322: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:53.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:53 smithi093 ceph-mon[97863]: osdmap e92: 12 total, 12 up, 12 in 2023-11-16T16:47:53.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.901+0000 7f9c201fb700 1 -- 172.21.15.93:0/3680526458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f9c1810b310 con 0x7f9c18100540 2023-11-16T16:47:53.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.902+0000 7f9c0effd700 1 -- 172.21.15.93:0/3680526458 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c14026090 con 0x7f9c18100540 2023-11-16T16:47:53.903 INFO:teuthology.orchestra.run.smithi093.stdout:253403070507 2023-11-16T16:47:53.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.904+0000 7f9c201fb700 1 -- 172.21.15.93:0/3680526458 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9c040a11e0 msgr2=0x7f9c040a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.904+0000 7f9c201fb700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9c040a11e0 0x7f9c040a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c1810b4c0 tx=0x7f9c080073d0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.904+0000 7f9c201fb700 1 -- 172.21.15.93:0/3680526458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c18100540 msgr2=0x7f9c1810ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.904+0000 7f9c201fb700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c18100540 0x7f9c1810ccb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9c14000c00 tx=0x7f9c1400f690 comp rx=0 tx=0).stop 2023-11-16T16:47:53.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.904+0000 7f9c201fb700 1 -- 172.21.15.93:0/3680526458 shutdown_connections 2023-11-16T16:47:53.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.904+0000 7f9c201fb700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f9c040a11e0 0x7f9c040a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.904+0000 7f9c201fb700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c181055e0 0x7f9c1810a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.904+0000 7f9c201fb700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c18100540 0x7f9c1810ccb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.904+0000 7f9c201fb700 1 --2- 172.21.15.93:0/3680526458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c18100ee0 0x7f9c18109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.904+0000 7f9c201fb700 1 -- 172.21.15.93:0/3680526458 >> 172.21.15.93:0/3680526458 conn(0x7f9c180fbdc0 msgr2=0x7f9c180fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:53.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.904+0000 7f9c201fb700 1 -- 172.21.15.93:0/3680526458 shutdown_connections 2023-11-16T16:47:53.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:53.905+0000 7f9c201fb700 1 -- 172.21.15.93:0/3680526458 wait complete. 2023-11-16T16:47:53.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:53 smithi159 ceph-mon[131528]: pgmap v322: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:53.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:53 smithi159 ceph-mon[131528]: osdmap e92: 12 total, 12 up, 12 in 2023-11-16T16:47:54.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:53 smithi125 ceph-mon[131076]: pgmap v322: 129 pgs: 129 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:54.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:53 smithi125 ceph-mon[131076]: osdmap e92: 12 total, 12 up, 12 in 2023-11-16T16:47:54.424 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920825 got 141733920827 for osd.4 2023-11-16T16:47:54.424 DEBUG:teuthology.parallel:result is None 2023-11-16T16:47:54.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:54 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:54.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:54 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:54.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:54 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3680526458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-16T16:47:55.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:54 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:55.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:54 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:55.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:54 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3680526458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-16T16:47:55.194 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070504 got 253403070507 for osd.8 2023-11-16T16:47:55.194 DEBUG:teuthology.parallel:result is None 2023-11-16T16:47:55.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:54 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:55.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:54 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:55.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:54 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3680526458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-16T16:47:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:55 smithi093 ceph-mon[97863]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-11-16T16:47:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:55 smithi093 ceph-mon[97863]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-11-16T16:47:55.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:47:55.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:47:55.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:55 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:47:55.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:55.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:55 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:47:55.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:55 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:47:55.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:55 smithi093 ceph-mon[97863]: Service mds.all has no related daemon already placed. 2023-11-16T16:47:55.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.986+0000 7fee430b2700 1 -- 172.21.15.93:0/2348535441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee3c100ee0 msgr2=0x7fee3c104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:55.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.986+0000 7fee430b2700 1 --2- 172.21.15.93:0/2348535441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee3c100ee0 0x7fee3c104eb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fee280099c0 tx=0x7fee28009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:55.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.986+0000 7fee430b2700 1 -- 172.21.15.93:0/2348535441 shutdown_connections 2023-11-16T16:47:55.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.986+0000 7fee430b2700 1 --2- 172.21.15.93:0/2348535441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee3c1055e0 0x7fee3c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:55.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.986+0000 7fee430b2700 1 --2- 172.21.15.93:0/2348535441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3c100540 0x7fee3c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:55.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.986+0000 7fee430b2700 1 --2- 172.21.15.93:0/2348535441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee3c100ee0 0x7fee3c104eb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:55.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.986+0000 7fee430b2700 1 -- 172.21.15.93:0/2348535441 >> 172.21.15.93:0/2348535441 conn(0x7fee3c0fbdc0 msgr2=0x7fee3c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:55.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.986+0000 7fee430b2700 1 -- 172.21.15.93:0/2348535441 shutdown_connections 2023-11-16T16:47:55.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.986+0000 7fee430b2700 1 -- 172.21.15.93:0/2348535441 wait complete. 2023-11-16T16:47:55.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.987+0000 7fee430b2700 1 Processor -- start 2023-11-16T16:47:55.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.988+0000 7fee430b2700 1 -- start start 2023-11-16T16:47:55.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.988+0000 7fee430b2700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3c100540 0x7fee3c1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:55.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.988+0000 7fee430b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee3c100ee0 0x7fee3c1968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:55.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.988+0000 7fee430b2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee3c1055e0 0x7fee3c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:55.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.988+0000 7fee430b2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c19b360 con 0x7fee3c100ee0 2023-11-16T16:47:55.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.988+0000 7fee430b2700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c19b4d0 con 0x7fee3c100540 2023-11-16T16:47:55.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.988+0000 7fee430b2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c19b690 con 0x7fee3c1055e0 2023-11-16T16:47:55.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.988+0000 7fee3bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee3c100ee0 0x7fee3c1968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:55.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.988+0000 7fee3bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee3c100ee0 0x7fee3c1968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42556/0 (socket says 172.21.15.93:42556) 2023-11-16T16:47:55.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.988+0000 7fee3bfff700 1 -- 172.21.15.93:0/2682592802 learned_addr learned my addr 172.21.15.93:0/2682592802 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:55.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.988+0000 7fee40e4e700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3c100540 0x7fee3c1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:55.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.988+0000 7fee4164f700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee3c1055e0 0x7fee3c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:55.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.989+0000 7fee3bfff700 1 -- 172.21.15.93:0/2682592802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee3c1055e0 msgr2=0x7fee3c19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:55.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.989+0000 7fee3bfff700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee3c1055e0 0x7fee3c19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:55.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.989+0000 7fee3bfff700 1 -- 172.21.15.93:0/2682592802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3c100540 msgr2=0x7fee3c1963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:55.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.989+0000 7fee3bfff700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3c100540 0x7fee3c1963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:55.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.989+0000 7fee3bfff700 1 -- 172.21.15.93:0/2682592802 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee280095f0 con 0x7fee3c100ee0 2023-11-16T16:47:55.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.989+0000 7fee3bfff700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee3c100ee0 0x7fee3c1968e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fee2800b580 tx=0x7fee2800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:55.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.989+0000 7fee39ffb700 1 -- 172.21.15.93:0/2682592802 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2801d070 con 0x7fee3c100ee0 2023-11-16T16:47:55.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.989+0000 7fee430b2700 1 -- 172.21.15.93:0/2682592802 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c110310 con 0x7fee3c100ee0 2023-11-16T16:47:55.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.989+0000 7fee39ffb700 1 -- 172.21.15.93:0/2682592802 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee28005050 con 0x7fee3c100ee0 2023-11-16T16:47:55.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.989+0000 7fee39ffb700 1 -- 172.21.15.93:0/2682592802 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28004a90 con 0x7fee3c100ee0 2023-11-16T16:47:55.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.989+0000 7fee430b2700 1 -- 172.21.15.93:0/2682592802 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c110180 con 0x7fee3c100ee0 2023-11-16T16:47:55.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.991+0000 7fee277fe700 1 -- 172.21.15.93:0/2682592802 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1c0052f0 con 0x7fee3c100ee0 2023-11-16T16:47:55.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.992+0000 7fee39ffb700 1 -- 172.21.15.93:0/2682592802 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fee280051c0 con 0x7fee3c100ee0 2023-11-16T16:47:55.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.994+0000 7fee39ffb700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fee2c0a11e0 0x7fee2c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:55.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.994+0000 7fee39ffb700 1 -- 172.21.15.93:0/2682592802 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fee280cdc00 con 0x7fee3c100ee0 2023-11-16T16:47:55.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.994+0000 7fee40e4e700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fee2c0a11e0 0x7fee2c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:55.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.994+0000 7fee40e4e700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fee2c0a11e0 0x7fee2c0a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fee30009c60 tx=0x7fee30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:56.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:55.998+0000 7fee39ffb700 1 -- 172.21.15.93:0/2682592802 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fee28096e50 con 0x7fee3c100ee0 2023-11-16T16:47:56.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:55 smithi125 ceph-mon[131076]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-11-16T16:47:56.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:55 smithi125 ceph-mon[131076]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-11-16T16:47:56.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:47:56.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:47:56.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:55 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:47:56.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:56.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:55 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:47:56.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:55 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:47:56.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:55 smithi125 ceph-mon[131076]: Service mds.all has no related daemon already placed. 2023-11-16T16:47:56.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.218+0000 7fee277fe700 1 -- 172.21.15.93:0/2682592802 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fee1c005160 con 0x7fee3c100ee0 2023-11-16T16:47:56.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.219+0000 7fee39ffb700 1 -- 172.21.15.93:0/2682592802 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fee2809a470 con 0x7fee3c100ee0 2023-11-16T16:47:56.221 INFO:teuthology.orchestra.run.smithi093.stdout:55834574922 2023-11-16T16:47:56.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.221+0000 7fee277fe700 1 -- 172.21.15.93:0/2682592802 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fee2c0a11e0 msgr2=0x7fee2c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:56.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.221+0000 7fee277fe700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fee2c0a11e0 0x7fee2c0a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fee30009c60 tx=0x7fee30009260 comp rx=0 tx=0).stop 2023-11-16T16:47:56.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.221+0000 7fee277fe700 1 -- 172.21.15.93:0/2682592802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee3c100ee0 msgr2=0x7fee3c1968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:56.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.221+0000 7fee277fe700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee3c100ee0 0x7fee3c1968e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fee2800b580 tx=0x7fee2800bfd0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.221+0000 7fee277fe700 1 -- 172.21.15.93:0/2682592802 shutdown_connections 2023-11-16T16:47:56.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.221+0000 7fee277fe700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fee2c0a11e0 0x7fee2c0a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.221+0000 7fee277fe700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee3c1055e0 0x7fee3c19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.221+0000 7fee277fe700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3c100540 0x7fee3c1963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.221+0000 7fee277fe700 1 --2- 172.21.15.93:0/2682592802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee3c100ee0 0x7fee3c1968e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.221+0000 7fee277fe700 1 -- 172.21.15.93:0/2682592802 >> 172.21.15.93:0/2682592802 conn(0x7fee3c0fbdc0 msgr2=0x7fee3c0fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:56.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.221+0000 7fee277fe700 1 -- 172.21.15.93:0/2682592802 shutdown_connections 2023-11-16T16:47:56.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.221+0000 7fee277fe700 1 -- 172.21.15.93:0/2682592802 wait complete. 2023-11-16T16:47:56.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:55 smithi159 ceph-mon[131528]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-11-16T16:47:56.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:55 smithi159 ceph-mon[131528]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-11-16T16:47:56.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:47:56.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:47:56.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:55 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:47:56.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:47:56.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:55 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:47:56.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:55 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:47:56.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:55 smithi159 ceph-mon[131528]: Service mds.all has no related daemon already placed. 2023-11-16T16:47:56.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.686+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/542777521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1ac100ee0 msgr2=0x7fe1ac104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:56.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.686+0000 7fe1b0ab9700 1 --2- 172.21.15.93:0/542777521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1ac100ee0 0x7fe1ac104eb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009920 tx=0x7fe1a0009c30 comp rx=0 tx=0).stop 2023-11-16T16:47:56.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.686+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/542777521 shutdown_connections 2023-11-16T16:47:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.686+0000 7fe1b0ab9700 1 --2- 172.21.15.93:0/542777521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1ac1055e0 0x7fe1ac10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.686+0000 7fe1b0ab9700 1 --2- 172.21.15.93:0/542777521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac100540 0x7fe1ac100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.686+0000 7fe1b0ab9700 1 --2- 172.21.15.93:0/542777521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1ac100ee0 0x7fe1ac104eb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.686+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/542777521 >> 172.21.15.93:0/542777521 conn(0x7fe1ac0fbdc0 msgr2=0x7fe1ac0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.686+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/542777521 shutdown_connections 2023-11-16T16:47:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.687+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/542777521 wait complete. 2023-11-16T16:47:56.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.687+0000 7fe1b0ab9700 1 Processor -- start 2023-11-16T16:47:56.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.688+0000 7fe1b0ab9700 1 -- start start 2023-11-16T16:47:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.688+0000 7fe1b0ab9700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac100540 0x7fe1ac196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.688+0000 7fe1b0ab9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1ac100ee0 0x7fe1ac196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.688+0000 7fe1b0ab9700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1ac1055e0 0x7fe1ac19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.688+0000 7fe1b0ab9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac19b3f0 con 0x7fe1ac100ee0 2023-11-16T16:47:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.688+0000 7fe1b0ab9700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac19b560 con 0x7fe1ac1055e0 2023-11-16T16:47:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.688+0000 7fe1b0ab9700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac19b720 con 0x7fe1ac100540 2023-11-16T16:47:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.688+0000 7fe1a9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1ac100ee0 0x7fe1ac196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.688+0000 7fe1a9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1ac100ee0 0x7fe1ac196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42586/0 (socket says 172.21.15.93:42586) 2023-11-16T16:47:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.688+0000 7fe1a9d9b700 1 -- 172.21.15.93:0/1002062212 learned_addr learned my addr 172.21.15.93:0/1002062212 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.688+0000 7fe1aa59c700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac100540 0x7fe1ac196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.688+0000 7fe1aad9d700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1ac1055e0 0x7fe1ac19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.689+0000 7fe1aa59c700 1 -- 172.21.15.93:0/1002062212 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1ac1055e0 msgr2=0x7fe1ac19ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.689+0000 7fe1aa59c700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1ac1055e0 0x7fe1ac19ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.689+0000 7fe1aa59c700 1 -- 172.21.15.93:0/1002062212 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1ac100ee0 msgr2=0x7fe1ac196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.689+0000 7fe1aa59c700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1ac100ee0 0x7fe1ac196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.689+0000 7fe1aa59c700 1 -- 172.21.15.93:0/1002062212 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19c0095f0 con 0x7fe1ac100540 2023-11-16T16:47:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.689+0000 7fe1a9d9b700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1ac100ee0 0x7fe1ac196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.689+0000 7fe1aa59c700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac100540 0x7fe1ac196430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00eb90 tx=0x7fe19c00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.689+0000 7fe19b7fe700 1 -- 172.21.15.93:0/1002062212 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c009970 con 0x7fe1ac100540 2023-11-16T16:47:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.689+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/1002062212 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a0009520 con 0x7fe1ac100540 2023-11-16T16:47:56.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.689+0000 7fe19b7fe700 1 -- 172.21.15.93:0/1002062212 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe19c0044e0 con 0x7fe1ac100540 2023-11-16T16:47:56.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.689+0000 7fe19b7fe700 1 -- 172.21.15.93:0/1002062212 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c003e50 con 0x7fe1ac100540 2023-11-16T16:47:56.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.690+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/1002062212 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ac110310 con 0x7fe1ac100540 2023-11-16T16:47:56.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.691+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/1002062212 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ac110180 con 0x7fe1ac100540 2023-11-16T16:47:56.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.693+0000 7fe19b7fe700 1 -- 172.21.15.93:0/1002062212 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe19c004000 con 0x7fe1ac100540 2023-11-16T16:47:56.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.694+0000 7fe19b7fe700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe1940a1260 0x7fe1940a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:56.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.694+0000 7fe19b7fe700 1 -- 172.21.15.93:0/1002062212 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe19c014070 con 0x7fe1ac100540 2023-11-16T16:47:56.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.694+0000 7fe1a9d9b700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe1940a1260 0x7fe1940a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:56.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.695+0000 7fe1a9d9b700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe1940a1260 0x7fe1940a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000c010 tx=0x7fe1a0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:56.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.698+0000 7fe19b7fe700 1 -- 172.21.15.93:0/1002062212 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe19c0964c0 con 0x7fe1ac100540 2023-11-16T16:47:56.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.900+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/1002062212 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe1ac1104a0 con 0x7fe1ac100540 2023-11-16T16:47:56.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.901+0000 7fe19b7fe700 1 -- 172.21.15.93:0/1002062212 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe19c099ae0 con 0x7fe1ac100540 2023-11-16T16:47:56.903 INFO:teuthology.orchestra.run.smithi093.stdout:77309411400 2023-11-16T16:47:56.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.903+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/1002062212 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe1940a1260 msgr2=0x7fe1940a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:56.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.903+0000 7fe1b0ab9700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe1940a1260 0x7fe1940a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000c010 tx=0x7fe1a0019040 comp rx=0 tx=0).stop 2023-11-16T16:47:56.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.903+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/1002062212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac100540 msgr2=0x7fe1ac196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:56.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.903+0000 7fe1b0ab9700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac100540 0x7fe1ac196430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00eb90 tx=0x7fe19c00eea0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.903+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/1002062212 shutdown_connections 2023-11-16T16:47:56.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.903+0000 7fe1b0ab9700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe1940a1260 0x7fe1940a3710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.903+0000 7fe1b0ab9700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1ac1055e0 0x7fe1ac19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.903+0000 7fe1b0ab9700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac100540 0x7fe1ac196430 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.903+0000 7fe1b0ab9700 1 --2- 172.21.15.93:0/1002062212 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1ac100ee0 0x7fe1ac196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:56.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.903+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/1002062212 >> 172.21.15.93:0/1002062212 conn(0x7fe1ac0fbdc0 msgr2=0x7fe1ac0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:56.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.903+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/1002062212 shutdown_connections 2023-11-16T16:47:56.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:56.903+0000 7fe1b0ab9700 1 -- 172.21.15.93:0/1002062212 wait complete. 2023-11-16T16:47:56.979 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:56 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2682592802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-16T16:47:57.045 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574922 for osd.1 2023-11-16T16:47:57.045 DEBUG:teuthology.parallel:result is None 2023-11-16T16:47:57.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:56 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2682592802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-16T16:47:57.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:56 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2682592802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-16T16:47:57.388 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411400 for osd.2 2023-11-16T16:47:57.389 DEBUG:teuthology.parallel:result is None 2023-11-16T16:47:57.389 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-11-16T16:47:57.389 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph pg dump --format=json 2023-11-16T16:47:58.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:57 smithi125 ceph-mon[131076]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-16T16:47:58.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:57 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1002062212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-16T16:47:58.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:57 smithi093 ceph-mon[97863]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-16T16:47:58.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:57 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1002062212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-16T16:47:58.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:57 smithi159 ceph-mon[131528]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-16T16:47:58.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:57 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1002062212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-16T16:47:58.237 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:47:59.085 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:58 smithi093 ceph-mon[97863]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2023-11-16T16:47:59.085 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:58 smithi093 ceph-mon[97863]: Cluster is now healthy 2023-11-16T16:47:59.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:58 smithi159 ceph-mon[131528]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2023-11-16T16:47:59.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:58 smithi159 ceph-mon[131528]: Cluster is now healthy 2023-11-16T16:47:59.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:58 smithi125 ceph-mon[131076]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2023-11-16T16:47:59.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:58 smithi125 ceph-mon[131076]: Cluster is now healthy 2023-11-16T16:47:59.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.744+0000 7f35e4ab6700 1 -- 172.21.15.93:0/2400315697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35e0100ee0 msgr2=0x7f35e0104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:59.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.744+0000 7f35e4ab6700 1 --2- 172.21.15.93:0/2400315697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35e0100ee0 0x7f35e0104eb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f35d40099c0 tx=0x7f35d4009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:59.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.744+0000 7f35e4ab6700 1 -- 172.21.15.93:0/2400315697 shutdown_connections 2023-11-16T16:47:59.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.744+0000 7f35e4ab6700 1 --2- 172.21.15.93:0/2400315697 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e01055e0 0x7f35e010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:59.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.744+0000 7f35e4ab6700 1 --2- 172.21.15.93:0/2400315697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0100540 0x7f35e0100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:59.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.744+0000 7f35e4ab6700 1 --2- 172.21.15.93:0/2400315697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35e0100ee0 0x7f35e0104eb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:59.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.744+0000 7f35e4ab6700 1 -- 172.21.15.93:0/2400315697 >> 172.21.15.93:0/2400315697 conn(0x7f35e00fbdc0 msgr2=0x7f35e00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:59.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.744+0000 7f35e4ab6700 1 -- 172.21.15.93:0/2400315697 shutdown_connections 2023-11-16T16:47:59.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.744+0000 7f35e4ab6700 1 -- 172.21.15.93:0/2400315697 wait complete. 2023-11-16T16:47:59.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.745+0000 7f35e4ab6700 1 Processor -- start 2023-11-16T16:47:59.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.745+0000 7f35e4ab6700 1 -- start start 2023-11-16T16:47:59.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35e4ab6700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e0100540 0x7f35e01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:59.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35e4ab6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0100ee0 0x7f35e0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:59.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35e4ab6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35e01055e0 0x7f35e019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:59.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35e4ab6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e019d560 con 0x7f35e01055e0 2023-11-16T16:47:59.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35e4ab6700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e019d6d0 con 0x7f35e0100540 2023-11-16T16:47:59.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35e4ab6700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e019d890 con 0x7f35e0100ee0 2023-11-16T16:47:59.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35ddd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0100ee0 0x7f35e0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:59.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35ddd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0100ee0 0x7f35e0198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:36560/0 (socket says 172.21.15.93:36560) 2023-11-16T16:47:59.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35ddd9b700 1 -- 172.21.15.93:0/622388458 learned_addr learned my addr 172.21.15.93:0/622388458 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:47:59.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35ded9d700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35e01055e0 0x7f35e019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:59.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35de59c700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e0100540 0x7f35e01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:59.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35ddd9b700 1 -- 172.21.15.93:0/622388458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e0100540 msgr2=0x7f35e01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:59.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35ddd9b700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e0100540 0x7f35e01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:59.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35ddd9b700 1 -- 172.21.15.93:0/622388458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35e01055e0 msgr2=0x7f35e019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:59.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35ddd9b700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35e01055e0 0x7f35e019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:59.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.746+0000 7f35ddd9b700 1 -- 172.21.15.93:0/622388458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40095f0 con 0x7f35e0100ee0 2023-11-16T16:47:59.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.747+0000 7f35ddd9b700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0100ee0 0x7f35e0198ae0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35d40098e0 tx=0x7f35d400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:59.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.747+0000 7f35cf7fe700 1 -- 172.21.15.93:0/622388458 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d401d070 con 0x7f35e0100ee0 2023-11-16T16:47:59.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.747+0000 7f35e4ab6700 1 -- 172.21.15.93:0/622388458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e0110310 con 0x7f35e0100ee0 2023-11-16T16:47:59.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.747+0000 7f35cf7fe700 1 -- 172.21.15.93:0/622388458 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35d4022470 con 0x7f35e0100ee0 2023-11-16T16:47:59.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.747+0000 7f35cf7fe700 1 -- 172.21.15.93:0/622388458 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4017650 con 0x7f35e0100ee0 2023-11-16T16:47:59.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.747+0000 7f35e4ab6700 1 -- 172.21.15.93:0/622388458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e0110180 con 0x7f35e0100ee0 2023-11-16T16:47:59.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.748+0000 7f35e4ab6700 1 -- 172.21.15.93:0/622388458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e01104a0 con 0x7f35e0100ee0 2023-11-16T16:47:59.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.750+0000 7f35cf7fe700 1 -- 172.21.15.93:0/622388458 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f35d4022850 con 0x7f35e0100ee0 2023-11-16T16:47:59.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.751+0000 7f35cf7fe700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f35c80a11e0 0x7f35c80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:59.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.751+0000 7f35cf7fe700 1 -- 172.21.15.93:0/622388458 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f35d40cdd00 con 0x7f35e0100ee0 2023-11-16T16:47:59.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.751+0000 7f35de59c700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f35c80a11e0 0x7f35c80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:59.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.752+0000 7f35de59c700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f35c80a11e0 0x7f35c80a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009c60 tx=0x7f35d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:59.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.753+0000 7f35cf7fe700 1 -- 172.21.15.93:0/622388458 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f35d4097180 con 0x7f35e0100ee0 2023-11-16T16:47:59.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.949+0000 7f35e4ab6700 1 -- 172.21.15.93:0/622388458 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35e0073d30 con 0x7f35c80a11e0 2023-11-16T16:47:59.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.962+0000 7f35cf7fe700 1 -- 172.21.15.93:0/622388458 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427465 (secure 0 0 0) 0x7f35e0073d30 con 0x7f35c80a11e0 2023-11-16T16:47:59.969 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:47:59.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.967+0000 7f35e4ab6700 1 -- 172.21.15.93:0/622388458 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f35c80a11e0 msgr2=0x7f35c80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:59.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.967+0000 7f35e4ab6700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f35c80a11e0 0x7f35c80a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009c60 tx=0x7f35d0009260 comp rx=0 tx=0).stop 2023-11-16T16:47:59.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.967+0000 7f35e4ab6700 1 -- 172.21.15.93:0/622388458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0100ee0 msgr2=0x7f35e0198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:59.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.967+0000 7f35e4ab6700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0100ee0 0x7f35e0198ae0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35d40098e0 tx=0x7f35d400f690 comp rx=0 tx=0).stop 2023-11-16T16:47:59.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.967+0000 7f35e4ab6700 1 -- 172.21.15.93:0/622388458 shutdown_connections 2023-11-16T16:47:59.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.967+0000 7f35e4ab6700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f35c80a11e0 0x7f35c80a3690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:59.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.967+0000 7f35e4ab6700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35e01055e0 0x7f35e019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:59.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.967+0000 7f35e4ab6700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e0100540 0x7f35e01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:59.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.967+0000 7f35e4ab6700 1 --2- 172.21.15.93:0/622388458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0100ee0 0x7f35e0198ae0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:59.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.967+0000 7f35e4ab6700 1 -- 172.21.15.93:0/622388458 >> 172.21.15.93:0/622388458 conn(0x7f35e00fbdc0 msgr2=0x7f35e00fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:59.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.967+0000 7f35e4ab6700 1 -- 172.21.15.93:0/622388458 shutdown_connections 2023-11-16T16:47:59.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:47:59.967+0000 7f35e4ab6700 1 -- 172.21.15.93:0/622388458 wait complete. 2023-11-16T16:47:59.973 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-11-16T16:48:00.114 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:47:59 smithi093 ceph-mon[97863]: pgmap v326: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:00.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:47:59 smithi159 ceph-mon[131528]: pgmap v326: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:00.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:47:59 smithi125 ceph-mon[131076]: pgmap v326: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:00.429 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2023-11-16T16:47:58.537617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655355,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":423,"num_read_kb":325,"num_write":337,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":415,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":83780,"kb_used_data":14244,"kb_used_omap":0,"kb_used_meta":69440,"kb_avail":1124858044,"statfs":{"total":1151940427776,"available":1151854637056,"internally_reserved":0,"allocated":14585856,"data_stored":8088237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":71106560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":20,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002138"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616946+0000","last_change":"2023-11-16T16:46:37.150429+0000","last_active":"2023-11-16T16:47:53.616946+0000","last_peered":"2023-11-16T16:47:53.616946+0000","last_clean":"2023-11-16T16:47:53.616946+0000","last_became_active":"2023-11-16T16:46:37.150133+0000","last_became_peered":"2023-11-16T16:46:37.150133+0000","last_unstale":"2023-11-16T16:47:53.616946+0000","last_undegraded":"2023-11-16T16:47:53.616946+0000","last_fullsized":"2023-11-16T16:47:53.616946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:54:50.775465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339959+0000","last_change":"2023-11-16T16:46:37.155729+0000","last_active":"2023-11-16T16:47:53.339959+0000","last_peered":"2023-11-16T16:47:53.339959+0000","last_clean":"2023-11-16T16:47:53.339959+0000","last_became_active":"2023-11-16T16:46:37.155178+0000","last_became_peered":"2023-11-16T16:46:37.155178+0000","last_unstale":"2023-11-16T16:47:53.339959+0000","last_undegraded":"2023-11-16T16:47:53.339959+0000","last_fullsized":"2023-11-16T16:47:53.339959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:50:41.988915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.338806+0000","last_change":"2023-11-16T16:46:37.154669+0000","last_active":"2023-11-16T16:47:53.338806+0000","last_peered":"2023-11-16T16:47:53.338806+0000","last_clean":"2023-11-16T16:47:53.338806+0000","last_became_active":"2023-11-16T16:46:37.154331+0000","last_became_peered":"2023-11-16T16:46:37.154331+0000","last_unstale":"2023-11-16T16:47:53.338806+0000","last_undegraded":"2023-11-16T16:47:53.338806+0000","last_fullsized":"2023-11-16T16:47:53.338806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:31:56.646312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340670+0000","last_change":"2023-11-16T16:46:37.140627+0000","last_active":"2023-11-16T16:47:53.340670+0000","last_peered":"2023-11-16T16:47:53.340670+0000","last_clean":"2023-11-16T16:47:53.340670+0000","last_became_active":"2023-11-16T16:46:37.138386+0000","last_became_peered":"2023-11-16T16:46:37.138386+0000","last_unstale":"2023-11-16T16:47:53.340670+0000","last_undegraded":"2023-11-16T16:47:53.340670+0000","last_fullsized":"2023-11-16T16:47:53.340670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:06:28.111745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339677+0000","last_change":"2023-11-16T16:46:37.154046+0000","last_active":"2023-11-16T16:47:53.339677+0000","last_peered":"2023-11-16T16:47:53.339677+0000","last_clean":"2023-11-16T16:47:53.339677+0000","last_became_active":"2023-11-16T16:46:37.153677+0000","last_became_peered":"2023-11-16T16:46:37.153677+0000","last_unstale":"2023-11-16T16:47:53.339677+0000","last_undegraded":"2023-11-16T16:47:53.339677+0000","last_fullsized":"2023-11-16T16:47:53.339677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:09:53.042162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.394422+0000","last_change":"2023-11-16T16:46:37.153809+0000","last_active":"2023-11-16T16:47:53.394422+0000","last_peered":"2023-11-16T16:47:53.394422+0000","last_clean":"2023-11-16T16:47:53.394422+0000","last_became_active":"2023-11-16T16:46:37.153511+0000","last_became_peered":"2023-11-16T16:46:37.153511+0000","last_unstale":"2023-11-16T16:47:53.394422+0000","last_undegraded":"2023-11-16T16:47:53.394422+0000","last_fullsized":"2023-11-16T16:47:53.394422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:39:46.354193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616168+0000","last_change":"2023-11-16T16:46:37.142530+0000","last_active":"2023-11-16T16:47:53.616168+0000","last_peered":"2023-11-16T16:47:53.616168+0000","last_clean":"2023-11-16T16:47:53.616168+0000","last_became_active":"2023-11-16T16:46:37.142216+0000","last_became_peered":"2023-11-16T16:46:37.142216+0000","last_unstale":"2023-11-16T16:47:53.616168+0000","last_undegraded":"2023-11-16T16:47:53.616168+0000","last_fullsized":"2023-11-16T16:47:53.616168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:57:03.834528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343565+0000","last_change":"2023-11-16T16:46:37.151956+0000","last_active":"2023-11-16T16:47:53.343565+0000","last_peered":"2023-11-16T16:47:53.343565+0000","last_clean":"2023-11-16T16:47:53.343565+0000","last_became_active":"2023-11-16T16:46:37.149432+0000","last_became_peered":"2023-11-16T16:46:37.149432+0000","last_unstale":"2023-11-16T16:47:53.343565+0000","last_undegraded":"2023-11-16T16:47:53.343565+0000","last_fullsized":"2023-11-16T16:47:53.343565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:35:24.994585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.619544+0000","last_change":"2023-11-16T16:47:53.618515+0000","last_active":"2023-11-16T16:47:53.619544+0000","last_peered":"2023-11-16T16:47:53.619544+0000","last_clean":"2023-11-16T16:47:53.619544+0000","last_became_active":"2023-11-16T16:47:53.617627+0000","last_became_peered":"2023-11-16T16:47:53.617627+0000","last_unstale":"2023-11-16T16:47:53.619544+0000","last_undegraded":"2023-11-16T16:47:53.619544+0000","last_fullsized":"2023-11-16T16:47:53.619544+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:25:01.480977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.395019+0000","last_change":"2023-11-16T16:46:37.153895+0000","last_active":"2023-11-16T16:47:53.395019+0000","last_peered":"2023-11-16T16:47:53.395019+0000","last_clean":"2023-11-16T16:47:53.395019+0000","last_became_active":"2023-11-16T16:46:37.153491+0000","last_became_peered":"2023-11-16T16:46:37.153491+0000","last_unstale":"2023-11-16T16:47:53.395019+0000","last_undegraded":"2023-11-16T16:47:53.395019+0000","last_fullsized":"2023-11-16T16:47:53.395019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:22:53.710576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617078+0000","last_change":"2023-11-16T16:46:37.148036+0000","last_active":"2023-11-16T16:47:53.617078+0000","last_peered":"2023-11-16T16:47:53.617078+0000","last_clean":"2023-11-16T16:47:53.617078+0000","last_became_active":"2023-11-16T16:46:37.147372+0000","last_became_peered":"2023-11-16T16:46:37.147372+0000","last_unstale":"2023-11-16T16:47:53.617078+0000","last_undegraded":"2023-11-16T16:47:53.617078+0000","last_fullsized":"2023-11-16T16:47:53.617078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:57:26.086798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340480+0000","last_change":"2023-11-16T16:46:37.147120+0000","last_active":"2023-11-16T16:47:53.340480+0000","last_peered":"2023-11-16T16:47:53.340480+0000","last_clean":"2023-11-16T16:47:53.340480+0000","last_became_active":"2023-11-16T16:46:37.146615+0000","last_became_peered":"2023-11-16T16:46:37.146615+0000","last_unstale":"2023-11-16T16:47:53.340480+0000","last_undegraded":"2023-11-16T16:47:53.340480+0000","last_fullsized":"2023-11-16T16:47:53.340480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:04:59.299232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.346418+0000","last_change":"2023-11-16T16:46:37.152029+0000","last_active":"2023-11-16T16:47:53.346418+0000","last_peered":"2023-11-16T16:47:53.346418+0000","last_clean":"2023-11-16T16:47:53.346418+0000","last_became_active":"2023-11-16T16:46:37.150662+0000","last_became_peered":"2023-11-16T16:46:37.150662+0000","last_unstale":"2023-11-16T16:47:53.346418+0000","last_undegraded":"2023-11-16T16:47:53.346418+0000","last_fullsized":"2023-11-16T16:47:53.346418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:36:09.281786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339317+0000","last_change":"2023-11-16T16:46:37.160562+0000","last_active":"2023-11-16T16:47:53.339317+0000","last_peered":"2023-11-16T16:47:53.339317+0000","last_clean":"2023-11-16T16:47:53.339317+0000","last_became_active":"2023-11-16T16:46:37.158175+0000","last_became_peered":"2023-11-16T16:46:37.158175+0000","last_unstale":"2023-11-16T16:47:53.339317+0000","last_undegraded":"2023-11-16T16:47:53.339317+0000","last_fullsized":"2023-11-16T16:47:53.339317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:29:39.193105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.338703+0000","last_change":"2023-11-16T16:46:37.156381+0000","last_active":"2023-11-16T16:47:53.338703+0000","last_peered":"2023-11-16T16:47:53.338703+0000","last_clean":"2023-11-16T16:47:53.338703+0000","last_became_active":"2023-11-16T16:46:37.150366+0000","last_became_peered":"2023-11-16T16:46:37.150366+0000","last_unstale":"2023-11-16T16:47:53.338703+0000","last_undegraded":"2023-11-16T16:47:53.338703+0000","last_fullsized":"2023-11-16T16:47:53.338703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:49:26.041804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615267+0000","last_change":"2023-11-16T16:46:37.158070+0000","last_active":"2023-11-16T16:47:53.615267+0000","last_peered":"2023-11-16T16:47:53.615267+0000","last_clean":"2023-11-16T16:47:53.615267+0000","last_became_active":"2023-11-16T16:46:37.157755+0000","last_became_peered":"2023-11-16T16:46:37.157755+0000","last_unstale":"2023-11-16T16:47:53.615267+0000","last_undegraded":"2023-11-16T16:47:53.615267+0000","last_fullsized":"2023-11-16T16:47:53.615267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:18:18.304516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617095+0000","last_change":"2023-11-16T16:46:37.162616+0000","last_active":"2023-11-16T16:47:53.617095+0000","last_peered":"2023-11-16T16:47:53.617095+0000","last_clean":"2023-11-16T16:47:53.617095+0000","last_became_active":"2023-11-16T16:46:37.162281+0000","last_became_peered":"2023-11-16T16:46:37.162281+0000","last_unstale":"2023-11-16T16:47:53.617095+0000","last_undegraded":"2023-11-16T16:47:53.617095+0000","last_fullsized":"2023-11-16T16:47:53.617095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:47:35.293940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.348022+0000","last_change":"2023-11-16T16:46:37.150337+0000","last_active":"2023-11-16T16:47:53.348022+0000","last_peered":"2023-11-16T16:47:53.348022+0000","last_clean":"2023-11-16T16:47:53.348022+0000","last_became_active":"2023-11-16T16:46:37.148826+0000","last_became_peered":"2023-11-16T16:46:37.148826+0000","last_unstale":"2023-11-16T16:47:53.348022+0000","last_undegraded":"2023-11-16T16:47:53.348022+0000","last_fullsized":"2023-11-16T16:47:53.348022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:26:33.021039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616437+0000","last_change":"2023-11-16T16:46:37.150253+0000","last_active":"2023-11-16T16:47:53.616437+0000","last_peered":"2023-11-16T16:47:53.616437+0000","last_clean":"2023-11-16T16:47:53.616437+0000","last_became_active":"2023-11-16T16:46:37.149962+0000","last_became_peered":"2023-11-16T16:46:37.149962+0000","last_unstale":"2023-11-16T16:47:53.616437+0000","last_undegraded":"2023-11-16T16:47:53.616437+0000","last_fullsized":"2023-11-16T16:47:53.616437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:39:37.347781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616251+0000","last_change":"2023-11-16T16:46:41.177888+0000","last_active":"2023-11-16T16:47:53.616251+0000","last_peered":"2023-11-16T16:47:53.616251+0000","last_clean":"2023-11-16T16:47:53.616251+0000","last_became_active":"2023-11-16T16:46:41.177442+0000","last_became_peered":"2023-11-16T16:46:41.177442+0000","last_unstale":"2023-11-16T16:47:53.616251+0000","last_undegraded":"2023-11-16T16:47:53.616251+0000","last_fullsized":"2023-11-16T16:47:53.616251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:53:10.830975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.347927+0000","last_change":"2023-11-16T16:46:36.595769+0000","last_active":"2023-11-16T16:47:53.347927+0000","last_peered":"2023-11-16T16:47:53.347927+0000","last_clean":"2023-11-16T16:47:53.347927+0000","last_became_active":"2023-11-16T16:46:36.593780+0000","last_became_peered":"2023-11-16T16:46:36.593780+0000","last_unstale":"2023-11-16T16:47:53.347927+0000","last_undegraded":"2023-11-16T16:47:53.347927+0000","last_fullsized":"2023-11-16T16:47:53.347927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:01:41.376943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343701+0000","last_change":"2023-11-16T16:46:37.146683+0000","last_active":"2023-11-16T16:47:53.343701+0000","last_peered":"2023-11-16T16:47:53.343701+0000","last_clean":"2023-11-16T16:47:53.343701+0000","last_became_active":"2023-11-16T16:46:37.146363+0000","last_became_peered":"2023-11-16T16:46:37.146363+0000","last_unstale":"2023-11-16T16:47:53.343701+0000","last_undegraded":"2023-11-16T16:47:53.343701+0000","last_fullsized":"2023-11-16T16:47:53.343701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:29:13.834403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339895+0000","last_change":"2023-11-16T16:46:41.174942+0000","last_active":"2023-11-16T16:47:53.339895+0000","last_peered":"2023-11-16T16:47:53.339895+0000","last_clean":"2023-11-16T16:47:53.339895+0000","last_became_active":"2023-11-16T16:46:41.174483+0000","last_became_peered":"2023-11-16T16:46:41.174483+0000","last_unstale":"2023-11-16T16:47:53.339895+0000","last_undegraded":"2023-11-16T16:47:53.339895+0000","last_fullsized":"2023-11-16T16:47:53.339895+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:18:17.625371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343903+0000","last_change":"2023-11-16T16:46:36.607478+0000","last_active":"2023-11-16T16:47:53.343903+0000","last_peered":"2023-11-16T16:47:53.343903+0000","last_clean":"2023-11-16T16:47:53.343903+0000","last_became_active":"2023-11-16T16:46:36.607137+0000","last_became_peered":"2023-11-16T16:46:36.607137+0000","last_unstale":"2023-11-16T16:47:53.343903+0000","last_undegraded":"2023-11-16T16:47:53.343903+0000","last_fullsized":"2023-11-16T16:47:53.343903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:01:51.466667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339746+0000","last_change":"2023-11-16T16:46:37.148576+0000","last_active":"2023-11-16T16:47:53.339746+0000","last_peered":"2023-11-16T16:47:53.339746+0000","last_clean":"2023-11-16T16:47:53.339746+0000","last_became_active":"2023-11-16T16:46:37.148284+0000","last_became_peered":"2023-11-16T16:46:37.148284+0000","last_unstale":"2023-11-16T16:47:53.339746+0000","last_undegraded":"2023-11-16T16:47:53.339746+0000","last_fullsized":"2023-11-16T16:47:53.339746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:07:30.298369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340210+0000","last_change":"2023-11-16T16:46:41.178717+0000","last_active":"2023-11-16T16:47:53.340210+0000","last_peered":"2023-11-16T16:47:53.340210+0000","last_clean":"2023-11-16T16:47:53.340210+0000","last_became_active":"2023-11-16T16:46:41.178350+0000","last_became_peered":"2023-11-16T16:46:41.178350+0000","last_unstale":"2023-11-16T16:47:53.340210+0000","last_undegraded":"2023-11-16T16:47:53.340210+0000","last_fullsized":"2023-11-16T16:47:53.340210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:27:43.240599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616800+0000","last_change":"2023-11-16T16:46:52.639272+0000","last_active":"2023-11-16T16:47:53.616800+0000","last_peered":"2023-11-16T16:47:53.616800+0000","last_clean":"2023-11-16T16:47:53.616800+0000","last_became_active":"2023-11-16T16:46:52.338319+0000","last_became_peered":"2023-11-16T16:46:52.338319+0000","last_unstale":"2023-11-16T16:47:53.616800+0000","last_undegraded":"2023-11-16T16:47:53.616800+0000","last_fullsized":"2023-11-16T16:47:53.616800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:38:42.664671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615916+0000","last_change":"2023-11-16T16:46:37.153404+0000","last_active":"2023-11-16T16:47:53.615916+0000","last_peered":"2023-11-16T16:47:53.615916+0000","last_clean":"2023-11-16T16:47:53.615916+0000","last_became_active":"2023-11-16T16:46:37.153064+0000","last_became_peered":"2023-11-16T16:46:37.153064+0000","last_unstale":"2023-11-16T16:47:53.615916+0000","last_undegraded":"2023-11-16T16:47:53.615916+0000","last_fullsized":"2023-11-16T16:47:53.615916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:58:18.745061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339998+0000","last_change":"2023-11-16T16:46:41.163307+0000","last_active":"2023-11-16T16:47:53.339998+0000","last_peered":"2023-11-16T16:47:53.339998+0000","last_clean":"2023-11-16T16:47:53.339998+0000","last_became_active":"2023-11-16T16:46:41.162822+0000","last_became_peered":"2023-11-16T16:46:41.162822+0000","last_unstale":"2023-11-16T16:47:53.339998+0000","last_undegraded":"2023-11-16T16:47:53.339998+0000","last_fullsized":"2023-11-16T16:47:53.339998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:24:50.552900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339961+0000","last_change":"2023-11-16T16:46:36.595905+0000","last_active":"2023-11-16T16:47:53.339961+0000","last_peered":"2023-11-16T16:47:53.339961+0000","last_clean":"2023-11-16T16:47:53.339961+0000","last_became_active":"2023-11-16T16:46:36.595403+0000","last_became_peered":"2023-11-16T16:46:36.595403+0000","last_unstale":"2023-11-16T16:47:53.339961+0000","last_undegraded":"2023-11-16T16:47:53.339961+0000","last_fullsized":"2023-11-16T16:47:53.339961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:35:19.238263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343480+0000","last_change":"2023-11-16T16:46:52.334655+0000","last_active":"2023-11-16T16:47:53.343480+0000","last_peered":"2023-11-16T16:47:53.343480+0000","last_clean":"2023-11-16T16:47:53.343480+0000","last_became_active":"2023-11-16T16:46:52.334345+0000","last_became_peered":"2023-11-16T16:46:52.334345+0000","last_unstale":"2023-11-16T16:47:53.343480+0000","last_undegraded":"2023-11-16T16:47:53.343480+0000","last_fullsized":"2023-11-16T16:47:53.343480+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:23:02.755769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616229+0000","last_change":"2023-11-16T16:46:41.169041+0000","last_active":"2023-11-16T16:47:53.616229+0000","last_peered":"2023-11-16T16:47:53.616229+0000","last_clean":"2023-11-16T16:47:53.616229+0000","last_became_active":"2023-11-16T16:46:41.168589+0000","last_became_peered":"2023-11-16T16:46:41.168589+0000","last_unstale":"2023-11-16T16:47:53.616229+0000","last_undegraded":"2023-11-16T16:47:53.616229+0000","last_fullsized":"2023-11-16T16:47:53.616229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:46:36.805567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":82,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.395661+0000","last_change":"2023-11-16T16:46:36.595442+0000","last_active":"2023-11-16T16:47:53.395661+0000","last_peered":"2023-11-16T16:47:53.395661+0000","last_clean":"2023-11-16T16:47:53.395661+0000","last_became_active":"2023-11-16T16:46:36.595122+0000","last_became_peered":"2023-11-16T16:46:36.595122+0000","last_unstale":"2023-11-16T16:47:53.395661+0000","last_undegraded":"2023-11-16T16:47:53.395661+0000","last_fullsized":"2023-11-16T16:47:53.395661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:26:46.774071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.618090+0000","last_change":"2023-11-16T16:47:53.618090+0000","last_active":"2023-11-16T16:47:53.618090+0000","last_peered":"2023-11-16T16:47:53.618090+0000","last_clean":"2023-11-16T16:47:53.618090+0000","last_became_active":"2023-11-16T16:47:53.617686+0000","last_became_peered":"2023-11-16T16:47:53.617686+0000","last_unstale":"2023-11-16T16:47:53.618090+0000","last_undegraded":"2023-11-16T16:47:53.618090+0000","last_fullsized":"2023-11-16T16:47:53.618090+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:44:16.967004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616334+0000","last_change":"2023-11-16T16:46:41.185001+0000","last_active":"2023-11-16T16:47:53.616334+0000","last_peered":"2023-11-16T16:47:53.616334+0000","last_clean":"2023-11-16T16:47:53.616334+0000","last_became_active":"2023-11-16T16:46:41.184603+0000","last_became_peered":"2023-11-16T16:46:41.184603+0000","last_unstale":"2023-11-16T16:47:53.616334+0000","last_undegraded":"2023-11-16T16:47:53.616334+0000","last_fullsized":"2023-11-16T16:47:53.616334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:56:54.738072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.394424+0000","last_change":"2023-11-16T16:46:36.611365+0000","last_active":"2023-11-16T16:47:53.394424+0000","last_peered":"2023-11-16T16:47:53.394424+0000","last_clean":"2023-11-16T16:47:53.394424+0000","last_became_active":"2023-11-16T16:46:36.611006+0000","last_became_peered":"2023-11-16T16:46:36.611006+0000","last_unstale":"2023-11-16T16:47:53.394424+0000","last_undegraded":"2023-11-16T16:47:53.394424+0000","last_fullsized":"2023-11-16T16:47:53.394424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:24:21.225507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.348823+0000","last_change":"2023-11-16T16:47:53.348823+0000","last_active":"2023-11-16T16:47:53.348823+0000","last_peered":"2023-11-16T16:47:53.348823+0000","last_clean":"2023-11-16T16:47:53.348823+0000","last_became_active":"2023-11-16T16:47:53.348523+0000","last_became_peered":"2023-11-16T16:47:53.348523+0000","last_unstale":"2023-11-16T16:47:53.348823+0000","last_undegraded":"2023-11-16T16:47:53.348823+0000","last_fullsized":"2023-11-16T16:47:53.348823+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:05:08.796096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615900+0000","last_change":"2023-11-16T16:46:41.182374+0000","last_active":"2023-11-16T16:47:53.615900+0000","last_peered":"2023-11-16T16:47:53.615900+0000","last_clean":"2023-11-16T16:47:53.615900+0000","last_became_active":"2023-11-16T16:46:41.181977+0000","last_became_peered":"2023-11-16T16:46:41.181977+0000","last_unstale":"2023-11-16T16:47:53.615900+0000","last_undegraded":"2023-11-16T16:47:53.615900+0000","last_fullsized":"2023-11-16T16:47:53.615900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:42:20.748527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.346218+0000","last_change":"2023-11-16T16:46:36.607647+0000","last_active":"2023-11-16T16:47:53.346218+0000","last_peered":"2023-11-16T16:47:53.346218+0000","last_clean":"2023-11-16T16:47:53.346218+0000","last_became_active":"2023-11-16T16:46:36.607282+0000","last_became_peered":"2023-11-16T16:46:36.607282+0000","last_unstale":"2023-11-16T16:47:53.346218+0000","last_undegraded":"2023-11-16T16:47:53.346218+0000","last_fullsized":"2023-11-16T16:47:53.346218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:27:41.957383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343442+0000","last_change":"2023-11-16T16:46:37.152659+0000","last_active":"2023-11-16T16:47:53.343442+0000","last_peered":"2023-11-16T16:47:53.343442+0000","last_clean":"2023-11-16T16:47:53.343442+0000","last_became_active":"2023-11-16T16:46:37.152313+0000","last_became_peered":"2023-11-16T16:46:37.152313+0000","last_unstale":"2023-11-16T16:47:53.343442+0000","last_undegraded":"2023-11-16T16:47:53.343442+0000","last_fullsized":"2023-11-16T16:47:53.343442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:41:24.243494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339825+0000","last_change":"2023-11-16T16:46:41.167312+0000","last_active":"2023-11-16T16:47:53.339825+0000","last_peered":"2023-11-16T16:47:53.339825+0000","last_clean":"2023-11-16T16:47:53.339825+0000","last_became_active":"2023-11-16T16:46:41.166211+0000","last_became_peered":"2023-11-16T16:46:41.166211+0000","last_unstale":"2023-11-16T16:47:53.339825+0000","last_undegraded":"2023-11-16T16:47:53.339825+0000","last_fullsized":"2023-11-16T16:47:53.339825+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:04:27.032057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342694+0000","last_change":"2023-11-16T16:46:36.595888+0000","last_active":"2023-11-16T16:47:53.342694+0000","last_peered":"2023-11-16T16:47:53.342694+0000","last_clean":"2023-11-16T16:47:53.342694+0000","last_became_active":"2023-11-16T16:46:36.595602+0000","last_became_peered":"2023-11-16T16:46:36.595602+0000","last_unstale":"2023-11-16T16:47:53.342694+0000","last_undegraded":"2023-11-16T16:47:53.342694+0000","last_fullsized":"2023-11-16T16:47:53.342694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:04:33.124818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.344885+0000","last_change":"2023-11-16T16:46:37.154431+0000","last_active":"2023-11-16T16:47:53.344885+0000","last_peered":"2023-11-16T16:47:53.344885+0000","last_clean":"2023-11-16T16:47:53.344885+0000","last_became_active":"2023-11-16T16:46:37.154106+0000","last_became_peered":"2023-11-16T16:46:37.154106+0000","last_unstale":"2023-11-16T16:47:53.344885+0000","last_undegraded":"2023-11-16T16:47:53.344885+0000","last_fullsized":"2023-11-16T16:47:53.344885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:35:20.330822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343030+0000","last_change":"2023-11-16T16:46:41.168644+0000","last_active":"2023-11-16T16:47:53.343030+0000","last_peered":"2023-11-16T16:47:53.343030+0000","last_clean":"2023-11-16T16:47:53.343030+0000","last_became_active":"2023-11-16T16:46:41.168121+0000","last_became_peered":"2023-11-16T16:46:41.168121+0000","last_unstale":"2023-11-16T16:47:53.343030+0000","last_undegraded":"2023-11-16T16:47:53.343030+0000","last_fullsized":"2023-11-16T16:47:53.343030+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:37:47.753436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":77,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.395060+0000","last_change":"2023-11-16T16:46:36.597293+0000","last_active":"2023-11-16T16:47:53.395060+0000","last_peered":"2023-11-16T16:47:53.395060+0000","last_clean":"2023-11-16T16:47:53.395060+0000","last_became_active":"2023-11-16T16:46:36.596992+0000","last_became_peered":"2023-11-16T16:46:36.596992+0000","last_unstale":"2023-11-16T16:47:53.395060+0000","last_undegraded":"2023-11-16T16:47:53.395060+0000","last_fullsized":"2023-11-16T16:47:53.395060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:10:25.375233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616418+0000","last_change":"2023-11-16T16:46:37.149339+0000","last_active":"2023-11-16T16:47:53.616418+0000","last_peered":"2023-11-16T16:47:53.616418+0000","last_clean":"2023-11-16T16:47:53.616418+0000","last_became_active":"2023-11-16T16:46:37.149000+0000","last_became_peered":"2023-11-16T16:46:37.149000+0000","last_unstale":"2023-11-16T16:47:53.616418+0000","last_undegraded":"2023-11-16T16:47:53.616418+0000","last_fullsized":"2023-11-16T16:47:53.616418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:55:38.165691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339296+0000","last_change":"2023-11-16T16:46:52.338731+0000","last_active":"2023-11-16T16:47:53.339296+0000","last_peered":"2023-11-16T16:47:53.339296+0000","last_clean":"2023-11-16T16:47:53.339296+0000","last_became_active":"2023-11-16T16:46:52.338254+0000","last_became_peered":"2023-11-16T16:46:52.338254+0000","last_unstale":"2023-11-16T16:47:53.339296+0000","last_undegraded":"2023-11-16T16:47:53.339296+0000","last_fullsized":"2023-11-16T16:47:53.339296+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:51:39.487241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617194+0000","last_change":"2023-11-16T16:46:36.595407+0000","last_active":"2023-11-16T16:47:53.617194+0000","last_peered":"2023-11-16T16:47:53.617194+0000","last_clean":"2023-11-16T16:47:53.617194+0000","last_became_active":"2023-11-16T16:46:36.595132+0000","last_became_peered":"2023-11-16T16:46:36.595132+0000","last_unstale":"2023-11-16T16:47:53.617194+0000","last_undegraded":"2023-11-16T16:47:53.617194+0000","last_fullsized":"2023-11-16T16:47:53.617194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:16:48.765412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.619303+0000","last_change":"2023-11-16T16:47:53.619303+0000","last_active":"2023-11-16T16:47:53.619303+0000","last_peered":"2023-11-16T16:47:53.619303+0000","last_clean":"2023-11-16T16:47:53.619303+0000","last_became_active":"2023-11-16T16:47:53.618438+0000","last_became_peered":"2023-11-16T16:47:53.618438+0000","last_unstale":"2023-11-16T16:47:53.619303+0000","last_undegraded":"2023-11-16T16:47:53.619303+0000","last_fullsized":"2023-11-16T16:47:53.619303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:41:13.994013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340737+0000","last_change":"2023-11-16T16:46:52.339033+0000","last_active":"2023-11-16T16:47:53.340737+0000","last_peered":"2023-11-16T16:47:53.340737+0000","last_clean":"2023-11-16T16:47:53.340737+0000","last_became_active":"2023-11-16T16:46:52.338451+0000","last_became_peered":"2023-11-16T16:46:52.338451+0000","last_unstale":"2023-11-16T16:47:53.340737+0000","last_undegraded":"2023-11-16T16:47:53.340737+0000","last_fullsized":"2023-11-16T16:47:53.340737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:54:56.472106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343009+0000","last_change":"2023-11-16T16:46:36.596779+0000","last_active":"2023-11-16T16:47:53.343009+0000","last_peered":"2023-11-16T16:47:53.343009+0000","last_clean":"2023-11-16T16:47:53.343009+0000","last_became_active":"2023-11-16T16:46:36.596460+0000","last_became_peered":"2023-11-16T16:46:36.596460+0000","last_unstale":"2023-11-16T16:47:53.343009+0000","last_undegraded":"2023-11-16T16:47:53.343009+0000","last_fullsized":"2023-11-16T16:47:53.343009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:55:26.723679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616040+0000","last_change":"2023-11-16T16:46:52.427912+0000","last_active":"2023-11-16T16:47:53.616040+0000","last_peered":"2023-11-16T16:47:53.616040+0000","last_clean":"2023-11-16T16:47:53.616040+0000","last_became_active":"2023-11-16T16:46:52.427426+0000","last_became_peered":"2023-11-16T16:46:52.427426+0000","last_unstale":"2023-11-16T16:47:53.616040+0000","last_undegraded":"2023-11-16T16:47:53.616040+0000","last_fullsized":"2023-11-16T16:47:53.616040+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:11:47.521498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343532+0000","last_change":"2023-11-16T16:46:41.177483+0000","last_active":"2023-11-16T16:47:53.343532+0000","last_peered":"2023-11-16T16:47:53.343532+0000","last_clean":"2023-11-16T16:47:53.343532+0000","last_became_active":"2023-11-16T16:46:41.173165+0000","last_became_peered":"2023-11-16T16:46:41.173165+0000","last_unstale":"2023-11-16T16:47:53.343532+0000","last_undegraded":"2023-11-16T16:47:53.343532+0000","last_fullsized":"2023-11-16T16:47:53.343532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:16:58.604325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'32","reported_seq":158,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.394792+0000","last_change":"2023-11-16T16:46:36.598279+0000","last_active":"2023-11-16T16:47:53.394792+0000","last_peered":"2023-11-16T16:47:53.394792+0000","last_clean":"2023-11-16T16:47:53.394792+0000","last_became_active":"2023-11-16T16:46:36.597975+0000","last_became_peered":"2023-11-16T16:46:36.597975+0000","last_unstale":"2023-11-16T16:47:53.394792+0000","last_undegraded":"2023-11-16T16:47:53.394792+0000","last_fullsized":"2023-11-16T16:47:53.394792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:39:42.505182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60837,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":96,"num_read_kb":107,"num_write":33,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340272+0000","last_change":"2023-11-16T16:46:37.143014+0000","last_active":"2023-11-16T16:47:53.340272+0000","last_peered":"2023-11-16T16:47:53.340272+0000","last_clean":"2023-11-16T16:47:53.340272+0000","last_became_active":"2023-11-16T16:46:37.142662+0000","last_became_peered":"2023-11-16T16:46:37.142662+0000","last_unstale":"2023-11-16T16:47:53.340272+0000","last_undegraded":"2023-11-16T16:47:53.340272+0000","last_fullsized":"2023-11-16T16:47:53.340272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:28:38.845881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343681+0000","last_change":"2023-11-16T16:46:41.169921+0000","last_active":"2023-11-16T16:47:53.343681+0000","last_peered":"2023-11-16T16:47:53.343681+0000","last_clean":"2023-11-16T16:47:53.343681+0000","last_became_active":"2023-11-16T16:46:41.169037+0000","last_became_peered":"2023-11-16T16:46:41.169037+0000","last_unstale":"2023-11-16T16:47:53.343681+0000","last_undegraded":"2023-11-16T16:47:53.343681+0000","last_fullsized":"2023-11-16T16:47:53.343681+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:46:12.638029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616096+0000","last_change":"2023-11-16T16:46:36.596159+0000","last_active":"2023-11-16T16:47:53.616096+0000","last_peered":"2023-11-16T16:47:53.616096+0000","last_clean":"2023-11-16T16:47:53.616096+0000","last_became_active":"2023-11-16T16:46:36.595847+0000","last_became_peered":"2023-11-16T16:46:36.595847+0000","last_unstale":"2023-11-16T16:47:53.616096+0000","last_undegraded":"2023-11-16T16:47:53.616096+0000","last_fullsized":"2023-11-16T16:47:53.616096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:36:29.758981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.344201+0000","last_change":"2023-11-16T16:46:37.156046+0000","last_active":"2023-11-16T16:47:53.344201+0000","last_peered":"2023-11-16T16:47:53.344201+0000","last_clean":"2023-11-16T16:47:53.344201+0000","last_became_active":"2023-11-16T16:46:37.155657+0000","last_became_peered":"2023-11-16T16:46:37.155657+0000","last_unstale":"2023-11-16T16:47:53.344201+0000","last_undegraded":"2023-11-16T16:47:53.344201+0000","last_fullsized":"2023-11-16T16:47:53.344201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:18:49.240097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615915+0000","last_change":"2023-11-16T16:46:41.182485+0000","last_active":"2023-11-16T16:47:53.615915+0000","last_peered":"2023-11-16T16:47:53.615915+0000","last_clean":"2023-11-16T16:47:53.615915+0000","last_became_active":"2023-11-16T16:46:41.181990+0000","last_became_peered":"2023-11-16T16:46:41.181990+0000","last_unstale":"2023-11-16T16:47:53.615915+0000","last_undegraded":"2023-11-16T16:47:53.615915+0000","last_fullsized":"2023-11-16T16:47:53.615915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:34:53.010727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343074+0000","last_change":"2023-11-16T16:46:36.608501+0000","last_active":"2023-11-16T16:47:53.343074+0000","last_peered":"2023-11-16T16:47:53.343074+0000","last_clean":"2023-11-16T16:47:53.343074+0000","last_became_active":"2023-11-16T16:46:36.608210+0000","last_became_peered":"2023-11-16T16:46:36.608210+0000","last_unstale":"2023-11-16T16:47:53.343074+0000","last_undegraded":"2023-11-16T16:47:53.343074+0000","last_fullsized":"2023-11-16T16:47:53.343074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:59:43.646005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616810+0000","last_change":"2023-11-16T16:46:37.155431+0000","last_active":"2023-11-16T16:47:53.616810+0000","last_peered":"2023-11-16T16:47:53.616810+0000","last_clean":"2023-11-16T16:47:53.616810+0000","last_became_active":"2023-11-16T16:46:37.155106+0000","last_became_peered":"2023-11-16T16:46:37.155106+0000","last_unstale":"2023-11-16T16:47:53.616810+0000","last_undegraded":"2023-11-16T16:47:53.616810+0000","last_fullsized":"2023-11-16T16:47:53.616810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:37:09.743670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.348242+0000","last_change":"2023-11-16T16:46:41.180903+0000","last_active":"2023-11-16T16:47:53.348242+0000","last_peered":"2023-11-16T16:47:53.348242+0000","last_clean":"2023-11-16T16:47:53.348242+0000","last_became_active":"2023-11-16T16:46:41.179842+0000","last_became_peered":"2023-11-16T16:46:41.179842+0000","last_unstale":"2023-11-16T16:47:53.348242+0000","last_undegraded":"2023-11-16T16:47:53.348242+0000","last_fullsized":"2023-11-16T16:47:53.348242+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:38:02.957328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":84,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:55.392387+0000","last_change":"2023-11-16T16:46:36.592436+0000","last_active":"2023-11-16T16:47:55.392387+0000","last_peered":"2023-11-16T16:47:55.392387+0000","last_clean":"2023-11-16T16:47:55.392387+0000","last_became_active":"2023-11-16T16:46:36.592078+0000","last_became_peered":"2023-11-16T16:46:36.592078+0000","last_unstale":"2023-11-16T16:47:55.392387+0000","last_undegraded":"2023-11-16T16:47:55.392387+0000","last_fullsized":"2023-11-16T16:47:55.392387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:09:44.438155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616011+0000","last_change":"2023-11-16T16:46:37.153071+0000","last_active":"2023-11-16T16:47:53.616011+0000","last_peered":"2023-11-16T16:47:53.616011+0000","last_clean":"2023-11-16T16:47:53.616011+0000","last_became_active":"2023-11-16T16:46:37.152742+0000","last_became_peered":"2023-11-16T16:46:37.152742+0000","last_unstale":"2023-11-16T16:47:53.616011+0000","last_undegraded":"2023-11-16T16:47:53.616011+0000","last_fullsized":"2023-11-16T16:47:53.616011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:13:23.845741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343289+0000","last_change":"2023-11-16T16:46:41.176434+0000","last_active":"2023-11-16T16:47:53.343289+0000","last_peered":"2023-11-16T16:47:53.343289+0000","last_clean":"2023-11-16T16:47:53.343289+0000","last_became_active":"2023-11-16T16:46:41.175771+0000","last_became_peered":"2023-11-16T16:46:41.175771+0000","last_unstale":"2023-11-16T16:47:53.343289+0000","last_undegraded":"2023-11-16T16:47:53.343289+0000","last_fullsized":"2023-11-16T16:47:53.343289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:41:28.313123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339373+0000","last_change":"2023-11-16T16:46:36.594148+0000","last_active":"2023-11-16T16:47:53.339373+0000","last_peered":"2023-11-16T16:47:53.339373+0000","last_clean":"2023-11-16T16:47:53.339373+0000","last_became_active":"2023-11-16T16:46:36.593437+0000","last_became_peered":"2023-11-16T16:46:36.593437+0000","last_unstale":"2023-11-16T16:47:53.339373+0000","last_undegraded":"2023-11-16T16:47:53.339373+0000","last_fullsized":"2023-11-16T16:47:53.339373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:39:28.474515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343308+0000","last_change":"2023-11-16T16:46:37.149044+0000","last_active":"2023-11-16T16:47:53.343308+0000","last_peered":"2023-11-16T16:47:53.343308+0000","last_clean":"2023-11-16T16:47:53.343308+0000","last_became_active":"2023-11-16T16:46:37.147215+0000","last_became_peered":"2023-11-16T16:46:37.147215+0000","last_unstale":"2023-11-16T16:47:53.343308+0000","last_undegraded":"2023-11-16T16:47:53.343308+0000","last_fullsized":"2023-11-16T16:47:53.343308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:52:29.239788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342546+0000","last_change":"2023-11-16T16:46:41.168754+0000","last_active":"2023-11-16T16:47:53.342546+0000","last_peered":"2023-11-16T16:47:53.342546+0000","last_clean":"2023-11-16T16:47:53.342546+0000","last_became_active":"2023-11-16T16:46:41.168056+0000","last_became_peered":"2023-11-16T16:46:41.168056+0000","last_unstale":"2023-11-16T16:47:53.342546+0000","last_undegraded":"2023-11-16T16:47:53.342546+0000","last_fullsized":"2023-11-16T16:47:53.342546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:06:38.901207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340338+0000","last_change":"2023-11-16T16:46:36.594892+0000","last_active":"2023-11-16T16:47:53.340338+0000","last_peered":"2023-11-16T16:47:53.340338+0000","last_clean":"2023-11-16T16:47:53.340338+0000","last_became_active":"2023-11-16T16:46:36.594557+0000","last_became_peered":"2023-11-16T16:46:36.594557+0000","last_unstale":"2023-11-16T16:47:53.340338+0000","last_undegraded":"2023-11-16T16:47:53.340338+0000","last_fullsized":"2023-11-16T16:47:53.340338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:31:22.201870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616072+0000","last_change":"2023-11-16T16:46:37.148203+0000","last_active":"2023-11-16T16:47:53.616072+0000","last_peered":"2023-11-16T16:47:53.616072+0000","last_clean":"2023-11-16T16:47:53.616072+0000","last_became_active":"2023-11-16T16:46:37.147846+0000","last_became_peered":"2023-11-16T16:46:37.147846+0000","last_unstale":"2023-11-16T16:47:53.616072+0000","last_undegraded":"2023-11-16T16:47:53.616072+0000","last_fullsized":"2023-11-16T16:47:53.616072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:01:32.063666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339599+0000","last_change":"2023-11-16T16:46:53.046150+0000","last_active":"2023-11-16T16:47:53.339599+0000","last_peered":"2023-11-16T16:47:53.339599+0000","last_clean":"2023-11-16T16:47:53.339599+0000","last_became_active":"2023-11-16T16:46:53.045619+0000","last_became_peered":"2023-11-16T16:46:53.045619+0000","last_unstale":"2023-11-16T16:47:53.339599+0000","last_undegraded":"2023-11-16T16:47:53.339599+0000","last_fullsized":"2023-11-16T16:47:53.339599+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:58:25.709808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.347053+0000","last_change":"2023-11-16T16:46:36.600779+0000","last_active":"2023-11-16T16:47:53.347053+0000","last_peered":"2023-11-16T16:47:53.347053+0000","last_clean":"2023-11-16T16:47:53.347053+0000","last_became_active":"2023-11-16T16:46:36.598664+0000","last_became_peered":"2023-11-16T16:46:36.598664+0000","last_unstale":"2023-11-16T16:47:53.347053+0000","last_undegraded":"2023-11-16T16:47:53.347053+0000","last_fullsized":"2023-11-16T16:47:53.347053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:41:36.477468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.394758+0000","last_change":"2023-11-16T16:46:37.155694+0000","last_active":"2023-11-16T16:47:53.394758+0000","last_peered":"2023-11-16T16:47:53.394758+0000","last_clean":"2023-11-16T16:47:53.394758+0000","last_became_active":"2023-11-16T16:46:37.155105+0000","last_became_peered":"2023-11-16T16:46:37.155105+0000","last_unstale":"2023-11-16T16:47:53.394758+0000","last_undegraded":"2023-11-16T16:47:53.394758+0000","last_fullsized":"2023-11-16T16:47:53.394758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:23:05.949449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.345451+0000","last_change":"2023-11-16T16:46:41.172161+0000","last_active":"2023-11-16T16:47:53.345451+0000","last_peered":"2023-11-16T16:47:53.345451+0000","last_clean":"2023-11-16T16:47:53.345451+0000","last_became_active":"2023-11-16T16:46:41.169661+0000","last_became_peered":"2023-11-16T16:46:41.169661+0000","last_unstale":"2023-11-16T16:47:53.345451+0000","last_undegraded":"2023-11-16T16:47:53.345451+0000","last_fullsized":"2023-11-16T16:47:53.345451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:02:08.887753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616617+0000","last_change":"2023-11-16T16:46:36.611376+0000","last_active":"2023-11-16T16:47:53.616617+0000","last_peered":"2023-11-16T16:47:53.616617+0000","last_clean":"2023-11-16T16:47:53.616617+0000","last_became_active":"2023-11-16T16:46:36.611045+0000","last_became_peered":"2023-11-16T16:46:36.611045+0000","last_unstale":"2023-11-16T16:47:53.616617+0000","last_undegraded":"2023-11-16T16:47:53.616617+0000","last_fullsized":"2023-11-16T16:47:53.616617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:39:01.262829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.394469+0000","last_change":"2023-11-16T16:46:37.164850+0000","last_active":"2023-11-16T16:47:53.394469+0000","last_peered":"2023-11-16T16:47:53.394469+0000","last_clean":"2023-11-16T16:47:53.394469+0000","last_became_active":"2023-11-16T16:46:37.164532+0000","last_became_peered":"2023-11-16T16:46:37.164532+0000","last_unstale":"2023-11-16T16:47:53.394469+0000","last_undegraded":"2023-11-16T16:47:53.394469+0000","last_fullsized":"2023-11-16T16:47:53.394469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:32:19.174367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343351+0000","last_change":"2023-11-16T16:46:41.203618+0000","last_active":"2023-11-16T16:47:53.343351+0000","last_peered":"2023-11-16T16:47:53.343351+0000","last_clean":"2023-11-16T16:47:53.343351+0000","last_became_active":"2023-11-16T16:46:41.202322+0000","last_became_peered":"2023-11-16T16:46:41.202322+0000","last_unstale":"2023-11-16T16:47:53.343351+0000","last_undegraded":"2023-11-16T16:47:53.343351+0000","last_fullsized":"2023-11-16T16:47:53.343351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:52:45.238718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343138+0000","last_change":"2023-11-16T16:46:37.154064+0000","last_active":"2023-11-16T16:47:53.343138+0000","last_peered":"2023-11-16T16:47:53.343138+0000","last_clean":"2023-11-16T16:47:53.343138+0000","last_became_active":"2023-11-16T16:46:37.153776+0000","last_became_peered":"2023-11-16T16:46:37.153776+0000","last_unstale":"2023-11-16T16:47:53.343138+0000","last_undegraded":"2023-11-16T16:47:53.343138+0000","last_fullsized":"2023-11-16T16:47:53.343138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:05:27.979843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":80,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:54.390155+0000","last_change":"2023-11-16T16:46:36.612613+0000","last_active":"2023-11-16T16:47:54.390155+0000","last_peered":"2023-11-16T16:47:54.390155+0000","last_clean":"2023-11-16T16:47:54.390155+0000","last_became_active":"2023-11-16T16:46:36.612303+0000","last_became_peered":"2023-11-16T16:46:36.612303+0000","last_unstale":"2023-11-16T16:47:54.390155+0000","last_undegraded":"2023-11-16T16:47:54.390155+0000","last_fullsized":"2023-11-16T16:47:54.390155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:01:35.412661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616882+0000","last_change":"2023-11-16T16:46:53.046753+0000","last_active":"2023-11-16T16:47:53.616882+0000","last_peered":"2023-11-16T16:47:53.616882+0000","last_clean":"2023-11-16T16:47:53.616882+0000","last_became_active":"2023-11-16T16:46:53.046231+0000","last_became_peered":"2023-11-16T16:46:53.046231+0000","last_unstale":"2023-11-16T16:47:53.616882+0000","last_undegraded":"2023-11-16T16:47:53.616882+0000","last_fullsized":"2023-11-16T16:47:53.616882+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:45:28.669638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617623+0000","last_change":"2023-11-16T16:47:53.617623+0000","last_active":"2023-11-16T16:47:53.617623+0000","last_peered":"2023-11-16T16:47:53.617623+0000","last_clean":"2023-11-16T16:47:53.617623+0000","last_became_active":"2023-11-16T16:47:53.617223+0000","last_became_peered":"2023-11-16T16:47:53.617223+0000","last_unstale":"2023-11-16T16:47:53.617623+0000","last_undegraded":"2023-11-16T16:47:53.617623+0000","last_fullsized":"2023-11-16T16:47:53.617623+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:50:23.328919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340296+0000","last_change":"2023-11-16T16:46:36.597520+0000","last_active":"2023-11-16T16:47:53.340296+0000","last_peered":"2023-11-16T16:47:53.340296+0000","last_clean":"2023-11-16T16:47:53.340296+0000","last_became_active":"2023-11-16T16:46:36.596542+0000","last_became_peered":"2023-11-16T16:46:36.596542+0000","last_unstale":"2023-11-16T16:47:53.340296+0000","last_undegraded":"2023-11-16T16:47:53.340296+0000","last_fullsized":"2023-11-16T16:47:53.340296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:19:23.260941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615707+0000","last_change":"2023-11-16T16:45:27.937850+0000","last_active":"2023-11-16T16:47:53.615707+0000","last_peered":"2023-11-16T16:47:53.615707+0000","last_clean":"2023-11-16T16:47:53.615707+0000","last_became_active":"2023-11-16T16:45:27.937451+0000","last_became_peered":"2023-11-16T16:45:27.937451+0000","last_unstale":"2023-11-16T16:47:53.615707+0000","last_undegraded":"2023-11-16T16:47:53.615707+0000","last_fullsized":"2023-11-16T16:47:53.615707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:42:13.306703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:42:13.306703+0000","last_clean_scrub_stamp":"2023-11-16T16:42:13.306703+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:36:26.247109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343317+0000","last_change":"2023-11-16T16:46:41.203736+0000","last_active":"2023-11-16T16:47:53.343317+0000","last_peered":"2023-11-16T16:47:53.343317+0000","last_clean":"2023-11-16T16:47:53.343317+0000","last_became_active":"2023-11-16T16:46:41.203282+0000","last_became_peered":"2023-11-16T16:46:41.203282+0000","last_unstale":"2023-11-16T16:47:53.343317+0000","last_undegraded":"2023-11-16T16:47:53.343317+0000","last_fullsized":"2023-11-16T16:47:53.343317+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:25:51.413002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.345806+0000","last_change":"2023-11-16T16:46:37.157408+0000","last_active":"2023-11-16T16:47:53.345806+0000","last_peered":"2023-11-16T16:47:53.345806+0000","last_clean":"2023-11-16T16:47:53.345806+0000","last_became_active":"2023-11-16T16:46:37.156972+0000","last_became_peered":"2023-11-16T16:46:37.156972+0000","last_unstale":"2023-11-16T16:47:53.345806+0000","last_undegraded":"2023-11-16T16:47:53.345806+0000","last_fullsized":"2023-11-16T16:47:53.345806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:17:05.643572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615870+0000","last_change":"2023-11-16T16:46:36.594284+0000","last_active":"2023-11-16T16:47:53.615870+0000","last_peered":"2023-11-16T16:47:53.615870+0000","last_clean":"2023-11-16T16:47:53.615870+0000","last_became_active":"2023-11-16T16:46:36.593964+0000","last_became_peered":"2023-11-16T16:46:36.593964+0000","last_unstale":"2023-11-16T16:47:53.615870+0000","last_undegraded":"2023-11-16T16:47:53.615870+0000","last_fullsized":"2023-11-16T16:47:53.615870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:08:17.083883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616510+0000","last_change":"2023-11-16T16:46:41.173944+0000","last_active":"2023-11-16T16:47:53.616510+0000","last_peered":"2023-11-16T16:47:53.616510+0000","last_clean":"2023-11-16T16:47:53.616510+0000","last_became_active":"2023-11-16T16:46:41.173391+0000","last_became_peered":"2023-11-16T16:46:41.173391+0000","last_unstale":"2023-11-16T16:47:53.616510+0000","last_undegraded":"2023-11-16T16:47:53.616510+0000","last_fullsized":"2023-11-16T16:47:53.616510+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:33:33.745909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340049+0000","last_change":"2023-11-16T16:46:36.596360+0000","last_active":"2023-11-16T16:47:53.340049+0000","last_peered":"2023-11-16T16:47:53.340049+0000","last_clean":"2023-11-16T16:47:53.340049+0000","last_became_active":"2023-11-16T16:46:36.596042+0000","last_became_peered":"2023-11-16T16:46:36.596042+0000","last_unstale":"2023-11-16T16:47:53.340049+0000","last_undegraded":"2023-11-16T16:47:53.340049+0000","last_fullsized":"2023-11-16T16:47:53.340049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:25:42.128187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.345267+0000","last_change":"2023-11-16T16:46:37.160277+0000","last_active":"2023-11-16T16:47:53.345267+0000","last_peered":"2023-11-16T16:47:53.345267+0000","last_clean":"2023-11-16T16:47:53.345267+0000","last_became_active":"2023-11-16T16:46:37.159047+0000","last_became_peered":"2023-11-16T16:46:37.159047+0000","last_unstale":"2023-11-16T16:47:53.345267+0000","last_undegraded":"2023-11-16T16:47:53.345267+0000","last_fullsized":"2023-11-16T16:47:53.345267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:51:56.521259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343322+0000","last_change":"2023-11-16T16:46:41.202909+0000","last_active":"2023-11-16T16:47:53.343322+0000","last_peered":"2023-11-16T16:47:53.343322+0000","last_clean":"2023-11-16T16:47:53.343322+0000","last_became_active":"2023-11-16T16:46:41.201969+0000","last_became_peered":"2023-11-16T16:46:41.201969+0000","last_unstale":"2023-11-16T16:47:53.343322+0000","last_undegraded":"2023-11-16T16:47:53.343322+0000","last_fullsized":"2023-11-16T16:47:53.343322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:27:24.361657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'5","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.347878+0000","last_change":"2023-11-16T16:46:36.599389+0000","last_active":"2023-11-16T16:47:53.347878+0000","last_peered":"2023-11-16T16:47:53.347878+0000","last_clean":"2023-11-16T16:47:53.347878+0000","last_became_active":"2023-11-16T16:46:36.599020+0000","last_became_peered":"2023-11-16T16:46:36.599020+0000","last_unstale":"2023-11-16T16:47:53.347878+0000","last_undegraded":"2023-11-16T16:47:53.347878+0000","last_fullsized":"2023-11-16T16:47:53.347878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:22:15.097589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342429+0000","last_change":"2023-11-16T16:46:37.145412+0000","last_active":"2023-11-16T16:47:53.342429+0000","last_peered":"2023-11-16T16:47:53.342429+0000","last_clean":"2023-11-16T16:47:53.342429+0000","last_became_active":"2023-11-16T16:46:37.141380+0000","last_became_peered":"2023-11-16T16:46:37.141380+0000","last_unstale":"2023-11-16T16:47:53.342429+0000","last_undegraded":"2023-11-16T16:47:53.342429+0000","last_fullsized":"2023-11-16T16:47:53.342429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:05:27.521717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615980+0000","last_change":"2023-11-16T16:46:41.163578+0000","last_active":"2023-11-16T16:47:53.615980+0000","last_peered":"2023-11-16T16:47:53.615980+0000","last_clean":"2023-11-16T16:47:53.615980+0000","last_became_active":"2023-11-16T16:46:41.163153+0000","last_became_peered":"2023-11-16T16:46:41.163153+0000","last_unstale":"2023-11-16T16:47:53.615980+0000","last_undegraded":"2023-11-16T16:47:53.615980+0000","last_fullsized":"2023-11-16T16:47:53.615980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:44:16.408506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340663+0000","last_change":"2023-11-16T16:46:36.606973+0000","last_active":"2023-11-16T16:47:53.340663+0000","last_peered":"2023-11-16T16:47:53.340663+0000","last_clean":"2023-11-16T16:47:53.340663+0000","last_became_active":"2023-11-16T16:46:36.606744+0000","last_became_peered":"2023-11-16T16:46:36.606744+0000","last_unstale":"2023-11-16T16:47:53.340663+0000","last_undegraded":"2023-11-16T16:47:53.340663+0000","last_fullsized":"2023-11-16T16:47:53.340663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:59:58.817297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.345611+0000","last_change":"2023-11-16T16:47:53.345611+0000","last_active":"2023-11-16T16:47:53.345611+0000","last_peered":"2023-11-16T16:47:53.345611+0000","last_clean":"2023-11-16T16:47:53.345611+0000","last_became_active":"2023-11-16T16:47:53.345139+0000","last_became_peered":"2023-11-16T16:47:53.345139+0000","last_unstale":"2023-11-16T16:47:53.345611+0000","last_undegraded":"2023-11-16T16:47:53.345611+0000","last_fullsized":"2023-11-16T16:47:53.345611+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:14:14.759670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340481+0000","last_change":"2023-11-16T16:46:41.204645+0000","last_active":"2023-11-16T16:47:53.340481+0000","last_peered":"2023-11-16T16:47:53.340481+0000","last_clean":"2023-11-16T16:47:53.340481+0000","last_became_active":"2023-11-16T16:46:41.204167+0000","last_became_peered":"2023-11-16T16:46:41.204167+0000","last_unstale":"2023-11-16T16:47:53.340481+0000","last_undegraded":"2023-11-16T16:47:53.340481+0000","last_fullsized":"2023-11-16T16:47:53.340481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:44:18.216270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615303+0000","last_change":"2023-11-16T16:46:52.334958+0000","last_active":"2023-11-16T16:47:53.615303+0000","last_peered":"2023-11-16T16:47:53.615303+0000","last_clean":"2023-11-16T16:47:53.615303+0000","last_became_active":"2023-11-16T16:46:52.334607+0000","last_became_peered":"2023-11-16T16:46:52.334607+0000","last_unstale":"2023-11-16T16:47:53.615303+0000","last_undegraded":"2023-11-16T16:47:53.615303+0000","last_fullsized":"2023-11-16T16:47:53.615303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:28:02.214272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339379+0000","last_change":"2023-11-16T16:46:37.153357+0000","last_active":"2023-11-16T16:47:53.339379+0000","last_peered":"2023-11-16T16:47:53.339379+0000","last_clean":"2023-11-16T16:47:53.339379+0000","last_became_active":"2023-11-16T16:46:37.153064+0000","last_became_peered":"2023-11-16T16:46:37.153064+0000","last_unstale":"2023-11-16T16:47:53.339379+0000","last_undegraded":"2023-11-16T16:47:53.339379+0000","last_fullsized":"2023-11-16T16:47:53.339379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:43:35.983547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.348077+0000","last_change":"2023-11-16T16:46:41.183938+0000","last_active":"2023-11-16T16:47:53.348077+0000","last_peered":"2023-11-16T16:47:53.348077+0000","last_clean":"2023-11-16T16:47:53.348077+0000","last_became_active":"2023-11-16T16:46:41.181850+0000","last_became_peered":"2023-11-16T16:46:41.181850+0000","last_unstale":"2023-11-16T16:47:53.348077+0000","last_undegraded":"2023-11-16T16:47:53.348077+0000","last_fullsized":"2023-11-16T16:47:53.348077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:53:49.918986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342820+0000","last_change":"2023-11-16T16:46:36.603257+0000","last_active":"2023-11-16T16:47:53.342820+0000","last_peered":"2023-11-16T16:47:53.342820+0000","last_clean":"2023-11-16T16:47:53.342820+0000","last_became_active":"2023-11-16T16:46:36.602852+0000","last_became_peered":"2023-11-16T16:46:36.602852+0000","last_unstale":"2023-11-16T16:47:53.342820+0000","last_undegraded":"2023-11-16T16:47:53.342820+0000","last_fullsized":"2023-11-16T16:47:53.342820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:03:55.431436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615710+0000","last_change":"2023-11-16T16:46:37.162689+0000","last_active":"2023-11-16T16:47:53.615710+0000","last_peered":"2023-11-16T16:47:53.615710+0000","last_clean":"2023-11-16T16:47:53.615710+0000","last_became_active":"2023-11-16T16:46:37.162356+0000","last_became_peered":"2023-11-16T16:46:37.162356+0000","last_unstale":"2023-11-16T16:47:53.615710+0000","last_undegraded":"2023-11-16T16:47:53.615710+0000","last_fullsized":"2023-11-16T16:47:53.615710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:38:37.707256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340208+0000","last_change":"2023-11-16T16:46:41.182286+0000","last_active":"2023-11-16T16:47:53.340208+0000","last_peered":"2023-11-16T16:47:53.340208+0000","last_clean":"2023-11-16T16:47:53.340208+0000","last_became_active":"2023-11-16T16:46:41.181820+0000","last_became_peered":"2023-11-16T16:46:41.181820+0000","last_unstale":"2023-11-16T16:47:53.340208+0000","last_undegraded":"2023-11-16T16:47:53.340208+0000","last_fullsized":"2023-11-16T16:47:53.340208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:17:47.324972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615819+0000","last_change":"2023-11-16T16:46:36.598952+0000","last_active":"2023-11-16T16:47:53.615819+0000","last_peered":"2023-11-16T16:47:53.615819+0000","last_clean":"2023-11-16T16:47:53.615819+0000","last_became_active":"2023-11-16T16:46:36.598617+0000","last_became_peered":"2023-11-16T16:46:36.598617+0000","last_unstale":"2023-11-16T16:47:53.615819+0000","last_undegraded":"2023-11-16T16:47:53.615819+0000","last_fullsized":"2023-11-16T16:47:53.615819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:23:33.595837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342335+0000","last_change":"2023-11-16T16:46:37.154350+0000","last_active":"2023-11-16T16:47:53.342335+0000","last_peered":"2023-11-16T16:47:53.342335+0000","last_clean":"2023-11-16T16:47:53.342335+0000","last_became_active":"2023-11-16T16:46:37.154045+0000","last_became_peered":"2023-11-16T16:46:37.154045+0000","last_unstale":"2023-11-16T16:47:53.342335+0000","last_undegraded":"2023-11-16T16:47:53.342335+0000","last_fullsized":"2023-11-16T16:47:53.342335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:42:00.370393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340495+0000","last_change":"2023-11-16T16:46:41.202563+0000","last_active":"2023-11-16T16:47:53.340495+0000","last_peered":"2023-11-16T16:47:53.340495+0000","last_clean":"2023-11-16T16:47:53.340495+0000","last_became_active":"2023-11-16T16:46:41.202125+0000","last_became_peered":"2023-11-16T16:46:41.202125+0000","last_unstale":"2023-11-16T16:47:53.340495+0000","last_undegraded":"2023-11-16T16:47:53.340495+0000","last_fullsized":"2023-11-16T16:47:53.340495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:29:21.341816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339619+0000","last_change":"2023-11-16T16:46:52.341548+0000","last_active":"2023-11-16T16:47:53.339619+0000","last_peered":"2023-11-16T16:47:53.339619+0000","last_clean":"2023-11-16T16:47:53.339619+0000","last_became_active":"2023-11-16T16:46:52.337771+0000","last_became_peered":"2023-11-16T16:46:52.337771+0000","last_unstale":"2023-11-16T16:47:53.339619+0000","last_undegraded":"2023-11-16T16:47:53.339619+0000","last_fullsized":"2023-11-16T16:47:53.339619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:29:45.214101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616366+0000","last_change":"2023-11-16T16:46:37.159095+0000","last_active":"2023-11-16T16:47:53.616366+0000","last_peered":"2023-11-16T16:47:53.616366+0000","last_clean":"2023-11-16T16:47:53.616366+0000","last_became_active":"2023-11-16T16:46:37.158668+0000","last_became_peered":"2023-11-16T16:46:37.158668+0000","last_unstale":"2023-11-16T16:47:53.616366+0000","last_undegraded":"2023-11-16T16:47:53.616366+0000","last_fullsized":"2023-11-16T16:47:53.616366+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:43:01.651855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615482+0000","last_change":"2023-11-16T16:46:41.165572+0000","last_active":"2023-11-16T16:47:53.615482+0000","last_peered":"2023-11-16T16:47:53.615482+0000","last_clean":"2023-11-16T16:47:53.615482+0000","last_became_active":"2023-11-16T16:46:41.165174+0000","last_became_peered":"2023-11-16T16:46:41.165174+0000","last_unstale":"2023-11-16T16:47:53.615482+0000","last_undegraded":"2023-11-16T16:47:53.615482+0000","last_fullsized":"2023-11-16T16:47:53.615482+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:26:17.992824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617090+0000","last_change":"2023-11-16T16:46:36.593871+0000","last_active":"2023-11-16T16:47:53.617090+0000","last_peered":"2023-11-16T16:47:53.617090+0000","last_clean":"2023-11-16T16:47:53.617090+0000","last_became_active":"2023-11-16T16:46:36.593579+0000","last_became_peered":"2023-11-16T16:46:36.593579+0000","last_unstale":"2023-11-16T16:47:53.617090+0000","last_undegraded":"2023-11-16T16:47:53.617090+0000","last_fullsized":"2023-11-16T16:47:53.617090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:58:29.714501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617143+0000","last_change":"2023-11-16T16:46:37.140823+0000","last_active":"2023-11-16T16:47:53.617143+0000","last_peered":"2023-11-16T16:47:53.617143+0000","last_clean":"2023-11-16T16:47:53.617143+0000","last_became_active":"2023-11-16T16:46:37.140505+0000","last_became_peered":"2023-11-16T16:46:37.140505+0000","last_unstale":"2023-11-16T16:47:53.617143+0000","last_undegraded":"2023-11-16T16:47:53.617143+0000","last_fullsized":"2023-11-16T16:47:53.617143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:25:37.585706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.338806+0000","last_change":"2023-11-16T16:46:41.182041+0000","last_active":"2023-11-16T16:47:53.338806+0000","last_peered":"2023-11-16T16:47:53.338806+0000","last_clean":"2023-11-16T16:47:53.338806+0000","last_became_active":"2023-11-16T16:46:41.181420+0000","last_became_peered":"2023-11-16T16:46:41.181420+0000","last_unstale":"2023-11-16T16:47:53.338806+0000","last_undegraded":"2023-11-16T16:47:53.338806+0000","last_fullsized":"2023-11-16T16:47:53.338806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:41:11.223688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343181+0000","last_change":"2023-11-16T16:46:36.602184+0000","last_active":"2023-11-16T16:47:53.343181+0000","last_peered":"2023-11-16T16:47:53.343181+0000","last_clean":"2023-11-16T16:47:53.343181+0000","last_became_active":"2023-11-16T16:46:36.601893+0000","last_became_peered":"2023-11-16T16:46:36.601893+0000","last_unstale":"2023-11-16T16:47:53.343181+0000","last_undegraded":"2023-11-16T16:47:53.343181+0000","last_fullsized":"2023-11-16T16:47:53.343181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:11:04.147511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616798+0000","last_change":"2023-11-16T16:46:37.153649+0000","last_active":"2023-11-16T16:47:53.616798+0000","last_peered":"2023-11-16T16:47:53.616798+0000","last_clean":"2023-11-16T16:47:53.616798+0000","last_became_active":"2023-11-16T16:46:37.152420+0000","last_became_peered":"2023-11-16T16:46:37.152420+0000","last_unstale":"2023-11-16T16:47:53.616798+0000","last_undegraded":"2023-11-16T16:47:53.616798+0000","last_fullsized":"2023-11-16T16:47:53.616798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:47:46.597259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342448+0000","last_change":"2023-11-16T16:46:41.168363+0000","last_active":"2023-11-16T16:47:53.342448+0000","last_peered":"2023-11-16T16:47:53.342448+0000","last_clean":"2023-11-16T16:47:53.342448+0000","last_became_active":"2023-11-16T16:46:41.167615+0000","last_became_peered":"2023-11-16T16:46:41.167615+0000","last_unstale":"2023-11-16T16:47:53.342448+0000","last_undegraded":"2023-11-16T16:47:53.342448+0000","last_fullsized":"2023-11-16T16:47:53.342448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:48:16.995034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343357+0000","last_change":"2023-11-16T16:46:36.611353+0000","last_active":"2023-11-16T16:47:53.343357+0000","last_peered":"2023-11-16T16:47:53.343357+0000","last_clean":"2023-11-16T16:47:53.343357+0000","last_became_active":"2023-11-16T16:46:36.610886+0000","last_became_peered":"2023-11-16T16:46:36.610886+0000","last_unstale":"2023-11-16T16:47:53.343357+0000","last_undegraded":"2023-11-16T16:47:53.343357+0000","last_fullsized":"2023-11-16T16:47:53.343357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:24:34.228785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617020+0000","last_change":"2023-11-16T16:46:37.146227+0000","last_active":"2023-11-16T16:47:53.617020+0000","last_peered":"2023-11-16T16:47:53.617020+0000","last_clean":"2023-11-16T16:47:53.617020+0000","last_became_active":"2023-11-16T16:46:37.143239+0000","last_became_peered":"2023-11-16T16:46:37.143239+0000","last_unstale":"2023-11-16T16:47:53.617020+0000","last_undegraded":"2023-11-16T16:47:53.617020+0000","last_fullsized":"2023-11-16T16:47:53.617020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:44:02.854936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615999+0000","last_change":"2023-11-16T16:46:37.153845+0000","last_active":"2023-11-16T16:47:53.615999+0000","last_peered":"2023-11-16T16:47:53.615999+0000","last_clean":"2023-11-16T16:47:53.615999+0000","last_became_active":"2023-11-16T16:46:37.153505+0000","last_became_peered":"2023-11-16T16:46:37.153505+0000","last_unstale":"2023-11-16T16:47:53.615999+0000","last_undegraded":"2023-11-16T16:47:53.615999+0000","last_fullsized":"2023-11-16T16:47:53.615999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:24:41.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616723+0000","last_change":"2023-11-16T16:46:37.148451+0000","last_active":"2023-11-16T16:47:53.616723+0000","last_peered":"2023-11-16T16:47:53.616723+0000","last_clean":"2023-11-16T16:47:53.616723+0000","last_became_active":"2023-11-16T16:46:37.148142+0000","last_became_peered":"2023-11-16T16:46:37.148142+0000","last_unstale":"2023-11-16T16:47:53.616723+0000","last_undegraded":"2023-11-16T16:47:53.616723+0000","last_fullsized":"2023-11-16T16:47:53.616723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:36:55.683708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616000+0000","last_change":"2023-11-16T16:46:37.150360+0000","last_active":"2023-11-16T16:47:53.616000+0000","last_peered":"2023-11-16T16:47:53.616000+0000","last_clean":"2023-11-16T16:47:53.616000+0000","last_became_active":"2023-11-16T16:46:37.150045+0000","last_became_peered":"2023-11-16T16:46:37.150045+0000","last_unstale":"2023-11-16T16:47:53.616000+0000","last_undegraded":"2023-11-16T16:47:53.616000+0000","last_fullsized":"2023-11-16T16:47:53.616000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:17:58.442185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339093+0000","last_change":"2023-11-16T16:46:37.161736+0000","last_active":"2023-11-16T16:47:53.339093+0000","last_peered":"2023-11-16T16:47:53.339093+0000","last_clean":"2023-11-16T16:47:53.339093+0000","last_became_active":"2023-11-16T16:46:37.158273+0000","last_became_peered":"2023-11-16T16:46:37.158273+0000","last_unstale":"2023-11-16T16:47:53.339093+0000","last_undegraded":"2023-11-16T16:47:53.339093+0000","last_fullsized":"2023-11-16T16:47:53.339093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:30:05.153453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.349210+0000","last_change":"2023-11-16T16:47:53.349210+0000","last_active":"2023-11-16T16:47:53.349210+0000","last_peered":"2023-11-16T16:47:53.349210+0000","last_clean":"2023-11-16T16:47:53.349210+0000","last_became_active":"2023-11-16T16:47:53.348819+0000","last_became_peered":"2023-11-16T16:47:53.348819+0000","last_unstale":"2023-11-16T16:47:53.349210+0000","last_undegraded":"2023-11-16T16:47:53.349210+0000","last_fullsized":"2023-11-16T16:47:53.349210+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:24:00.116336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616870+0000","last_change":"2023-11-16T16:46:37.154475+0000","last_active":"2023-11-16T16:47:53.616870+0000","last_peered":"2023-11-16T16:47:53.616870+0000","last_clean":"2023-11-16T16:47:53.616870+0000","last_became_active":"2023-11-16T16:46:37.154161+0000","last_became_peered":"2023-11-16T16:46:37.154161+0000","last_unstale":"2023-11-16T16:47:53.616870+0000","last_undegraded":"2023-11-16T16:47:53.616870+0000","last_fullsized":"2023-11-16T16:47:53.616870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:19:04.354750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616438+0000","last_change":"2023-11-16T16:46:37.140293+0000","last_active":"2023-11-16T16:47:53.616438+0000","last_peered":"2023-11-16T16:47:53.616438+0000","last_clean":"2023-11-16T16:47:53.616438+0000","last_became_active":"2023-11-16T16:46:37.139966+0000","last_became_peered":"2023-11-16T16:46:37.139966+0000","last_unstale":"2023-11-16T16:47:53.616438+0000","last_undegraded":"2023-11-16T16:47:53.616438+0000","last_fullsized":"2023-11-16T16:47:53.616438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:05:23.489857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342605+0000","last_change":"2023-11-16T16:46:37.134807+0000","last_active":"2023-11-16T16:47:53.342605+0000","last_peered":"2023-11-16T16:47:53.342605+0000","last_clean":"2023-11-16T16:47:53.342605+0000","last_became_active":"2023-11-16T16:46:37.131833+0000","last_became_peered":"2023-11-16T16:46:37.131833+0000","last_unstale":"2023-11-16T16:47:53.342605+0000","last_undegraded":"2023-11-16T16:47:53.342605+0000","last_fullsized":"2023-11-16T16:47:53.342605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:22:51.275681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342895+0000","last_change":"2023-11-16T16:46:37.152424+0000","last_active":"2023-11-16T16:47:53.342895+0000","last_peered":"2023-11-16T16:47:53.342895+0000","last_clean":"2023-11-16T16:47:53.342895+0000","last_became_active":"2023-11-16T16:46:37.152082+0000","last_became_peered":"2023-11-16T16:46:37.152082+0000","last_unstale":"2023-11-16T16:47:53.342895+0000","last_undegraded":"2023-11-16T16:47:53.342895+0000","last_fullsized":"2023-11-16T16:47:53.342895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:48:35.526315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343126+0000","last_change":"2023-11-16T16:46:37.140267+0000","last_active":"2023-11-16T16:47:53.343126+0000","last_peered":"2023-11-16T16:47:53.343126+0000","last_clean":"2023-11-16T16:47:53.343126+0000","last_became_active":"2023-11-16T16:46:37.139689+0000","last_became_peered":"2023-11-16T16:46:37.139689+0000","last_unstale":"2023-11-16T16:47:53.343126+0000","last_undegraded":"2023-11-16T16:47:53.343126+0000","last_fullsized":"2023-11-16T16:47:53.343126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:22:15.270033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615789+0000","last_change":"2023-11-16T16:46:37.151911+0000","last_active":"2023-11-16T16:47:53.615789+0000","last_peered":"2023-11-16T16:47:53.615789+0000","last_clean":"2023-11-16T16:47:53.615789+0000","last_became_active":"2023-11-16T16:46:37.151614+0000","last_became_peered":"2023-11-16T16:46:37.151614+0000","last_unstale":"2023-11-16T16:47:53.615789+0000","last_undegraded":"2023-11-16T16:47:53.615789+0000","last_fullsized":"2023-11-16T16:47:53.615789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:48:46.891259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.346604+0000","last_change":"2023-11-16T16:46:37.153265+0000","last_active":"2023-11-16T16:47:53.346604+0000","last_peered":"2023-11-16T16:47:53.346604+0000","last_clean":"2023-11-16T16:47:53.346604+0000","last_became_active":"2023-11-16T16:46:37.152933+0000","last_became_peered":"2023-11-16T16:46:37.152933+0000","last_unstale":"2023-11-16T16:47:53.346604+0000","last_undegraded":"2023-11-16T16:47:53.346604+0000","last_fullsized":"2023-11-16T16:47:53.346604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:09:28.674677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616247+0000","last_change":"2023-11-16T16:46:37.142831+0000","last_active":"2023-11-16T16:47:53.616247+0000","last_peered":"2023-11-16T16:47:53.616247+0000","last_clean":"2023-11-16T16:47:53.616247+0000","last_became_active":"2023-11-16T16:46:37.142480+0000","last_became_peered":"2023-11-16T16:46:37.142480+0000","last_unstale":"2023-11-16T16:47:53.616247+0000","last_undegraded":"2023-11-16T16:47:53.616247+0000","last_fullsized":"2023-11-16T16:47:53.616247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:10:16.785368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64741,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":297,"num_read_kb":216,"num_write":154,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416412,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6272,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738880,"statfs":{"total":95995035648,"available":95988613120,"internally_reserved":0,"allocated":1040384,"data_stored":513127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.748,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47399999999999998,"15min":0.46999999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68999999999999995,"5min":0.879,"15min":0.879},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.32400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86699999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70299999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81999999999999995,"5min":1.756,"15min":1.756},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.433,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84699999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.625}]},{"osd":6,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71699999999999997,"15min":0.71999999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97099999999999997,"5min":1.228,"15min":1.228},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.52900000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":1.845,"15min":1.845},"last":0.73899999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.58999999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.99299999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75,"15min":0.746},"min":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93000000000000005,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.73099999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.78100000000000003}]},{"osd":9,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.80400000000000005,"15min":0.80800000000000005},"min":{"1min":0.56699999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91700000000000004,"5min":1.819,"15min":1.819},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78400000000000003,"15min":0.78000000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0620000000000001,"5min":1.28,"15min":1.28},"last":0.86399999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.78700000000000003,"15min":0.78000000000000003},"min":{"1min":0.61399999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96499999999999997,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93700000000000006,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.79500000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7204,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93737948,"statfs":{"total":95995035648,"available":95987658752,"internally_reserved":0,"allocated":1732608,"data_stored":1165670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.752},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.59699999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.61399999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.52300000000000002,"15min":0.53300000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64400000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.624}]},{"osd":2,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69699999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80200000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.82299999999999995,"15min":0.86499999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.70599999999999996,"5min":4.1299999999999999,"15min":4.1299999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.85899999999999999,"15min":0.89500000000000002},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76600000000000001,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":0.79200000000000004}]},{"osd":4,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.623,"15min":0.63200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73099999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71899999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83499999999999996,"5min":0.873,"15min":0.873},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.85299999999999998,"5min":1.048,"15min":1.048},"last":0.77800000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79200000000000004,"15min":0.80200000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":1.054,"15min":1.054},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.56599999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94699999999999995,"5min":1.357,"15min":1.357},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.79600000000000004,"15min":0.81000000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90000000000000002,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.86199999999999999}]},{"osd":9,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.84999999999999998,"15min":0.86399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92000000000000004,"5min":1.387,"15min":1.387},"last":0.88},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.89400000000000002,"5min":1.226,"15min":1.226},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Nov 16 16:47:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78400000000000003,"15min":0.79200000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.88500000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.93799999999999994,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.93100000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6420,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738732,"statfs":{"total":95995035648,"available":95988461568,"internally_reserved":0,"allocated":1060864,"data_stored":517459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.55100000000000005,"15min":0.56399999999999995},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72599999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.749,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.54900000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72999999999999998,"15min":0.73899999999999999},"min":{"1min":0.47899999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96999999999999997,"5min":1.048,"15min":1.048},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.85899999999999999,"15min":0.89100000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98699999999999999,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.62}]},{"osd":3,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67900000000000005,"15min":0.68999999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78000000000000003,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.79100000000000004,"15min":0.80900000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.056,"5min":1.24,"15min":1.24},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.82399999999999995,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1120000000000001,"5min":1.216,"15min":1.216},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.82199999999999995,"15min":0.84399999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95899999999999996,"5min":1.304,"15min":1.304},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.86799999999999999,"15min":0.88600000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.145,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Nov 16 16:46:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.92500000000000004},"min":{"1min":0.61899999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.111,"5min":1.282,"15min":1.282},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.98899999999999999,"15min":1.0229999999999999},"min":{"1min":0.57799999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0860000000000001,"5min":4.5650000000000004,"15min":4.5650000000000004},"last":0.54800000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1,"15min":1.0389999999999999},"min":{"1min":0.55300000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.163,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.93799999999999994,"15min":0.96699999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.077,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.70699999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.002,"15min":1.022},"min":{"1min":0.69699999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.135,"5min":4.4870000000000001,"15min":4.4870000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.86699999999999999,"15min":0.872},"min":{"1min":0.58799999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2130000000000001,"5min":1.343,"15min":1.343},"last":0.56799999999999995}]},{"osd":11,"last update":"Thu Nov 16 16:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.82899999999999996}]}]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6540,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738612,"statfs":{"total":95995035648,"available":95988338688,"internally_reserved":0,"allocated":1052672,"data_stored":512952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.371,"5min":0.309,"15min":0.309},"max":{"1min":0.82199999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58399999999999996,"15min":0.59699999999999998},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.89000000000000001}]},{"osd":1,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.47599999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72799999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.46000000000000002,"15min":0.46500000000000002},"min":{"1min":0.32200000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72999999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46400000000000002,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.64300000000000002,"15min":0.67900000000000005},"min":{"1min":0.432,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78500000000000003,"5min":1.536,"15min":1.536},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.66600000000000004,"15min":0.69599999999999995},"min":{"1min":0.38800000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81399999999999995,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.47899999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89600000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64200000000000002,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.56599999999999995},"min":{"1min":0.441,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76200000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58399999999999996,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.75800000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.76900000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.72499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94099999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.56000000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.51500000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88200000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71899999999999997,"15min":0.73299999999999998},"min":{"1min":0.51400000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92800000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.84699999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:47:03 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.753},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.78800000000000003}]},{"osd":10,"last update":"Thu Nov 16 16:47:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.74199999999999999,"15min":0.73699999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.745,"15min":0.72299999999999998},"min":{"1min":0.59099999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.874}]},{"osd":11,"last update":"Thu Nov 16 16:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.93300000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7168,"kb_used_data":1592,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93737984,"statfs":{"total":95995035648,"available":95987695616,"internally_reserved":0,"allocated":1630208,"data_stored":1098842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41299999999999998,"15min":0.40500000000000003},"min":{"1min":0.28199999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60899999999999999,"5min":0.624,"15min":0.624},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.42599999999999999,"15min":0.41499999999999998},"min":{"1min":0.34000000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.60299999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.38500000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58799999999999997,"15min":0.60799999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73899999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.504},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76800000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84599999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66600000000000004,"15min":0.66400000000000003},"min":{"1min":0.48299999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.307,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72899999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68999999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.754,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.46400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53600000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71199999999999997,"15min":0.70399999999999996},"min":{"1min":0.55700000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84799999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.80100000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.46300000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73399999999999999,"15min":0.72199999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91400000000000003,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.751},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.754,"15min":0.73599999999999999},"min":{"1min":0.48599999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96599999999999997,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.77600000000000002}]},{"osd":9,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90600000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.60899999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93600000000000005,"5min":1.054,"15min":1.054},"last":0.65900000000000003}]},{"osd":10,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70999999999999996,"15min":0.70599999999999996},"min":{"1min":0.53600000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94899999999999995,"5min":1.002,"15min":1.002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55200000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.98299999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Thu Nov 16 16:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73899999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":1036288,"data_stored":508539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.36599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71999999999999997,"5min":0.87,"15min":0.87},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.36199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78300000000000003,"5min":1.139,"15min":1.139},"last":0.51500000000000001}]},{"osd":1,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89000000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85299999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66100000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63,"15min":0.622},"min":{"1min":0.501,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.753,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.67500000000000004,"15min":0.69499999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":1.25,"15min":1.25},"last":0.998}]},{"osd":3,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.96699999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.625,"15min":0.629},"min":{"1min":0.40600000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72499999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.53100000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89300000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73499999999999999,"15min":0.749},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88800000000000001,"5min":1.343,"15min":1.343},"last":0.93700000000000006}]},{"osd":5,"last update":"Thu Nov 16 16:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.54600000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.91300000000000003,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.74299999999999999,"15min":0.73799999999999999},"min":{"1min":0.51600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.93500000000000005,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.55400000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.0569999999999999,"5min":1.109,"15min":1.109},"last":0.629},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0109999999999999,"5min":1.121,"15min":1.121},"last":0.92300000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78500000000000003,"15min":0.79200000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.151,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.60699999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.78900000000000003}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.86699999999999999,"15min":0.877},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.089,"5min":1.165,"15min":1.165},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84299999999999997,"15min":0.84699999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.097,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.80500000000000005}]},{"osd":10,"last update":"Thu Nov 16 16:47:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.85499999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.873,"15min":0.88500000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.133,"5min":1.2,"15min":1.2},"last":0.97999999999999998}]},{"osd":11,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.878}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7612,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737540,"statfs":{"total":95995035648,"available":95987240960,"internally_reserved":0,"allocated":1101824,"data_stored":569286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52200000000000002,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59899999999999998,"5min":1.431,"15min":1.431},"last":0.439},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57299999999999995,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76500000000000001,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Nov 16 16:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.436,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.627},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.58399999999999996,"15min":0.55600000000000005},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83699999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.69099999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.55100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91300000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60199999999999998,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.93200000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.40899999999999997,"15min":0.39200000000000002},"min":{"1min":0.32200000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.432,"15min":0.40300000000000002},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.58999999999999997,"15min":0.53800000000000003},"min":{"1min":0.44400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.82099999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61699999999999999,"15min":0.55700000000000005},"min":{"1min":0.52900000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.74299999999999999}]},{"osd":6,"last update":"Thu Nov 16 16:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.58999999999999997},"min":{"1min":0.48199999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85899999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.626,"15min":0.60199999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91500000000000004,"5min":0.995,"15min":0.995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.72499999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.72399999999999998},"min":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.90000000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71999999999999997,"15min":0.70499999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96899999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.70699999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94299999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.036,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.73699999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.034,"5min":1.052,"15min":1.052},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77900000000000003,"15min":0.79000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88800000000000001,"5min":1.075,"15min":1.075},"last":0.78300000000000003}]},{"osd":11,"last update":"Thu Nov 16 16:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.82099999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7508,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737644,"statfs":{"total":95995035648,"available":95987347456,"internally_reserved":0,"allocated":1060864,"data_stored":509927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89100000000000001,"5min":1.01,"15min":1.01},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.61699999999999999,"15min":0.55900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Nov 16 16:47:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61499999999999999,"15min":0.55700000000000005},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.503},{"interface":"front","average":{"1min":0.624,"5min":0.60699999999999998,"15min":0.55900000000000005},"min":{"1min":0.432,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95099999999999996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.629}]},{"osd":3,"last update":"Thu Nov 16 16:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":1.0569999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:47:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64000000000000001,"15min":0.60799999999999998},"min":{"1min":0.44800000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.62,"15min":0.58399999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86299999999999999,"5min":0.871,"15min":0.871},"last":0.54500000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.65600000000000003,"15min":0.623},"min":{"1min":0.47599999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.52300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:46:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.90900000000000003,"5min":1.03,"15min":1.03},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Nov 16 16:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.83199999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.59699999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77900000000000003,"15min":0.78800000000000003},"min":{"1min":0.60399999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.79500000000000004}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.79000000000000004,"15min":0.80500000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81100000000000005,"15min":0.81299999999999994},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.121,"5min":1.145,"15min":1.145},"last":0.64500000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.042,"5min":1.075,"15min":1.075},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.876,"5min":0.84999999999999998,"15min":0.84599999999999997},"min":{"1min":0.60899999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.70199999999999996}]},{"osd":11,"last update":"Thu Nov 16 16:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.85799999999999998,"15min":0.84499999999999997},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.81699999999999995,"15min":0.80300000000000005},"min":{"1min":0.70699999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.755}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7700,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737452,"statfs":{"total":95995035648,"available":95987150848,"internally_reserved":0,"allocated":1126400,"data_stored":570588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.40200000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55700000000000005,"15min":0.53900000000000003},"min":{"1min":0.38200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68999999999999995,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Nov 16 16:47:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56100000000000005,"15min":0.53100000000000003},"min":{"1min":0.39700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80000000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57299999999999995,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81399999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.67100000000000004}]},{"osd":3,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.47699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.66600000000000004,"15min":0.60999999999999999},"min":{"1min":0.46300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":4.8769999999999998,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":0.75900000000000001}]},{"osd":4,"last update":"Thu Nov 16 16:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.504}]},{"osd":5,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.48699999999999999,"15min":0.45500000000000002},"min":{"1min":0.441,"5min":0.314,"15min":0.314},"max":{"1min":0.86799999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.497,"15min":0.47099999999999997},"min":{"1min":0.42199999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73299999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.73399999999999999,"15min":0.65200000000000002},"min":{"1min":0.51800000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":7.2789999999999999,"5min":7.2789999999999999,"15min":7.2789999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.137,"5min":0.74299999999999999,"15min":0.66500000000000004},"min":{"1min":0.53800000000000003,"5min":0.437,"15min":0.437},"max":{"1min":4.9299999999999997,"5min":4.9299999999999997,"15min":4.9299999999999997},"last":0.90500000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70699999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.66500000000000004,"15min":0.65300000000000002},"min":{"1min":0.434,"5min":0.375,"15min":0.375},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.39900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71999999999999997}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.745,"15min":0.72999999999999998},"min":{"1min":0.63900000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.77500000000000002,"15min":0.76100000000000001},"min":{"1min":0.59099999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.86399999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:47:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.64200000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.749},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.72899999999999998,"15min":0.71499999999999997},"min":{"1min":0.70599999999999996,"5min":0.495,"15min":0.495},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.85399999999999998}]},{"osd":11,"last update":"Thu Nov 16 16:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.80700000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6792,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738360,"statfs":{"total":95995035648,"available":95988080640,"internally_reserved":0,"allocated":1048576,"data_stored":508733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.45500000000000002,"15min":0.39700000000000002},"min":{"1min":0.29299999999999998,"5min":0.246,"15min":0.246},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.47199999999999998},"min":{"1min":0.35199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.2110000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:47:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.65000000000000002,"15min":0.60299999999999998},"min":{"1min":0.497,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80300000000000005,"5min":1.119,"15min":1.119},"last":0.68300000000000005}]},{"osd":2,"last update":"Thu Nov 16 16:47:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.622},"min":{"1min":0.51000000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91500000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":1.137},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.70399999999999996,"15min":0.63200000000000001},"min":{"1min":0.51900000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.82699999999999996}]},{"osd":4,"last update":"Thu Nov 16 16:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":1.181},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.61199999999999999},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.88800000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67700000000000005,"15min":0.66100000000000003},"min":{"1min":0.56100000000000005,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.036,"5min":1.133,"15min":1.133},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.74099999999999999,"5min":1.149,"15min":1.149},"last":1.028}]},{"osd":6,"last update":"Thu Nov 16 16:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.48399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.88900000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77100000000000002,"15min":0.76500000000000001},"min":{"1min":0.54600000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0609999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.107}]},{"osd":8,"last update":"Thu Nov 16 16:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.753,"15min":0.753},"min":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1559999999999999,"5min":1.171,"15min":1.171},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.60699999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.306}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.4019999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81599999999999995,"15min":0.81899999999999995},"min":{"1min":0.64500000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0649999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.81799999999999995,"15min":0.83199999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":1.365}]},{"osd":11,"last update":"Thu Nov 16 16:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":1.2949999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.2090000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6724,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738428,"statfs":{"total":95995035648,"available":95988150272,"internally_reserved":0,"allocated":1044480,"data_stored":513200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.52500000000000002,"15min":0.501},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.377,"15min":0.34799999999999998},"min":{"1min":0.33700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.35199999999999998},"min":{"1min":0.26500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.5,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.36799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.754,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.53900000000000003,"15min":0.51000000000000001},"min":{"1min":0.45200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72599999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58399999999999996,"15min":0.53800000000000003},"min":{"1min":0.44400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75700000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82399999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56200000000000006,"15min":0.53800000000000003},"min":{"1min":0.48599999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60799999999999998,"15min":0.56799999999999995},"min":{"1min":0.47499999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81699999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.64100000000000001}]},{"osd":6,"last update":"Thu Nov 16 16:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.40899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83699999999999997,"5min":1.097,"15min":1.097},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.496,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82699999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.501,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83899999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.53000000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.66000000000000003,"15min":0.63800000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.70999999999999996,"15min":0.67600000000000005},"min":{"1min":0.58599999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94399999999999995,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72299999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.80000000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.92900000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.754},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73099999999999998,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.878,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67100000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:47:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89700000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67000000000000004,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.76800000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61299999999999999}]},{"osd":11,"last update":"Thu Nov 16 16:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.80400000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7316,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737836,"statfs":{"total":95995035648,"available":95987544064,"internally_reserved":0,"allocated":1650688,"data_stored":1099914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76600000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.625}]},{"osd":1,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.40300000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.55700000000000005,"15min":0.52000000000000002},"min":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77100000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.42299999999999999,"15min":0.39500000000000002},"min":{"1min":0.30599999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77700000000000002,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.44,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79400000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70399999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.436,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.60799999999999998,"15min":0.55800000000000005},"min":{"1min":0.45900000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.0149999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.747},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.58599999999999997,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0229999999999999,"5min":1.046,"15min":1.046},"last":0.83599999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64900000000000002,"15min":0.63500000000000001},"min":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67300000000000004,"15min":0.65500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.81399999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65900000000000003,"15min":0.64500000000000002},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.0069999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.63800000000000001,"15min":0.621},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95499999999999996,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.91900000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.65100000000000002,"15min":0.63100000000000001},"min":{"1min":0.47299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.625,"15min":0.60499999999999998},"min":{"1min":0.52400000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90700000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.95099999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.75,"15min":0.746},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.87}]},{"osd":10,"last update":"Thu Nov 16 16:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.754,"15min":0.748},"min":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0229999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.96999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:48:00.437 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph pg dump --format=json 2023-11-16T16:48:01.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:00 smithi093 ceph-mon[97863]: from='client.25015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:48:01.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:00 smithi159 ceph-mon[131528]: from='client.25015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:48:01.281 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:48:01.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:00 smithi125 ceph-mon[131076]: from='client.25015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:48:02.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:01 smithi093 ceph-mon[97863]: pgmap v327: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-11-16T16:48:02.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:01 smithi159 ceph-mon[131528]: pgmap v327: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-11-16T16:48:02.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:01 smithi125 ceph-mon[131076]: pgmap v327: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-11-16T16:48:02.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.824+0000 7f2404e9e700 1 -- 172.21.15.93:0/1026901964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24001055e0 msgr2=0x7f240010d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:02.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.824+0000 7f2404e9e700 1 --2- 172.21.15.93:0/1026901964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24001055e0 0x7f240010d160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009fc0 tx=0x7f23f800b330 comp rx=0 tx=0).stop 2023-11-16T16:48:02.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.824+0000 7f2404e9e700 1 -- 172.21.15.93:0/1026901964 shutdown_connections 2023-11-16T16:48:02.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.824+0000 7f2404e9e700 1 --2- 172.21.15.93:0/1026901964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24001055e0 0x7f240010d160 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:02.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.824+0000 7f2404e9e700 1 --2- 172.21.15.93:0/1026901964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2400100540 0x7f2400100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:02.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.824+0000 7f2404e9e700 1 --2- 172.21.15.93:0/1026901964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2400100ee0 0x7f2400104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:02.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.824+0000 7f2404e9e700 1 -- 172.21.15.93:0/1026901964 >> 172.21.15.93:0/1026901964 conn(0x7f24000fbdc0 msgr2=0x7f24000fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:02.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.824+0000 7f2404e9e700 1 -- 172.21.15.93:0/1026901964 shutdown_connections 2023-11-16T16:48:02.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.824+0000 7f2404e9e700 1 -- 172.21.15.93:0/1026901964 wait complete. 2023-11-16T16:48:02.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.825+0000 7f2404e9e700 1 Processor -- start 2023-11-16T16:48:02.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.825+0000 7f2404e9e700 1 -- start start 2023-11-16T16:48:02.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f2404e9e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2400100540 0x7f2400072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:02.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f2404e9e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2400100ee0 0x7f2400073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:02.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f2404e9e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24001055e0 0x7f24001a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:02.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f2404e9e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2400073c20 con 0x7f2400100ee0 2023-11-16T16:48:02.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f2404e9e700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24001a4c60 con 0x7f24001055e0 2023-11-16T16:48:02.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f2404e9e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24001a4dd0 con 0x7f2400100540 2023-11-16T16:48:02.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f23fdd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2400100ee0 0x7f2400073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:02.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f23fe59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2400100540 0x7f2400072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:02.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f23fdd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2400100ee0 0x7f2400073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40126/0 (socket says 172.21.15.93:40126) 2023-11-16T16:48:02.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f23fdd9b700 1 -- 172.21.15.93:0/2274773324 learned_addr learned my addr 172.21.15.93:0/2274773324 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:02.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f23fed9d700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24001055e0 0x7f24001a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:02.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f23fdd9b700 1 -- 172.21.15.93:0/2274773324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2400100540 msgr2=0x7f2400072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:02.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f23fdd9b700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2400100540 0x7f2400072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:02.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f23fdd9b700 1 -- 172.21.15.93:0/2274773324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24001055e0 msgr2=0x7f24001a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:02.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f23fdd9b700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24001055e0 0x7f24001a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:02.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.826+0000 7f23fdd9b700 1 -- 172.21.15.93:0/2274773324 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f8009c40 con 0x7f2400100ee0 2023-11-16T16:48:02.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.827+0000 7f23fdd9b700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2400100ee0 0x7f2400073330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f23f400e970 tx=0x7f23f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:02.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.827+0000 7f23ef7fe700 1 -- 172.21.15.93:0/2274773324 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4009790 con 0x7f2400100ee0 2023-11-16T16:48:02.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.827+0000 7f23ef7fe700 1 -- 172.21.15.93:0/2274773324 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f400c7c0 con 0x7f2400100ee0 2023-11-16T16:48:02.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.827+0000 7f2404e9e700 1 -- 172.21.15.93:0/2274773324 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2400110370 con 0x7f2400100ee0 2023-11-16T16:48:02.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.827+0000 7f23ef7fe700 1 -- 172.21.15.93:0/2274773324 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f40054b0 con 0x7f2400100ee0 2023-11-16T16:48:02.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.827+0000 7f2404e9e700 1 -- 172.21.15.93:0/2274773324 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24001101e0 con 0x7f2400100ee0 2023-11-16T16:48:02.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.828+0000 7f2404e9e700 1 -- 172.21.15.93:0/2274773324 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2400110500 con 0x7f2400100ee0 2023-11-16T16:48:02.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.830+0000 7f23ef7fe700 1 -- 172.21.15.93:0/2274773324 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f23f400c930 con 0x7f2400100ee0 2023-11-16T16:48:02.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.831+0000 7f23ef7fe700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f23e80a1260 0x7f23e80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:02.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.831+0000 7f23ef7fe700 1 -- 172.21.15.93:0/2274773324 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f23f4014070 con 0x7f2400100ee0 2023-11-16T16:48:02.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.831+0000 7f23fe59c700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f23e80a1260 0x7f23e80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:02.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.832+0000 7f23fe59c700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f23e80a1260 0x7f23e80a3710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f23f0005fd0 tx=0x7f23f0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:02.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:02.835+0000 7f23ef7fe700 1 -- 172.21.15.93:0/2274773324 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f23f40960e0 con 0x7f2400100ee0 2023-11-16T16:48:03.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.031+0000 7f2404e9e700 1 -- 172.21.15.93:0/2274773324 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24000fce30 con 0x7f23e80a1260 2023-11-16T16:48:03.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.044+0000 7f23ef7fe700 1 -- 172.21.15.93:0/2274773324 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427353 (secure 0 0 0) 0x7f24000fce30 con 0x7f23e80a1260 2023-11-16T16:48:03.051 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:03.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.050+0000 7f2404e9e700 1 -- 172.21.15.93:0/2274773324 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f23e80a1260 msgr2=0x7f23e80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:03.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.050+0000 7f2404e9e700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f23e80a1260 0x7f23e80a3710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f23f0005fd0 tx=0x7f23f0009310 comp rx=0 tx=0).stop 2023-11-16T16:48:03.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.050+0000 7f2404e9e700 1 -- 172.21.15.93:0/2274773324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2400100ee0 msgr2=0x7f2400073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:03.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.050+0000 7f2404e9e700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2400100ee0 0x7f2400073330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f23f400e970 tx=0x7f23f400ec80 comp rx=0 tx=0).stop 2023-11-16T16:48:03.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.050+0000 7f2404e9e700 1 -- 172.21.15.93:0/2274773324 shutdown_connections 2023-11-16T16:48:03.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.050+0000 7f2404e9e700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f23e80a1260 0x7f23e80a3710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:03.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.050+0000 7f2404e9e700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24001055e0 0x7f24001a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:03.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.050+0000 7f2404e9e700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2400100540 0x7f2400072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:03.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.050+0000 7f2404e9e700 1 --2- 172.21.15.93:0/2274773324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2400100ee0 0x7f2400073330 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:03.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.050+0000 7f2404e9e700 1 -- 172.21.15.93:0/2274773324 >> 172.21.15.93:0/2274773324 conn(0x7f24000fbdc0 msgr2=0x7f24000fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:03.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.050+0000 7f2404e9e700 1 -- 172.21.15.93:0/2274773324 shutdown_connections 2023-11-16T16:48:03.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:03.050+0000 7f2404e9e700 1 -- 172.21.15.93:0/2274773324 wait complete. 2023-11-16T16:48:03.056 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-11-16T16:48:03.573 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2023-11-16T16:48:02.538739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655355,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":453,"num_read_kb":340,"num_write":337,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":83796,"kb_used_data":14260,"kb_used_omap":0,"kb_used_meta":69440,"kb_avail":1124858028,"statfs":{"total":1151940427776,"available":1151854620672,"internally_reserved":0,"allocated":14602240,"data_stored":8098159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":71106560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002601"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616946+0000","last_change":"2023-11-16T16:46:37.150429+0000","last_active":"2023-11-16T16:47:53.616946+0000","last_peered":"2023-11-16T16:47:53.616946+0000","last_clean":"2023-11-16T16:47:53.616946+0000","last_became_active":"2023-11-16T16:46:37.150133+0000","last_became_peered":"2023-11-16T16:46:37.150133+0000","last_unstale":"2023-11-16T16:47:53.616946+0000","last_undegraded":"2023-11-16T16:47:53.616946+0000","last_fullsized":"2023-11-16T16:47:53.616946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:54:50.775465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339959+0000","last_change":"2023-11-16T16:46:37.155729+0000","last_active":"2023-11-16T16:47:53.339959+0000","last_peered":"2023-11-16T16:47:53.339959+0000","last_clean":"2023-11-16T16:47:53.339959+0000","last_became_active":"2023-11-16T16:46:37.155178+0000","last_became_peered":"2023-11-16T16:46:37.155178+0000","last_unstale":"2023-11-16T16:47:53.339959+0000","last_undegraded":"2023-11-16T16:47:53.339959+0000","last_fullsized":"2023-11-16T16:47:53.339959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:50:41.988915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.338806+0000","last_change":"2023-11-16T16:46:37.154669+0000","last_active":"2023-11-16T16:47:53.338806+0000","last_peered":"2023-11-16T16:47:53.338806+0000","last_clean":"2023-11-16T16:47:53.338806+0000","last_became_active":"2023-11-16T16:46:37.154331+0000","last_became_peered":"2023-11-16T16:46:37.154331+0000","last_unstale":"2023-11-16T16:47:53.338806+0000","last_undegraded":"2023-11-16T16:47:53.338806+0000","last_fullsized":"2023-11-16T16:47:53.338806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:31:56.646312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340670+0000","last_change":"2023-11-16T16:46:37.140627+0000","last_active":"2023-11-16T16:47:53.340670+0000","last_peered":"2023-11-16T16:47:53.340670+0000","last_clean":"2023-11-16T16:47:53.340670+0000","last_became_active":"2023-11-16T16:46:37.138386+0000","last_became_peered":"2023-11-16T16:46:37.138386+0000","last_unstale":"2023-11-16T16:47:53.340670+0000","last_undegraded":"2023-11-16T16:47:53.340670+0000","last_fullsized":"2023-11-16T16:47:53.340670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:06:28.111745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339677+0000","last_change":"2023-11-16T16:46:37.154046+0000","last_active":"2023-11-16T16:47:53.339677+0000","last_peered":"2023-11-16T16:47:53.339677+0000","last_clean":"2023-11-16T16:47:53.339677+0000","last_became_active":"2023-11-16T16:46:37.153677+0000","last_became_peered":"2023-11-16T16:46:37.153677+0000","last_unstale":"2023-11-16T16:47:53.339677+0000","last_undegraded":"2023-11-16T16:47:53.339677+0000","last_fullsized":"2023-11-16T16:47:53.339677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:09:53.042162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.394422+0000","last_change":"2023-11-16T16:46:37.153809+0000","last_active":"2023-11-16T16:47:53.394422+0000","last_peered":"2023-11-16T16:47:53.394422+0000","last_clean":"2023-11-16T16:47:53.394422+0000","last_became_active":"2023-11-16T16:46:37.153511+0000","last_became_peered":"2023-11-16T16:46:37.153511+0000","last_unstale":"2023-11-16T16:47:53.394422+0000","last_undegraded":"2023-11-16T16:47:53.394422+0000","last_fullsized":"2023-11-16T16:47:53.394422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:39:46.354193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616168+0000","last_change":"2023-11-16T16:46:37.142530+0000","last_active":"2023-11-16T16:47:53.616168+0000","last_peered":"2023-11-16T16:47:53.616168+0000","last_clean":"2023-11-16T16:47:53.616168+0000","last_became_active":"2023-11-16T16:46:37.142216+0000","last_became_peered":"2023-11-16T16:46:37.142216+0000","last_unstale":"2023-11-16T16:47:53.616168+0000","last_undegraded":"2023-11-16T16:47:53.616168+0000","last_fullsized":"2023-11-16T16:47:53.616168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:57:03.834528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343565+0000","last_change":"2023-11-16T16:46:37.151956+0000","last_active":"2023-11-16T16:47:53.343565+0000","last_peered":"2023-11-16T16:47:53.343565+0000","last_clean":"2023-11-16T16:47:53.343565+0000","last_became_active":"2023-11-16T16:46:37.149432+0000","last_became_peered":"2023-11-16T16:46:37.149432+0000","last_unstale":"2023-11-16T16:47:53.343565+0000","last_undegraded":"2023-11-16T16:47:53.343565+0000","last_fullsized":"2023-11-16T16:47:53.343565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:35:24.994585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.619544+0000","last_change":"2023-11-16T16:47:53.618515+0000","last_active":"2023-11-16T16:47:53.619544+0000","last_peered":"2023-11-16T16:47:53.619544+0000","last_clean":"2023-11-16T16:47:53.619544+0000","last_became_active":"2023-11-16T16:47:53.617627+0000","last_became_peered":"2023-11-16T16:47:53.617627+0000","last_unstale":"2023-11-16T16:47:53.619544+0000","last_undegraded":"2023-11-16T16:47:53.619544+0000","last_fullsized":"2023-11-16T16:47:53.619544+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:25:01.480977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.395019+0000","last_change":"2023-11-16T16:46:37.153895+0000","last_active":"2023-11-16T16:47:53.395019+0000","last_peered":"2023-11-16T16:47:53.395019+0000","last_clean":"2023-11-16T16:47:53.395019+0000","last_became_active":"2023-11-16T16:46:37.153491+0000","last_became_peered":"2023-11-16T16:46:37.153491+0000","last_unstale":"2023-11-16T16:47:53.395019+0000","last_undegraded":"2023-11-16T16:47:53.395019+0000","last_fullsized":"2023-11-16T16:47:53.395019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:22:53.710576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617078+0000","last_change":"2023-11-16T16:46:37.148036+0000","last_active":"2023-11-16T16:47:53.617078+0000","last_peered":"2023-11-16T16:47:53.617078+0000","last_clean":"2023-11-16T16:47:53.617078+0000","last_became_active":"2023-11-16T16:46:37.147372+0000","last_became_peered":"2023-11-16T16:46:37.147372+0000","last_unstale":"2023-11-16T16:47:53.617078+0000","last_undegraded":"2023-11-16T16:47:53.617078+0000","last_fullsized":"2023-11-16T16:47:53.617078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:57:26.086798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340480+0000","last_change":"2023-11-16T16:46:37.147120+0000","last_active":"2023-11-16T16:47:53.340480+0000","last_peered":"2023-11-16T16:47:53.340480+0000","last_clean":"2023-11-16T16:47:53.340480+0000","last_became_active":"2023-11-16T16:46:37.146615+0000","last_became_peered":"2023-11-16T16:46:37.146615+0000","last_unstale":"2023-11-16T16:47:53.340480+0000","last_undegraded":"2023-11-16T16:47:53.340480+0000","last_fullsized":"2023-11-16T16:47:53.340480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:04:59.299232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.346418+0000","last_change":"2023-11-16T16:46:37.152029+0000","last_active":"2023-11-16T16:47:53.346418+0000","last_peered":"2023-11-16T16:47:53.346418+0000","last_clean":"2023-11-16T16:47:53.346418+0000","last_became_active":"2023-11-16T16:46:37.150662+0000","last_became_peered":"2023-11-16T16:46:37.150662+0000","last_unstale":"2023-11-16T16:47:53.346418+0000","last_undegraded":"2023-11-16T16:47:53.346418+0000","last_fullsized":"2023-11-16T16:47:53.346418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:36:09.281786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339317+0000","last_change":"2023-11-16T16:46:37.160562+0000","last_active":"2023-11-16T16:47:53.339317+0000","last_peered":"2023-11-16T16:47:53.339317+0000","last_clean":"2023-11-16T16:47:53.339317+0000","last_became_active":"2023-11-16T16:46:37.158175+0000","last_became_peered":"2023-11-16T16:46:37.158175+0000","last_unstale":"2023-11-16T16:47:53.339317+0000","last_undegraded":"2023-11-16T16:47:53.339317+0000","last_fullsized":"2023-11-16T16:47:53.339317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:29:39.193105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.338703+0000","last_change":"2023-11-16T16:46:37.156381+0000","last_active":"2023-11-16T16:47:53.338703+0000","last_peered":"2023-11-16T16:47:53.338703+0000","last_clean":"2023-11-16T16:47:53.338703+0000","last_became_active":"2023-11-16T16:46:37.150366+0000","last_became_peered":"2023-11-16T16:46:37.150366+0000","last_unstale":"2023-11-16T16:47:53.338703+0000","last_undegraded":"2023-11-16T16:47:53.338703+0000","last_fullsized":"2023-11-16T16:47:53.338703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:49:26.041804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615267+0000","last_change":"2023-11-16T16:46:37.158070+0000","last_active":"2023-11-16T16:47:53.615267+0000","last_peered":"2023-11-16T16:47:53.615267+0000","last_clean":"2023-11-16T16:47:53.615267+0000","last_became_active":"2023-11-16T16:46:37.157755+0000","last_became_peered":"2023-11-16T16:46:37.157755+0000","last_unstale":"2023-11-16T16:47:53.615267+0000","last_undegraded":"2023-11-16T16:47:53.615267+0000","last_fullsized":"2023-11-16T16:47:53.615267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:18:18.304516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617095+0000","last_change":"2023-11-16T16:46:37.162616+0000","last_active":"2023-11-16T16:47:53.617095+0000","last_peered":"2023-11-16T16:47:53.617095+0000","last_clean":"2023-11-16T16:47:53.617095+0000","last_became_active":"2023-11-16T16:46:37.162281+0000","last_became_peered":"2023-11-16T16:46:37.162281+0000","last_unstale":"2023-11-16T16:47:53.617095+0000","last_undegraded":"2023-11-16T16:47:53.617095+0000","last_fullsized":"2023-11-16T16:47:53.617095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:47:35.293940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.348022+0000","last_change":"2023-11-16T16:46:37.150337+0000","last_active":"2023-11-16T16:47:53.348022+0000","last_peered":"2023-11-16T16:47:53.348022+0000","last_clean":"2023-11-16T16:47:53.348022+0000","last_became_active":"2023-11-16T16:46:37.148826+0000","last_became_peered":"2023-11-16T16:46:37.148826+0000","last_unstale":"2023-11-16T16:47:53.348022+0000","last_undegraded":"2023-11-16T16:47:53.348022+0000","last_fullsized":"2023-11-16T16:47:53.348022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:26:33.021039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616437+0000","last_change":"2023-11-16T16:46:37.150253+0000","last_active":"2023-11-16T16:47:53.616437+0000","last_peered":"2023-11-16T16:47:53.616437+0000","last_clean":"2023-11-16T16:47:53.616437+0000","last_became_active":"2023-11-16T16:46:37.149962+0000","last_became_peered":"2023-11-16T16:46:37.149962+0000","last_unstale":"2023-11-16T16:47:53.616437+0000","last_undegraded":"2023-11-16T16:47:53.616437+0000","last_fullsized":"2023-11-16T16:47:53.616437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:39:37.347781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616251+0000","last_change":"2023-11-16T16:46:41.177888+0000","last_active":"2023-11-16T16:47:53.616251+0000","last_peered":"2023-11-16T16:47:53.616251+0000","last_clean":"2023-11-16T16:47:53.616251+0000","last_became_active":"2023-11-16T16:46:41.177442+0000","last_became_peered":"2023-11-16T16:46:41.177442+0000","last_unstale":"2023-11-16T16:47:53.616251+0000","last_undegraded":"2023-11-16T16:47:53.616251+0000","last_fullsized":"2023-11-16T16:47:53.616251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:53:10.830975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.347927+0000","last_change":"2023-11-16T16:46:36.595769+0000","last_active":"2023-11-16T16:47:53.347927+0000","last_peered":"2023-11-16T16:47:53.347927+0000","last_clean":"2023-11-16T16:47:53.347927+0000","last_became_active":"2023-11-16T16:46:36.593780+0000","last_became_peered":"2023-11-16T16:46:36.593780+0000","last_unstale":"2023-11-16T16:47:53.347927+0000","last_undegraded":"2023-11-16T16:47:53.347927+0000","last_fullsized":"2023-11-16T16:47:53.347927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:01:41.376943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343701+0000","last_change":"2023-11-16T16:46:37.146683+0000","last_active":"2023-11-16T16:47:53.343701+0000","last_peered":"2023-11-16T16:47:53.343701+0000","last_clean":"2023-11-16T16:47:53.343701+0000","last_became_active":"2023-11-16T16:46:37.146363+0000","last_became_peered":"2023-11-16T16:46:37.146363+0000","last_unstale":"2023-11-16T16:47:53.343701+0000","last_undegraded":"2023-11-16T16:47:53.343701+0000","last_fullsized":"2023-11-16T16:47:53.343701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:29:13.834403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339895+0000","last_change":"2023-11-16T16:46:41.174942+0000","last_active":"2023-11-16T16:47:53.339895+0000","last_peered":"2023-11-16T16:47:53.339895+0000","last_clean":"2023-11-16T16:47:53.339895+0000","last_became_active":"2023-11-16T16:46:41.174483+0000","last_became_peered":"2023-11-16T16:46:41.174483+0000","last_unstale":"2023-11-16T16:47:53.339895+0000","last_undegraded":"2023-11-16T16:47:53.339895+0000","last_fullsized":"2023-11-16T16:47:53.339895+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:18:17.625371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343903+0000","last_change":"2023-11-16T16:46:36.607478+0000","last_active":"2023-11-16T16:47:53.343903+0000","last_peered":"2023-11-16T16:47:53.343903+0000","last_clean":"2023-11-16T16:47:53.343903+0000","last_became_active":"2023-11-16T16:46:36.607137+0000","last_became_peered":"2023-11-16T16:46:36.607137+0000","last_unstale":"2023-11-16T16:47:53.343903+0000","last_undegraded":"2023-11-16T16:47:53.343903+0000","last_fullsized":"2023-11-16T16:47:53.343903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:01:51.466667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339746+0000","last_change":"2023-11-16T16:46:37.148576+0000","last_active":"2023-11-16T16:47:53.339746+0000","last_peered":"2023-11-16T16:47:53.339746+0000","last_clean":"2023-11-16T16:47:53.339746+0000","last_became_active":"2023-11-16T16:46:37.148284+0000","last_became_peered":"2023-11-16T16:46:37.148284+0000","last_unstale":"2023-11-16T16:47:53.339746+0000","last_undegraded":"2023-11-16T16:47:53.339746+0000","last_fullsized":"2023-11-16T16:47:53.339746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:07:30.298369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340210+0000","last_change":"2023-11-16T16:46:41.178717+0000","last_active":"2023-11-16T16:47:53.340210+0000","last_peered":"2023-11-16T16:47:53.340210+0000","last_clean":"2023-11-16T16:47:53.340210+0000","last_became_active":"2023-11-16T16:46:41.178350+0000","last_became_peered":"2023-11-16T16:46:41.178350+0000","last_unstale":"2023-11-16T16:47:53.340210+0000","last_undegraded":"2023-11-16T16:47:53.340210+0000","last_fullsized":"2023-11-16T16:47:53.340210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:27:43.240599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616800+0000","last_change":"2023-11-16T16:46:52.639272+0000","last_active":"2023-11-16T16:47:53.616800+0000","last_peered":"2023-11-16T16:47:53.616800+0000","last_clean":"2023-11-16T16:47:53.616800+0000","last_became_active":"2023-11-16T16:46:52.338319+0000","last_became_peered":"2023-11-16T16:46:52.338319+0000","last_unstale":"2023-11-16T16:47:53.616800+0000","last_undegraded":"2023-11-16T16:47:53.616800+0000","last_fullsized":"2023-11-16T16:47:53.616800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:38:42.664671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615916+0000","last_change":"2023-11-16T16:46:37.153404+0000","last_active":"2023-11-16T16:47:53.615916+0000","last_peered":"2023-11-16T16:47:53.615916+0000","last_clean":"2023-11-16T16:47:53.615916+0000","last_became_active":"2023-11-16T16:46:37.153064+0000","last_became_peered":"2023-11-16T16:46:37.153064+0000","last_unstale":"2023-11-16T16:47:53.615916+0000","last_undegraded":"2023-11-16T16:47:53.615916+0000","last_fullsized":"2023-11-16T16:47:53.615916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:58:18.745061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339998+0000","last_change":"2023-11-16T16:46:41.163307+0000","last_active":"2023-11-16T16:47:53.339998+0000","last_peered":"2023-11-16T16:47:53.339998+0000","last_clean":"2023-11-16T16:47:53.339998+0000","last_became_active":"2023-11-16T16:46:41.162822+0000","last_became_peered":"2023-11-16T16:46:41.162822+0000","last_unstale":"2023-11-16T16:47:53.339998+0000","last_undegraded":"2023-11-16T16:47:53.339998+0000","last_fullsized":"2023-11-16T16:47:53.339998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:24:50.552900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339961+0000","last_change":"2023-11-16T16:46:36.595905+0000","last_active":"2023-11-16T16:47:53.339961+0000","last_peered":"2023-11-16T16:47:53.339961+0000","last_clean":"2023-11-16T16:47:53.339961+0000","last_became_active":"2023-11-16T16:46:36.595403+0000","last_became_peered":"2023-11-16T16:46:36.595403+0000","last_unstale":"2023-11-16T16:47:53.339961+0000","last_undegraded":"2023-11-16T16:47:53.339961+0000","last_fullsized":"2023-11-16T16:47:53.339961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:35:19.238263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343480+0000","last_change":"2023-11-16T16:46:52.334655+0000","last_active":"2023-11-16T16:47:53.343480+0000","last_peered":"2023-11-16T16:47:53.343480+0000","last_clean":"2023-11-16T16:47:53.343480+0000","last_became_active":"2023-11-16T16:46:52.334345+0000","last_became_peered":"2023-11-16T16:46:52.334345+0000","last_unstale":"2023-11-16T16:47:53.343480+0000","last_undegraded":"2023-11-16T16:47:53.343480+0000","last_fullsized":"2023-11-16T16:47:53.343480+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:23:02.755769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616229+0000","last_change":"2023-11-16T16:46:41.169041+0000","last_active":"2023-11-16T16:47:53.616229+0000","last_peered":"2023-11-16T16:47:53.616229+0000","last_clean":"2023-11-16T16:47:53.616229+0000","last_became_active":"2023-11-16T16:46:41.168589+0000","last_became_peered":"2023-11-16T16:46:41.168589+0000","last_unstale":"2023-11-16T16:47:53.616229+0000","last_undegraded":"2023-11-16T16:47:53.616229+0000","last_fullsized":"2023-11-16T16:47:53.616229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:46:36.805567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":87,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:58.403336+0000","last_change":"2023-11-16T16:46:36.595442+0000","last_active":"2023-11-16T16:47:58.403336+0000","last_peered":"2023-11-16T16:47:58.403336+0000","last_clean":"2023-11-16T16:47:58.403336+0000","last_became_active":"2023-11-16T16:46:36.595122+0000","last_became_peered":"2023-11-16T16:46:36.595122+0000","last_unstale":"2023-11-16T16:47:58.403336+0000","last_undegraded":"2023-11-16T16:47:58.403336+0000","last_fullsized":"2023-11-16T16:47:58.403336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:26:46.774071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.618090+0000","last_change":"2023-11-16T16:47:53.618090+0000","last_active":"2023-11-16T16:47:53.618090+0000","last_peered":"2023-11-16T16:47:53.618090+0000","last_clean":"2023-11-16T16:47:53.618090+0000","last_became_active":"2023-11-16T16:47:53.617686+0000","last_became_peered":"2023-11-16T16:47:53.617686+0000","last_unstale":"2023-11-16T16:47:53.618090+0000","last_undegraded":"2023-11-16T16:47:53.618090+0000","last_fullsized":"2023-11-16T16:47:53.618090+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:44:16.967004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616334+0000","last_change":"2023-11-16T16:46:41.185001+0000","last_active":"2023-11-16T16:47:53.616334+0000","last_peered":"2023-11-16T16:47:53.616334+0000","last_clean":"2023-11-16T16:47:53.616334+0000","last_became_active":"2023-11-16T16:46:41.184603+0000","last_became_peered":"2023-11-16T16:46:41.184603+0000","last_unstale":"2023-11-16T16:47:53.616334+0000","last_undegraded":"2023-11-16T16:47:53.616334+0000","last_fullsized":"2023-11-16T16:47:53.616334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:56:54.738072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.394424+0000","last_change":"2023-11-16T16:46:36.611365+0000","last_active":"2023-11-16T16:47:53.394424+0000","last_peered":"2023-11-16T16:47:53.394424+0000","last_clean":"2023-11-16T16:47:53.394424+0000","last_became_active":"2023-11-16T16:46:36.611006+0000","last_became_peered":"2023-11-16T16:46:36.611006+0000","last_unstale":"2023-11-16T16:47:53.394424+0000","last_undegraded":"2023-11-16T16:47:53.394424+0000","last_fullsized":"2023-11-16T16:47:53.394424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:24:21.225507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.348823+0000","last_change":"2023-11-16T16:47:53.348823+0000","last_active":"2023-11-16T16:47:53.348823+0000","last_peered":"2023-11-16T16:47:53.348823+0000","last_clean":"2023-11-16T16:47:53.348823+0000","last_became_active":"2023-11-16T16:47:53.348523+0000","last_became_peered":"2023-11-16T16:47:53.348523+0000","last_unstale":"2023-11-16T16:47:53.348823+0000","last_undegraded":"2023-11-16T16:47:53.348823+0000","last_fullsized":"2023-11-16T16:47:53.348823+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:05:08.796096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615900+0000","last_change":"2023-11-16T16:46:41.182374+0000","last_active":"2023-11-16T16:47:53.615900+0000","last_peered":"2023-11-16T16:47:53.615900+0000","last_clean":"2023-11-16T16:47:53.615900+0000","last_became_active":"2023-11-16T16:46:41.181977+0000","last_became_peered":"2023-11-16T16:46:41.181977+0000","last_unstale":"2023-11-16T16:47:53.615900+0000","last_undegraded":"2023-11-16T16:47:53.615900+0000","last_fullsized":"2023-11-16T16:47:53.615900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:42:20.748527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.346218+0000","last_change":"2023-11-16T16:46:36.607647+0000","last_active":"2023-11-16T16:47:53.346218+0000","last_peered":"2023-11-16T16:47:53.346218+0000","last_clean":"2023-11-16T16:47:53.346218+0000","last_became_active":"2023-11-16T16:46:36.607282+0000","last_became_peered":"2023-11-16T16:46:36.607282+0000","last_unstale":"2023-11-16T16:47:53.346218+0000","last_undegraded":"2023-11-16T16:47:53.346218+0000","last_fullsized":"2023-11-16T16:47:53.346218+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:27:41.957383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343442+0000","last_change":"2023-11-16T16:46:37.152659+0000","last_active":"2023-11-16T16:47:53.343442+0000","last_peered":"2023-11-16T16:47:53.343442+0000","last_clean":"2023-11-16T16:47:53.343442+0000","last_became_active":"2023-11-16T16:46:37.152313+0000","last_became_peered":"2023-11-16T16:46:37.152313+0000","last_unstale":"2023-11-16T16:47:53.343442+0000","last_undegraded":"2023-11-16T16:47:53.343442+0000","last_fullsized":"2023-11-16T16:47:53.343442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:41:24.243494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339825+0000","last_change":"2023-11-16T16:46:41.167312+0000","last_active":"2023-11-16T16:47:53.339825+0000","last_peered":"2023-11-16T16:47:53.339825+0000","last_clean":"2023-11-16T16:47:53.339825+0000","last_became_active":"2023-11-16T16:46:41.166211+0000","last_became_peered":"2023-11-16T16:46:41.166211+0000","last_unstale":"2023-11-16T16:47:53.339825+0000","last_undegraded":"2023-11-16T16:47:53.339825+0000","last_fullsized":"2023-11-16T16:47:53.339825+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:04:27.032057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342694+0000","last_change":"2023-11-16T16:46:36.595888+0000","last_active":"2023-11-16T16:47:53.342694+0000","last_peered":"2023-11-16T16:47:53.342694+0000","last_clean":"2023-11-16T16:47:53.342694+0000","last_became_active":"2023-11-16T16:46:36.595602+0000","last_became_peered":"2023-11-16T16:46:36.595602+0000","last_unstale":"2023-11-16T16:47:53.342694+0000","last_undegraded":"2023-11-16T16:47:53.342694+0000","last_fullsized":"2023-11-16T16:47:53.342694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:04:33.124818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.344885+0000","last_change":"2023-11-16T16:46:37.154431+0000","last_active":"2023-11-16T16:47:53.344885+0000","last_peered":"2023-11-16T16:47:53.344885+0000","last_clean":"2023-11-16T16:47:53.344885+0000","last_became_active":"2023-11-16T16:46:37.154106+0000","last_became_peered":"2023-11-16T16:46:37.154106+0000","last_unstale":"2023-11-16T16:47:53.344885+0000","last_undegraded":"2023-11-16T16:47:53.344885+0000","last_fullsized":"2023-11-16T16:47:53.344885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:35:20.330822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343030+0000","last_change":"2023-11-16T16:46:41.168644+0000","last_active":"2023-11-16T16:47:53.343030+0000","last_peered":"2023-11-16T16:47:53.343030+0000","last_clean":"2023-11-16T16:47:53.343030+0000","last_became_active":"2023-11-16T16:46:41.168121+0000","last_became_peered":"2023-11-16T16:46:41.168121+0000","last_unstale":"2023-11-16T16:47:53.343030+0000","last_undegraded":"2023-11-16T16:47:53.343030+0000","last_fullsized":"2023-11-16T16:47:53.343030+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:37:47.753436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":82,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:57.401057+0000","last_change":"2023-11-16T16:46:36.597293+0000","last_active":"2023-11-16T16:47:57.401057+0000","last_peered":"2023-11-16T16:47:57.401057+0000","last_clean":"2023-11-16T16:47:57.401057+0000","last_became_active":"2023-11-16T16:46:36.596992+0000","last_became_peered":"2023-11-16T16:46:36.596992+0000","last_unstale":"2023-11-16T16:47:57.401057+0000","last_undegraded":"2023-11-16T16:47:57.401057+0000","last_fullsized":"2023-11-16T16:47:57.401057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:10:25.375233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616418+0000","last_change":"2023-11-16T16:46:37.149339+0000","last_active":"2023-11-16T16:47:53.616418+0000","last_peered":"2023-11-16T16:47:53.616418+0000","last_clean":"2023-11-16T16:47:53.616418+0000","last_became_active":"2023-11-16T16:46:37.149000+0000","last_became_peered":"2023-11-16T16:46:37.149000+0000","last_unstale":"2023-11-16T16:47:53.616418+0000","last_undegraded":"2023-11-16T16:47:53.616418+0000","last_fullsized":"2023-11-16T16:47:53.616418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:55:38.165691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339296+0000","last_change":"2023-11-16T16:46:52.338731+0000","last_active":"2023-11-16T16:47:53.339296+0000","last_peered":"2023-11-16T16:47:53.339296+0000","last_clean":"2023-11-16T16:47:53.339296+0000","last_became_active":"2023-11-16T16:46:52.338254+0000","last_became_peered":"2023-11-16T16:46:52.338254+0000","last_unstale":"2023-11-16T16:47:53.339296+0000","last_undegraded":"2023-11-16T16:47:53.339296+0000","last_fullsized":"2023-11-16T16:47:53.339296+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:51:39.487241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617194+0000","last_change":"2023-11-16T16:46:36.595407+0000","last_active":"2023-11-16T16:47:53.617194+0000","last_peered":"2023-11-16T16:47:53.617194+0000","last_clean":"2023-11-16T16:47:53.617194+0000","last_became_active":"2023-11-16T16:46:36.595132+0000","last_became_peered":"2023-11-16T16:46:36.595132+0000","last_unstale":"2023-11-16T16:47:53.617194+0000","last_undegraded":"2023-11-16T16:47:53.617194+0000","last_fullsized":"2023-11-16T16:47:53.617194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:16:48.765412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.619303+0000","last_change":"2023-11-16T16:47:53.619303+0000","last_active":"2023-11-16T16:47:53.619303+0000","last_peered":"2023-11-16T16:47:53.619303+0000","last_clean":"2023-11-16T16:47:53.619303+0000","last_became_active":"2023-11-16T16:47:53.618438+0000","last_became_peered":"2023-11-16T16:47:53.618438+0000","last_unstale":"2023-11-16T16:47:53.619303+0000","last_undegraded":"2023-11-16T16:47:53.619303+0000","last_fullsized":"2023-11-16T16:47:53.619303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:41:13.994013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340737+0000","last_change":"2023-11-16T16:46:52.339033+0000","last_active":"2023-11-16T16:47:53.340737+0000","last_peered":"2023-11-16T16:47:53.340737+0000","last_clean":"2023-11-16T16:47:53.340737+0000","last_became_active":"2023-11-16T16:46:52.338451+0000","last_became_peered":"2023-11-16T16:46:52.338451+0000","last_unstale":"2023-11-16T16:47:53.340737+0000","last_undegraded":"2023-11-16T16:47:53.340737+0000","last_fullsized":"2023-11-16T16:47:53.340737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:54:56.472106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343009+0000","last_change":"2023-11-16T16:46:36.596779+0000","last_active":"2023-11-16T16:47:53.343009+0000","last_peered":"2023-11-16T16:47:53.343009+0000","last_clean":"2023-11-16T16:47:53.343009+0000","last_became_active":"2023-11-16T16:46:36.596460+0000","last_became_peered":"2023-11-16T16:46:36.596460+0000","last_unstale":"2023-11-16T16:47:53.343009+0000","last_undegraded":"2023-11-16T16:47:53.343009+0000","last_fullsized":"2023-11-16T16:47:53.343009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:55:26.723679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616040+0000","last_change":"2023-11-16T16:46:52.427912+0000","last_active":"2023-11-16T16:47:53.616040+0000","last_peered":"2023-11-16T16:47:53.616040+0000","last_clean":"2023-11-16T16:47:53.616040+0000","last_became_active":"2023-11-16T16:46:52.427426+0000","last_became_peered":"2023-11-16T16:46:52.427426+0000","last_unstale":"2023-11-16T16:47:53.616040+0000","last_undegraded":"2023-11-16T16:47:53.616040+0000","last_fullsized":"2023-11-16T16:47:53.616040+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:11:47.521498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343532+0000","last_change":"2023-11-16T16:46:41.177483+0000","last_active":"2023-11-16T16:47:53.343532+0000","last_peered":"2023-11-16T16:47:53.343532+0000","last_clean":"2023-11-16T16:47:53.343532+0000","last_became_active":"2023-11-16T16:46:41.173165+0000","last_became_peered":"2023-11-16T16:46:41.173165+0000","last_unstale":"2023-11-16T16:47:53.343532+0000","last_undegraded":"2023-11-16T16:47:53.343532+0000","last_fullsized":"2023-11-16T16:47:53.343532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:16:58.604325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'32","reported_seq":168,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:58.402650+0000","last_change":"2023-11-16T16:46:36.598279+0000","last_active":"2023-11-16T16:47:58.402650+0000","last_peered":"2023-11-16T16:47:58.402650+0000","last_clean":"2023-11-16T16:47:58.402650+0000","last_became_active":"2023-11-16T16:46:36.597975+0000","last_became_peered":"2023-11-16T16:46:36.597975+0000","last_unstale":"2023-11-16T16:47:58.402650+0000","last_undegraded":"2023-11-16T16:47:58.402650+0000","last_fullsized":"2023-11-16T16:47:58.402650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:39:42.505182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60837,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":106,"num_read_kb":112,"num_write":33,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340272+0000","last_change":"2023-11-16T16:46:37.143014+0000","last_active":"2023-11-16T16:47:53.340272+0000","last_peered":"2023-11-16T16:47:53.340272+0000","last_clean":"2023-11-16T16:47:53.340272+0000","last_became_active":"2023-11-16T16:46:37.142662+0000","last_became_peered":"2023-11-16T16:46:37.142662+0000","last_unstale":"2023-11-16T16:47:53.340272+0000","last_undegraded":"2023-11-16T16:47:53.340272+0000","last_fullsized":"2023-11-16T16:47:53.340272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:28:38.845881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343681+0000","last_change":"2023-11-16T16:46:41.169921+0000","last_active":"2023-11-16T16:47:53.343681+0000","last_peered":"2023-11-16T16:47:53.343681+0000","last_clean":"2023-11-16T16:47:53.343681+0000","last_became_active":"2023-11-16T16:46:41.169037+0000","last_became_peered":"2023-11-16T16:46:41.169037+0000","last_unstale":"2023-11-16T16:47:53.343681+0000","last_undegraded":"2023-11-16T16:47:53.343681+0000","last_fullsized":"2023-11-16T16:47:53.343681+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:46:12.638029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616096+0000","last_change":"2023-11-16T16:46:36.596159+0000","last_active":"2023-11-16T16:47:53.616096+0000","last_peered":"2023-11-16T16:47:53.616096+0000","last_clean":"2023-11-16T16:47:53.616096+0000","last_became_active":"2023-11-16T16:46:36.595847+0000","last_became_peered":"2023-11-16T16:46:36.595847+0000","last_unstale":"2023-11-16T16:47:53.616096+0000","last_undegraded":"2023-11-16T16:47:53.616096+0000","last_fullsized":"2023-11-16T16:47:53.616096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:36:29.758981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.344201+0000","last_change":"2023-11-16T16:46:37.156046+0000","last_active":"2023-11-16T16:47:53.344201+0000","last_peered":"2023-11-16T16:47:53.344201+0000","last_clean":"2023-11-16T16:47:53.344201+0000","last_became_active":"2023-11-16T16:46:37.155657+0000","last_became_peered":"2023-11-16T16:46:37.155657+0000","last_unstale":"2023-11-16T16:47:53.344201+0000","last_undegraded":"2023-11-16T16:47:53.344201+0000","last_fullsized":"2023-11-16T16:47:53.344201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:18:49.240097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615915+0000","last_change":"2023-11-16T16:46:41.182485+0000","last_active":"2023-11-16T16:47:53.615915+0000","last_peered":"2023-11-16T16:47:53.615915+0000","last_clean":"2023-11-16T16:47:53.615915+0000","last_became_active":"2023-11-16T16:46:41.181990+0000","last_became_peered":"2023-11-16T16:46:41.181990+0000","last_unstale":"2023-11-16T16:47:53.615915+0000","last_undegraded":"2023-11-16T16:47:53.615915+0000","last_fullsized":"2023-11-16T16:47:53.615915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:34:53.010727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343074+0000","last_change":"2023-11-16T16:46:36.608501+0000","last_active":"2023-11-16T16:47:53.343074+0000","last_peered":"2023-11-16T16:47:53.343074+0000","last_clean":"2023-11-16T16:47:53.343074+0000","last_became_active":"2023-11-16T16:46:36.608210+0000","last_became_peered":"2023-11-16T16:46:36.608210+0000","last_unstale":"2023-11-16T16:47:53.343074+0000","last_undegraded":"2023-11-16T16:47:53.343074+0000","last_fullsized":"2023-11-16T16:47:53.343074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:59:43.646005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616810+0000","last_change":"2023-11-16T16:46:37.155431+0000","last_active":"2023-11-16T16:47:53.616810+0000","last_peered":"2023-11-16T16:47:53.616810+0000","last_clean":"2023-11-16T16:47:53.616810+0000","last_became_active":"2023-11-16T16:46:37.155106+0000","last_became_peered":"2023-11-16T16:46:37.155106+0000","last_unstale":"2023-11-16T16:47:53.616810+0000","last_undegraded":"2023-11-16T16:47:53.616810+0000","last_fullsized":"2023-11-16T16:47:53.616810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:37:09.743670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.348242+0000","last_change":"2023-11-16T16:46:41.180903+0000","last_active":"2023-11-16T16:47:53.348242+0000","last_peered":"2023-11-16T16:47:53.348242+0000","last_clean":"2023-11-16T16:47:53.348242+0000","last_became_active":"2023-11-16T16:46:41.179842+0000","last_became_peered":"2023-11-16T16:46:41.179842+0000","last_unstale":"2023-11-16T16:47:53.348242+0000","last_undegraded":"2023-11-16T16:47:53.348242+0000","last_fullsized":"2023-11-16T16:47:53.348242+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:38:02.957328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":89,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:48:00.399855+0000","last_change":"2023-11-16T16:46:36.592436+0000","last_active":"2023-11-16T16:48:00.399855+0000","last_peered":"2023-11-16T16:48:00.399855+0000","last_clean":"2023-11-16T16:48:00.399855+0000","last_became_active":"2023-11-16T16:46:36.592078+0000","last_became_peered":"2023-11-16T16:46:36.592078+0000","last_unstale":"2023-11-16T16:48:00.399855+0000","last_undegraded":"2023-11-16T16:48:00.399855+0000","last_fullsized":"2023-11-16T16:48:00.399855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:09:44.438155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616011+0000","last_change":"2023-11-16T16:46:37.153071+0000","last_active":"2023-11-16T16:47:53.616011+0000","last_peered":"2023-11-16T16:47:53.616011+0000","last_clean":"2023-11-16T16:47:53.616011+0000","last_became_active":"2023-11-16T16:46:37.152742+0000","last_became_peered":"2023-11-16T16:46:37.152742+0000","last_unstale":"2023-11-16T16:47:53.616011+0000","last_undegraded":"2023-11-16T16:47:53.616011+0000","last_fullsized":"2023-11-16T16:47:53.616011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:13:23.845741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343289+0000","last_change":"2023-11-16T16:46:41.176434+0000","last_active":"2023-11-16T16:47:53.343289+0000","last_peered":"2023-11-16T16:47:53.343289+0000","last_clean":"2023-11-16T16:47:53.343289+0000","last_became_active":"2023-11-16T16:46:41.175771+0000","last_became_peered":"2023-11-16T16:46:41.175771+0000","last_unstale":"2023-11-16T16:47:53.343289+0000","last_undegraded":"2023-11-16T16:47:53.343289+0000","last_fullsized":"2023-11-16T16:47:53.343289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:41:28.313123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339373+0000","last_change":"2023-11-16T16:46:36.594148+0000","last_active":"2023-11-16T16:47:53.339373+0000","last_peered":"2023-11-16T16:47:53.339373+0000","last_clean":"2023-11-16T16:47:53.339373+0000","last_became_active":"2023-11-16T16:46:36.593437+0000","last_became_peered":"2023-11-16T16:46:36.593437+0000","last_unstale":"2023-11-16T16:47:53.339373+0000","last_undegraded":"2023-11-16T16:47:53.339373+0000","last_fullsized":"2023-11-16T16:47:53.339373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:39:28.474515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343308+0000","last_change":"2023-11-16T16:46:37.149044+0000","last_active":"2023-11-16T16:47:53.343308+0000","last_peered":"2023-11-16T16:47:53.343308+0000","last_clean":"2023-11-16T16:47:53.343308+0000","last_became_active":"2023-11-16T16:46:37.147215+0000","last_became_peered":"2023-11-16T16:46:37.147215+0000","last_unstale":"2023-11-16T16:47:53.343308+0000","last_undegraded":"2023-11-16T16:47:53.343308+0000","last_fullsized":"2023-11-16T16:47:53.343308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:52:29.239788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342546+0000","last_change":"2023-11-16T16:46:41.168754+0000","last_active":"2023-11-16T16:47:53.342546+0000","last_peered":"2023-11-16T16:47:53.342546+0000","last_clean":"2023-11-16T16:47:53.342546+0000","last_became_active":"2023-11-16T16:46:41.168056+0000","last_became_peered":"2023-11-16T16:46:41.168056+0000","last_unstale":"2023-11-16T16:47:53.342546+0000","last_undegraded":"2023-11-16T16:47:53.342546+0000","last_fullsized":"2023-11-16T16:47:53.342546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:06:38.901207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340338+0000","last_change":"2023-11-16T16:46:36.594892+0000","last_active":"2023-11-16T16:47:53.340338+0000","last_peered":"2023-11-16T16:47:53.340338+0000","last_clean":"2023-11-16T16:47:53.340338+0000","last_became_active":"2023-11-16T16:46:36.594557+0000","last_became_peered":"2023-11-16T16:46:36.594557+0000","last_unstale":"2023-11-16T16:47:53.340338+0000","last_undegraded":"2023-11-16T16:47:53.340338+0000","last_fullsized":"2023-11-16T16:47:53.340338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:31:22.201870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616072+0000","last_change":"2023-11-16T16:46:37.148203+0000","last_active":"2023-11-16T16:47:53.616072+0000","last_peered":"2023-11-16T16:47:53.616072+0000","last_clean":"2023-11-16T16:47:53.616072+0000","last_became_active":"2023-11-16T16:46:37.147846+0000","last_became_peered":"2023-11-16T16:46:37.147846+0000","last_unstale":"2023-11-16T16:47:53.616072+0000","last_undegraded":"2023-11-16T16:47:53.616072+0000","last_fullsized":"2023-11-16T16:47:53.616072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:01:32.063666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339599+0000","last_change":"2023-11-16T16:46:53.046150+0000","last_active":"2023-11-16T16:47:53.339599+0000","last_peered":"2023-11-16T16:47:53.339599+0000","last_clean":"2023-11-16T16:47:53.339599+0000","last_became_active":"2023-11-16T16:46:53.045619+0000","last_became_peered":"2023-11-16T16:46:53.045619+0000","last_unstale":"2023-11-16T16:47:53.339599+0000","last_undegraded":"2023-11-16T16:47:53.339599+0000","last_fullsized":"2023-11-16T16:47:53.339599+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:58:25.709808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.347053+0000","last_change":"2023-11-16T16:46:36.600779+0000","last_active":"2023-11-16T16:47:53.347053+0000","last_peered":"2023-11-16T16:47:53.347053+0000","last_clean":"2023-11-16T16:47:53.347053+0000","last_became_active":"2023-11-16T16:46:36.598664+0000","last_became_peered":"2023-11-16T16:46:36.598664+0000","last_unstale":"2023-11-16T16:47:53.347053+0000","last_undegraded":"2023-11-16T16:47:53.347053+0000","last_fullsized":"2023-11-16T16:47:53.347053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:41:36.477468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.394758+0000","last_change":"2023-11-16T16:46:37.155694+0000","last_active":"2023-11-16T16:47:53.394758+0000","last_peered":"2023-11-16T16:47:53.394758+0000","last_clean":"2023-11-16T16:47:53.394758+0000","last_became_active":"2023-11-16T16:46:37.155105+0000","last_became_peered":"2023-11-16T16:46:37.155105+0000","last_unstale":"2023-11-16T16:47:53.394758+0000","last_undegraded":"2023-11-16T16:47:53.394758+0000","last_fullsized":"2023-11-16T16:47:53.394758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:23:05.949449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.345451+0000","last_change":"2023-11-16T16:46:41.172161+0000","last_active":"2023-11-16T16:47:53.345451+0000","last_peered":"2023-11-16T16:47:53.345451+0000","last_clean":"2023-11-16T16:47:53.345451+0000","last_became_active":"2023-11-16T16:46:41.169661+0000","last_became_peered":"2023-11-16T16:46:41.169661+0000","last_unstale":"2023-11-16T16:47:53.345451+0000","last_undegraded":"2023-11-16T16:47:53.345451+0000","last_fullsized":"2023-11-16T16:47:53.345451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:02:08.887753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616617+0000","last_change":"2023-11-16T16:46:36.611376+0000","last_active":"2023-11-16T16:47:53.616617+0000","last_peered":"2023-11-16T16:47:53.616617+0000","last_clean":"2023-11-16T16:47:53.616617+0000","last_became_active":"2023-11-16T16:46:36.611045+0000","last_became_peered":"2023-11-16T16:46:36.611045+0000","last_unstale":"2023-11-16T16:47:53.616617+0000","last_undegraded":"2023-11-16T16:47:53.616617+0000","last_fullsized":"2023-11-16T16:47:53.616617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:39:01.262829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.394469+0000","last_change":"2023-11-16T16:46:37.164850+0000","last_active":"2023-11-16T16:47:53.394469+0000","last_peered":"2023-11-16T16:47:53.394469+0000","last_clean":"2023-11-16T16:47:53.394469+0000","last_became_active":"2023-11-16T16:46:37.164532+0000","last_became_peered":"2023-11-16T16:46:37.164532+0000","last_unstale":"2023-11-16T16:47:53.394469+0000","last_undegraded":"2023-11-16T16:47:53.394469+0000","last_fullsized":"2023-11-16T16:47:53.394469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:32:19.174367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343351+0000","last_change":"2023-11-16T16:46:41.203618+0000","last_active":"2023-11-16T16:47:53.343351+0000","last_peered":"2023-11-16T16:47:53.343351+0000","last_clean":"2023-11-16T16:47:53.343351+0000","last_became_active":"2023-11-16T16:46:41.202322+0000","last_became_peered":"2023-11-16T16:46:41.202322+0000","last_unstale":"2023-11-16T16:47:53.343351+0000","last_undegraded":"2023-11-16T16:47:53.343351+0000","last_fullsized":"2023-11-16T16:47:53.343351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:52:45.238718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343138+0000","last_change":"2023-11-16T16:46:37.154064+0000","last_active":"2023-11-16T16:47:53.343138+0000","last_peered":"2023-11-16T16:47:53.343138+0000","last_clean":"2023-11-16T16:47:53.343138+0000","last_became_active":"2023-11-16T16:46:37.153776+0000","last_became_peered":"2023-11-16T16:46:37.153776+0000","last_unstale":"2023-11-16T16:47:53.343138+0000","last_undegraded":"2023-11-16T16:47:53.343138+0000","last_fullsized":"2023-11-16T16:47:53.343138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:05:27.979843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":85,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:59.397262+0000","last_change":"2023-11-16T16:46:36.612613+0000","last_active":"2023-11-16T16:47:59.397262+0000","last_peered":"2023-11-16T16:47:59.397262+0000","last_clean":"2023-11-16T16:47:59.397262+0000","last_became_active":"2023-11-16T16:46:36.612303+0000","last_became_peered":"2023-11-16T16:46:36.612303+0000","last_unstale":"2023-11-16T16:47:59.397262+0000","last_undegraded":"2023-11-16T16:47:59.397262+0000","last_fullsized":"2023-11-16T16:47:59.397262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:01:35.412661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616882+0000","last_change":"2023-11-16T16:46:53.046753+0000","last_active":"2023-11-16T16:47:53.616882+0000","last_peered":"2023-11-16T16:47:53.616882+0000","last_clean":"2023-11-16T16:47:53.616882+0000","last_became_active":"2023-11-16T16:46:53.046231+0000","last_became_peered":"2023-11-16T16:46:53.046231+0000","last_unstale":"2023-11-16T16:47:53.616882+0000","last_undegraded":"2023-11-16T16:47:53.616882+0000","last_fullsized":"2023-11-16T16:47:53.616882+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:45:28.669638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617623+0000","last_change":"2023-11-16T16:47:53.617623+0000","last_active":"2023-11-16T16:47:53.617623+0000","last_peered":"2023-11-16T16:47:53.617623+0000","last_clean":"2023-11-16T16:47:53.617623+0000","last_became_active":"2023-11-16T16:47:53.617223+0000","last_became_peered":"2023-11-16T16:47:53.617223+0000","last_unstale":"2023-11-16T16:47:53.617623+0000","last_undegraded":"2023-11-16T16:47:53.617623+0000","last_fullsized":"2023-11-16T16:47:53.617623+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:50:23.328919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340296+0000","last_change":"2023-11-16T16:46:36.597520+0000","last_active":"2023-11-16T16:47:53.340296+0000","last_peered":"2023-11-16T16:47:53.340296+0000","last_clean":"2023-11-16T16:47:53.340296+0000","last_became_active":"2023-11-16T16:46:36.596542+0000","last_became_peered":"2023-11-16T16:46:36.596542+0000","last_unstale":"2023-11-16T16:47:53.340296+0000","last_undegraded":"2023-11-16T16:47:53.340296+0000","last_fullsized":"2023-11-16T16:47:53.340296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:19:23.260941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615707+0000","last_change":"2023-11-16T16:45:27.937850+0000","last_active":"2023-11-16T16:47:53.615707+0000","last_peered":"2023-11-16T16:47:53.615707+0000","last_clean":"2023-11-16T16:47:53.615707+0000","last_became_active":"2023-11-16T16:45:27.937451+0000","last_became_peered":"2023-11-16T16:45:27.937451+0000","last_unstale":"2023-11-16T16:47:53.615707+0000","last_undegraded":"2023-11-16T16:47:53.615707+0000","last_fullsized":"2023-11-16T16:47:53.615707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:42:13.306703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:42:13.306703+0000","last_clean_scrub_stamp":"2023-11-16T16:42:13.306703+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:36:26.247109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343317+0000","last_change":"2023-11-16T16:46:41.203736+0000","last_active":"2023-11-16T16:47:53.343317+0000","last_peered":"2023-11-16T16:47:53.343317+0000","last_clean":"2023-11-16T16:47:53.343317+0000","last_became_active":"2023-11-16T16:46:41.203282+0000","last_became_peered":"2023-11-16T16:46:41.203282+0000","last_unstale":"2023-11-16T16:47:53.343317+0000","last_undegraded":"2023-11-16T16:47:53.343317+0000","last_fullsized":"2023-11-16T16:47:53.343317+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:25:51.413002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.345806+0000","last_change":"2023-11-16T16:46:37.157408+0000","last_active":"2023-11-16T16:47:53.345806+0000","last_peered":"2023-11-16T16:47:53.345806+0000","last_clean":"2023-11-16T16:47:53.345806+0000","last_became_active":"2023-11-16T16:46:37.156972+0000","last_became_peered":"2023-11-16T16:46:37.156972+0000","last_unstale":"2023-11-16T16:47:53.345806+0000","last_undegraded":"2023-11-16T16:47:53.345806+0000","last_fullsized":"2023-11-16T16:47:53.345806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:17:05.643572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615870+0000","last_change":"2023-11-16T16:46:36.594284+0000","last_active":"2023-11-16T16:47:53.615870+0000","last_peered":"2023-11-16T16:47:53.615870+0000","last_clean":"2023-11-16T16:47:53.615870+0000","last_became_active":"2023-11-16T16:46:36.593964+0000","last_became_peered":"2023-11-16T16:46:36.593964+0000","last_unstale":"2023-11-16T16:47:53.615870+0000","last_undegraded":"2023-11-16T16:47:53.615870+0000","last_fullsized":"2023-11-16T16:47:53.615870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:08:17.083883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616510+0000","last_change":"2023-11-16T16:46:41.173944+0000","last_active":"2023-11-16T16:47:53.616510+0000","last_peered":"2023-11-16T16:47:53.616510+0000","last_clean":"2023-11-16T16:47:53.616510+0000","last_became_active":"2023-11-16T16:46:41.173391+0000","last_became_peered":"2023-11-16T16:46:41.173391+0000","last_unstale":"2023-11-16T16:47:53.616510+0000","last_undegraded":"2023-11-16T16:47:53.616510+0000","last_fullsized":"2023-11-16T16:47:53.616510+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:33:33.745909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340049+0000","last_change":"2023-11-16T16:46:36.596360+0000","last_active":"2023-11-16T16:47:53.340049+0000","last_peered":"2023-11-16T16:47:53.340049+0000","last_clean":"2023-11-16T16:47:53.340049+0000","last_became_active":"2023-11-16T16:46:36.596042+0000","last_became_peered":"2023-11-16T16:46:36.596042+0000","last_unstale":"2023-11-16T16:47:53.340049+0000","last_undegraded":"2023-11-16T16:47:53.340049+0000","last_fullsized":"2023-11-16T16:47:53.340049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:25:42.128187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.345267+0000","last_change":"2023-11-16T16:46:37.160277+0000","last_active":"2023-11-16T16:47:53.345267+0000","last_peered":"2023-11-16T16:47:53.345267+0000","last_clean":"2023-11-16T16:47:53.345267+0000","last_became_active":"2023-11-16T16:46:37.159047+0000","last_became_peered":"2023-11-16T16:46:37.159047+0000","last_unstale":"2023-11-16T16:47:53.345267+0000","last_undegraded":"2023-11-16T16:47:53.345267+0000","last_fullsized":"2023-11-16T16:47:53.345267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:51:56.521259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343322+0000","last_change":"2023-11-16T16:46:41.202909+0000","last_active":"2023-11-16T16:47:53.343322+0000","last_peered":"2023-11-16T16:47:53.343322+0000","last_clean":"2023-11-16T16:47:53.343322+0000","last_became_active":"2023-11-16T16:46:41.201969+0000","last_became_peered":"2023-11-16T16:46:41.201969+0000","last_unstale":"2023-11-16T16:47:53.343322+0000","last_undegraded":"2023-11-16T16:47:53.343322+0000","last_fullsized":"2023-11-16T16:47:53.343322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:27:24.361657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'5","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.347878+0000","last_change":"2023-11-16T16:46:36.599389+0000","last_active":"2023-11-16T16:47:53.347878+0000","last_peered":"2023-11-16T16:47:53.347878+0000","last_clean":"2023-11-16T16:47:53.347878+0000","last_became_active":"2023-11-16T16:46:36.599020+0000","last_became_peered":"2023-11-16T16:46:36.599020+0000","last_unstale":"2023-11-16T16:47:53.347878+0000","last_undegraded":"2023-11-16T16:47:53.347878+0000","last_fullsized":"2023-11-16T16:47:53.347878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:22:15.097589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342429+0000","last_change":"2023-11-16T16:46:37.145412+0000","last_active":"2023-11-16T16:47:53.342429+0000","last_peered":"2023-11-16T16:47:53.342429+0000","last_clean":"2023-11-16T16:47:53.342429+0000","last_became_active":"2023-11-16T16:46:37.141380+0000","last_became_peered":"2023-11-16T16:46:37.141380+0000","last_unstale":"2023-11-16T16:47:53.342429+0000","last_undegraded":"2023-11-16T16:47:53.342429+0000","last_fullsized":"2023-11-16T16:47:53.342429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:05:27.521717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615980+0000","last_change":"2023-11-16T16:46:41.163578+0000","last_active":"2023-11-16T16:47:53.615980+0000","last_peered":"2023-11-16T16:47:53.615980+0000","last_clean":"2023-11-16T16:47:53.615980+0000","last_became_active":"2023-11-16T16:46:41.163153+0000","last_became_peered":"2023-11-16T16:46:41.163153+0000","last_unstale":"2023-11-16T16:47:53.615980+0000","last_undegraded":"2023-11-16T16:47:53.615980+0000","last_fullsized":"2023-11-16T16:47:53.615980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:44:16.408506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340663+0000","last_change":"2023-11-16T16:46:36.606973+0000","last_active":"2023-11-16T16:47:53.340663+0000","last_peered":"2023-11-16T16:47:53.340663+0000","last_clean":"2023-11-16T16:47:53.340663+0000","last_became_active":"2023-11-16T16:46:36.606744+0000","last_became_peered":"2023-11-16T16:46:36.606744+0000","last_unstale":"2023-11-16T16:47:53.340663+0000","last_undegraded":"2023-11-16T16:47:53.340663+0000","last_fullsized":"2023-11-16T16:47:53.340663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:59:58.817297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.345611+0000","last_change":"2023-11-16T16:47:53.345611+0000","last_active":"2023-11-16T16:47:53.345611+0000","last_peered":"2023-11-16T16:47:53.345611+0000","last_clean":"2023-11-16T16:47:53.345611+0000","last_became_active":"2023-11-16T16:47:53.345139+0000","last_became_peered":"2023-11-16T16:47:53.345139+0000","last_unstale":"2023-11-16T16:47:53.345611+0000","last_undegraded":"2023-11-16T16:47:53.345611+0000","last_fullsized":"2023-11-16T16:47:53.345611+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:14:14.759670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340481+0000","last_change":"2023-11-16T16:46:41.204645+0000","last_active":"2023-11-16T16:47:53.340481+0000","last_peered":"2023-11-16T16:47:53.340481+0000","last_clean":"2023-11-16T16:47:53.340481+0000","last_became_active":"2023-11-16T16:46:41.204167+0000","last_became_peered":"2023-11-16T16:46:41.204167+0000","last_unstale":"2023-11-16T16:47:53.340481+0000","last_undegraded":"2023-11-16T16:47:53.340481+0000","last_fullsized":"2023-11-16T16:47:53.340481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:44:18.216270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615303+0000","last_change":"2023-11-16T16:46:52.334958+0000","last_active":"2023-11-16T16:47:53.615303+0000","last_peered":"2023-11-16T16:47:53.615303+0000","last_clean":"2023-11-16T16:47:53.615303+0000","last_became_active":"2023-11-16T16:46:52.334607+0000","last_became_peered":"2023-11-16T16:46:52.334607+0000","last_unstale":"2023-11-16T16:47:53.615303+0000","last_undegraded":"2023-11-16T16:47:53.615303+0000","last_fullsized":"2023-11-16T16:47:53.615303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:28:02.214272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339379+0000","last_change":"2023-11-16T16:46:37.153357+0000","last_active":"2023-11-16T16:47:53.339379+0000","last_peered":"2023-11-16T16:47:53.339379+0000","last_clean":"2023-11-16T16:47:53.339379+0000","last_became_active":"2023-11-16T16:46:37.153064+0000","last_became_peered":"2023-11-16T16:46:37.153064+0000","last_unstale":"2023-11-16T16:47:53.339379+0000","last_undegraded":"2023-11-16T16:47:53.339379+0000","last_fullsized":"2023-11-16T16:47:53.339379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:43:35.983547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.348077+0000","last_change":"2023-11-16T16:46:41.183938+0000","last_active":"2023-11-16T16:47:53.348077+0000","last_peered":"2023-11-16T16:47:53.348077+0000","last_clean":"2023-11-16T16:47:53.348077+0000","last_became_active":"2023-11-16T16:46:41.181850+0000","last_became_peered":"2023-11-16T16:46:41.181850+0000","last_unstale":"2023-11-16T16:47:53.348077+0000","last_undegraded":"2023-11-16T16:47:53.348077+0000","last_fullsized":"2023-11-16T16:47:53.348077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:53:49.918986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342820+0000","last_change":"2023-11-16T16:46:36.603257+0000","last_active":"2023-11-16T16:47:53.342820+0000","last_peered":"2023-11-16T16:47:53.342820+0000","last_clean":"2023-11-16T16:47:53.342820+0000","last_became_active":"2023-11-16T16:46:36.602852+0000","last_became_peered":"2023-11-16T16:46:36.602852+0000","last_unstale":"2023-11-16T16:47:53.342820+0000","last_undegraded":"2023-11-16T16:47:53.342820+0000","last_fullsized":"2023-11-16T16:47:53.342820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:03:55.431436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615710+0000","last_change":"2023-11-16T16:46:37.162689+0000","last_active":"2023-11-16T16:47:53.615710+0000","last_peered":"2023-11-16T16:47:53.615710+0000","last_clean":"2023-11-16T16:47:53.615710+0000","last_became_active":"2023-11-16T16:46:37.162356+0000","last_became_peered":"2023-11-16T16:46:37.162356+0000","last_unstale":"2023-11-16T16:47:53.615710+0000","last_undegraded":"2023-11-16T16:47:53.615710+0000","last_fullsized":"2023-11-16T16:47:53.615710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:38:37.707256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340208+0000","last_change":"2023-11-16T16:46:41.182286+0000","last_active":"2023-11-16T16:47:53.340208+0000","last_peered":"2023-11-16T16:47:53.340208+0000","last_clean":"2023-11-16T16:47:53.340208+0000","last_became_active":"2023-11-16T16:46:41.181820+0000","last_became_peered":"2023-11-16T16:46:41.181820+0000","last_unstale":"2023-11-16T16:47:53.340208+0000","last_undegraded":"2023-11-16T16:47:53.340208+0000","last_fullsized":"2023-11-16T16:47:53.340208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:17:47.324972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615819+0000","last_change":"2023-11-16T16:46:36.598952+0000","last_active":"2023-11-16T16:47:53.615819+0000","last_peered":"2023-11-16T16:47:53.615819+0000","last_clean":"2023-11-16T16:47:53.615819+0000","last_became_active":"2023-11-16T16:46:36.598617+0000","last_became_peered":"2023-11-16T16:46:36.598617+0000","last_unstale":"2023-11-16T16:47:53.615819+0000","last_undegraded":"2023-11-16T16:47:53.615819+0000","last_fullsized":"2023-11-16T16:47:53.615819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:23:33.595837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342335+0000","last_change":"2023-11-16T16:46:37.154350+0000","last_active":"2023-11-16T16:47:53.342335+0000","last_peered":"2023-11-16T16:47:53.342335+0000","last_clean":"2023-11-16T16:47:53.342335+0000","last_became_active":"2023-11-16T16:46:37.154045+0000","last_became_peered":"2023-11-16T16:46:37.154045+0000","last_unstale":"2023-11-16T16:47:53.342335+0000","last_undegraded":"2023-11-16T16:47:53.342335+0000","last_fullsized":"2023-11-16T16:47:53.342335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:42:00.370393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.340495+0000","last_change":"2023-11-16T16:46:41.202563+0000","last_active":"2023-11-16T16:47:53.340495+0000","last_peered":"2023-11-16T16:47:53.340495+0000","last_clean":"2023-11-16T16:47:53.340495+0000","last_became_active":"2023-11-16T16:46:41.202125+0000","last_became_peered":"2023-11-16T16:46:41.202125+0000","last_unstale":"2023-11-16T16:47:53.340495+0000","last_undegraded":"2023-11-16T16:47:53.340495+0000","last_fullsized":"2023-11-16T16:47:53.340495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:29:21.341816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339619+0000","last_change":"2023-11-16T16:46:52.341548+0000","last_active":"2023-11-16T16:47:53.339619+0000","last_peered":"2023-11-16T16:47:53.339619+0000","last_clean":"2023-11-16T16:47:53.339619+0000","last_became_active":"2023-11-16T16:46:52.337771+0000","last_became_peered":"2023-11-16T16:46:52.337771+0000","last_unstale":"2023-11-16T16:47:53.339619+0000","last_undegraded":"2023-11-16T16:47:53.339619+0000","last_fullsized":"2023-11-16T16:47:53.339619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:29:45.214101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616366+0000","last_change":"2023-11-16T16:46:37.159095+0000","last_active":"2023-11-16T16:47:53.616366+0000","last_peered":"2023-11-16T16:47:53.616366+0000","last_clean":"2023-11-16T16:47:53.616366+0000","last_became_active":"2023-11-16T16:46:37.158668+0000","last_became_peered":"2023-11-16T16:46:37.158668+0000","last_unstale":"2023-11-16T16:47:53.616366+0000","last_undegraded":"2023-11-16T16:47:53.616366+0000","last_fullsized":"2023-11-16T16:47:53.616366+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:43:01.651855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615482+0000","last_change":"2023-11-16T16:46:41.165572+0000","last_active":"2023-11-16T16:47:53.615482+0000","last_peered":"2023-11-16T16:47:53.615482+0000","last_clean":"2023-11-16T16:47:53.615482+0000","last_became_active":"2023-11-16T16:46:41.165174+0000","last_became_peered":"2023-11-16T16:46:41.165174+0000","last_unstale":"2023-11-16T16:47:53.615482+0000","last_undegraded":"2023-11-16T16:47:53.615482+0000","last_fullsized":"2023-11-16T16:47:53.615482+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:26:17.992824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617090+0000","last_change":"2023-11-16T16:46:36.593871+0000","last_active":"2023-11-16T16:47:53.617090+0000","last_peered":"2023-11-16T16:47:53.617090+0000","last_clean":"2023-11-16T16:47:53.617090+0000","last_became_active":"2023-11-16T16:46:36.593579+0000","last_became_peered":"2023-11-16T16:46:36.593579+0000","last_unstale":"2023-11-16T16:47:53.617090+0000","last_undegraded":"2023-11-16T16:47:53.617090+0000","last_fullsized":"2023-11-16T16:47:53.617090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:58:29.714501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617143+0000","last_change":"2023-11-16T16:46:37.140823+0000","last_active":"2023-11-16T16:47:53.617143+0000","last_peered":"2023-11-16T16:47:53.617143+0000","last_clean":"2023-11-16T16:47:53.617143+0000","last_became_active":"2023-11-16T16:46:37.140505+0000","last_became_peered":"2023-11-16T16:46:37.140505+0000","last_unstale":"2023-11-16T16:47:53.617143+0000","last_undegraded":"2023-11-16T16:47:53.617143+0000","last_fullsized":"2023-11-16T16:47:53.617143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:25:37.585706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.338806+0000","last_change":"2023-11-16T16:46:41.182041+0000","last_active":"2023-11-16T16:47:53.338806+0000","last_peered":"2023-11-16T16:47:53.338806+0000","last_clean":"2023-11-16T16:47:53.338806+0000","last_became_active":"2023-11-16T16:46:41.181420+0000","last_became_peered":"2023-11-16T16:46:41.181420+0000","last_unstale":"2023-11-16T16:47:53.338806+0000","last_undegraded":"2023-11-16T16:47:53.338806+0000","last_fullsized":"2023-11-16T16:47:53.338806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:41:11.223688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343181+0000","last_change":"2023-11-16T16:46:36.602184+0000","last_active":"2023-11-16T16:47:53.343181+0000","last_peered":"2023-11-16T16:47:53.343181+0000","last_clean":"2023-11-16T16:47:53.343181+0000","last_became_active":"2023-11-16T16:46:36.601893+0000","last_became_peered":"2023-11-16T16:46:36.601893+0000","last_unstale":"2023-11-16T16:47:53.343181+0000","last_undegraded":"2023-11-16T16:47:53.343181+0000","last_fullsized":"2023-11-16T16:47:53.343181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:11:04.147511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616798+0000","last_change":"2023-11-16T16:46:37.153649+0000","last_active":"2023-11-16T16:47:53.616798+0000","last_peered":"2023-11-16T16:47:53.616798+0000","last_clean":"2023-11-16T16:47:53.616798+0000","last_became_active":"2023-11-16T16:46:37.152420+0000","last_became_peered":"2023-11-16T16:46:37.152420+0000","last_unstale":"2023-11-16T16:47:53.616798+0000","last_undegraded":"2023-11-16T16:47:53.616798+0000","last_fullsized":"2023-11-16T16:47:53.616798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:47:46.597259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342448+0000","last_change":"2023-11-16T16:46:41.168363+0000","last_active":"2023-11-16T16:47:53.342448+0000","last_peered":"2023-11-16T16:47:53.342448+0000","last_clean":"2023-11-16T16:47:53.342448+0000","last_became_active":"2023-11-16T16:46:41.167615+0000","last_became_peered":"2023-11-16T16:46:41.167615+0000","last_unstale":"2023-11-16T16:47:53.342448+0000","last_undegraded":"2023-11-16T16:47:53.342448+0000","last_fullsized":"2023-11-16T16:47:53.342448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:40.138884+0000","last_clean_scrub_stamp":"2023-11-16T16:46:40.138884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:48:16.995034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343357+0000","last_change":"2023-11-16T16:46:36.611353+0000","last_active":"2023-11-16T16:47:53.343357+0000","last_peered":"2023-11-16T16:47:53.343357+0000","last_clean":"2023-11-16T16:47:53.343357+0000","last_became_active":"2023-11-16T16:46:36.610886+0000","last_became_peered":"2023-11-16T16:46:36.610886+0000","last_unstale":"2023-11-16T16:47:53.343357+0000","last_undegraded":"2023-11-16T16:47:53.343357+0000","last_fullsized":"2023-11-16T16:47:53.343357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:35.567945+0000","last_clean_scrub_stamp":"2023-11-16T16:46:35.567945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:24:34.228785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.617020+0000","last_change":"2023-11-16T16:46:37.146227+0000","last_active":"2023-11-16T16:47:53.617020+0000","last_peered":"2023-11-16T16:47:53.617020+0000","last_clean":"2023-11-16T16:47:53.617020+0000","last_became_active":"2023-11-16T16:46:37.143239+0000","last_became_peered":"2023-11-16T16:46:37.143239+0000","last_unstale":"2023-11-16T16:47:53.617020+0000","last_undegraded":"2023-11-16T16:47:53.617020+0000","last_fullsized":"2023-11-16T16:47:53.617020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:44:02.854936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615999+0000","last_change":"2023-11-16T16:46:37.153845+0000","last_active":"2023-11-16T16:47:53.615999+0000","last_peered":"2023-11-16T16:47:53.615999+0000","last_clean":"2023-11-16T16:47:53.615999+0000","last_became_active":"2023-11-16T16:46:37.153505+0000","last_became_peered":"2023-11-16T16:46:37.153505+0000","last_unstale":"2023-11-16T16:47:53.615999+0000","last_undegraded":"2023-11-16T16:47:53.615999+0000","last_fullsized":"2023-11-16T16:47:53.615999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:24:41.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616723+0000","last_change":"2023-11-16T16:46:37.148451+0000","last_active":"2023-11-16T16:47:53.616723+0000","last_peered":"2023-11-16T16:47:53.616723+0000","last_clean":"2023-11-16T16:47:53.616723+0000","last_became_active":"2023-11-16T16:46:37.148142+0000","last_became_peered":"2023-11-16T16:46:37.148142+0000","last_unstale":"2023-11-16T16:47:53.616723+0000","last_undegraded":"2023-11-16T16:47:53.616723+0000","last_fullsized":"2023-11-16T16:47:53.616723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:36:55.683708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616000+0000","last_change":"2023-11-16T16:46:37.150360+0000","last_active":"2023-11-16T16:47:53.616000+0000","last_peered":"2023-11-16T16:47:53.616000+0000","last_clean":"2023-11-16T16:47:53.616000+0000","last_became_active":"2023-11-16T16:46:37.150045+0000","last_became_peered":"2023-11-16T16:46:37.150045+0000","last_unstale":"2023-11-16T16:47:53.616000+0000","last_undegraded":"2023-11-16T16:47:53.616000+0000","last_fullsized":"2023-11-16T16:47:53.616000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:17:58.442185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.339093+0000","last_change":"2023-11-16T16:46:37.161736+0000","last_active":"2023-11-16T16:47:53.339093+0000","last_peered":"2023-11-16T16:47:53.339093+0000","last_clean":"2023-11-16T16:47:53.339093+0000","last_became_active":"2023-11-16T16:46:37.158273+0000","last_became_peered":"2023-11-16T16:46:37.158273+0000","last_unstale":"2023-11-16T16:47:53.339093+0000","last_undegraded":"2023-11-16T16:47:53.339093+0000","last_fullsized":"2023-11-16T16:47:53.339093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:30:05.153453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.349210+0000","last_change":"2023-11-16T16:47:53.349210+0000","last_active":"2023-11-16T16:47:53.349210+0000","last_peered":"2023-11-16T16:47:53.349210+0000","last_clean":"2023-11-16T16:47:53.349210+0000","last_became_active":"2023-11-16T16:47:53.348819+0000","last_became_peered":"2023-11-16T16:47:53.348819+0000","last_unstale":"2023-11-16T16:47:53.349210+0000","last_undegraded":"2023-11-16T16:47:53.349210+0000","last_fullsized":"2023-11-16T16:47:53.349210+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:24:00.116336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616870+0000","last_change":"2023-11-16T16:46:37.154475+0000","last_active":"2023-11-16T16:47:53.616870+0000","last_peered":"2023-11-16T16:47:53.616870+0000","last_clean":"2023-11-16T16:47:53.616870+0000","last_became_active":"2023-11-16T16:46:37.154161+0000","last_became_peered":"2023-11-16T16:46:37.154161+0000","last_unstale":"2023-11-16T16:47:53.616870+0000","last_undegraded":"2023-11-16T16:47:53.616870+0000","last_fullsized":"2023-11-16T16:47:53.616870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:19:04.354750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616438+0000","last_change":"2023-11-16T16:46:37.140293+0000","last_active":"2023-11-16T16:47:53.616438+0000","last_peered":"2023-11-16T16:47:53.616438+0000","last_clean":"2023-11-16T16:47:53.616438+0000","last_became_active":"2023-11-16T16:46:37.139966+0000","last_became_peered":"2023-11-16T16:46:37.139966+0000","last_unstale":"2023-11-16T16:47:53.616438+0000","last_undegraded":"2023-11-16T16:47:53.616438+0000","last_fullsized":"2023-11-16T16:47:53.616438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:05:23.489857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342605+0000","last_change":"2023-11-16T16:46:37.134807+0000","last_active":"2023-11-16T16:47:53.342605+0000","last_peered":"2023-11-16T16:47:53.342605+0000","last_clean":"2023-11-16T16:47:53.342605+0000","last_became_active":"2023-11-16T16:46:37.131833+0000","last_became_peered":"2023-11-16T16:46:37.131833+0000","last_unstale":"2023-11-16T16:47:53.342605+0000","last_undegraded":"2023-11-16T16:47:53.342605+0000","last_fullsized":"2023-11-16T16:47:53.342605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:22:51.275681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.342895+0000","last_change":"2023-11-16T16:46:37.152424+0000","last_active":"2023-11-16T16:47:53.342895+0000","last_peered":"2023-11-16T16:47:53.342895+0000","last_clean":"2023-11-16T16:47:53.342895+0000","last_became_active":"2023-11-16T16:46:37.152082+0000","last_became_peered":"2023-11-16T16:46:37.152082+0000","last_unstale":"2023-11-16T16:47:53.342895+0000","last_undegraded":"2023-11-16T16:47:53.342895+0000","last_fullsized":"2023-11-16T16:47:53.342895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:48:35.526315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.343126+0000","last_change":"2023-11-16T16:46:37.140267+0000","last_active":"2023-11-16T16:47:53.343126+0000","last_peered":"2023-11-16T16:47:53.343126+0000","last_clean":"2023-11-16T16:47:53.343126+0000","last_became_active":"2023-11-16T16:46:37.139689+0000","last_became_peered":"2023-11-16T16:46:37.139689+0000","last_unstale":"2023-11-16T16:47:53.343126+0000","last_undegraded":"2023-11-16T16:47:53.343126+0000","last_fullsized":"2023-11-16T16:47:53.343126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:22:15.270033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.615789+0000","last_change":"2023-11-16T16:46:37.151911+0000","last_active":"2023-11-16T16:47:53.615789+0000","last_peered":"2023-11-16T16:47:53.615789+0000","last_clean":"2023-11-16T16:47:53.615789+0000","last_became_active":"2023-11-16T16:46:37.151614+0000","last_became_peered":"2023-11-16T16:46:37.151614+0000","last_unstale":"2023-11-16T16:47:53.615789+0000","last_undegraded":"2023-11-16T16:47:53.615789+0000","last_fullsized":"2023-11-16T16:47:53.615789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:48:46.891259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.346604+0000","last_change":"2023-11-16T16:46:37.153265+0000","last_active":"2023-11-16T16:47:53.346604+0000","last_peered":"2023-11-16T16:47:53.346604+0000","last_clean":"2023-11-16T16:47:53.346604+0000","last_became_active":"2023-11-16T16:46:37.152933+0000","last_became_peered":"2023-11-16T16:46:37.152933+0000","last_unstale":"2023-11-16T16:47:53.346604+0000","last_undegraded":"2023-11-16T16:47:53.346604+0000","last_fullsized":"2023-11-16T16:47:53.346604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:09:28.674677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-16T16:47:53.616247+0000","last_change":"2023-11-16T16:46:37.142831+0000","last_active":"2023-11-16T16:47:53.616247+0000","last_peered":"2023-11-16T16:47:53.616247+0000","last_clean":"2023-11-16T16:47:53.616247+0000","last_became_active":"2023-11-16T16:46:37.142480+0000","last_became_peered":"2023-11-16T16:46:37.142480+0000","last_unstale":"2023-11-16T16:47:53.616247+0000","last_undegraded":"2023-11-16T16:47:53.616247+0000","last_fullsized":"2023-11-16T16:47:53.616247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:46:36.574615+0000","last_clean_scrub_stamp":"2023-11-16T16:46:36.574615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:10:16.785368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64741,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":327,"num_read_kb":231,"num_write":154,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416413,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6272,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738880,"statfs":{"total":95995035648,"available":95988613120,"internally_reserved":0,"allocated":1040384,"data_stored":513127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.748,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47399999999999998,"15min":0.46999999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68999999999999995,"5min":0.879,"15min":0.879},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.32400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86699999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.68500000000000005}]},{"osd":5,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81999999999999995,"5min":1.756,"15min":1.756},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.433,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84699999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71699999999999997,"15min":0.71999999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97099999999999997,"5min":1.228,"15min":1.228},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.52900000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":1.845,"15min":1.845},"last":0.81399999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.58999999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.99299999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.871},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75,"15min":0.746},"min":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93000000000000005,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.78500000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.65300000000000002}]},{"osd":9,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.80400000000000005,"15min":0.80800000000000005},"min":{"1min":0.56699999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91700000000000004,"5min":1.819,"15min":1.819},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78400000000000003,"15min":0.78000000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0620000000000001,"5min":1.28,"15min":1.28},"last":0.90300000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.78700000000000003,"15min":0.78000000000000003},"min":{"1min":0.61399999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96499999999999997,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93700000000000006,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.88600000000000001}]}]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7204,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93737948,"statfs":{"total":95995035648,"available":95987658752,"internally_reserved":0,"allocated":1732608,"data_stored":1165670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.61399999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.52300000000000002,"15min":0.53300000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64400000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.378}]},{"osd":2,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69699999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80200000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.82299999999999995,"15min":0.86499999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.70599999999999996,"5min":4.1299999999999999,"15min":4.1299999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.85899999999999999,"15min":0.89500000000000002},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76600000000000001,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.623,"15min":0.63200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73099999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.503},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71899999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.40600000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83499999999999996,"5min":0.873,"15min":0.873},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.85299999999999998,"5min":1.048,"15min":1.048},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79200000000000004,"15min":0.80200000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":1.054,"15min":1.054},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.56599999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94699999999999995,"5min":1.357,"15min":1.357},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.79600000000000004,"15min":0.81000000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90000000000000002,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.56699999999999995}]},{"osd":9,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.84999999999999998,"15min":0.86399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92000000000000004,"5min":1.387,"15min":1.387},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.89400000000000002,"5min":1.226,"15min":1.226},"last":0.60299999999999998}]},{"osd":11,"last update":"Thu Nov 16 16:47:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78400000000000003,"15min":0.79200000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.88500000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.93799999999999994,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.70799999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808872,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6420,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738732,"statfs":{"total":95995035648,"available":95988461568,"internally_reserved":0,"allocated":1060864,"data_stored":517459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.55100000000000005,"15min":0.56399999999999995},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72599999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.749,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.36199999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.54900000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.61499999999999999}]},{"osd":2,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72999999999999998,"15min":0.73899999999999999},"min":{"1min":0.47899999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96999999999999997,"5min":1.048,"15min":1.048},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.85899999999999999,"15min":0.89100000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98699999999999999,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.56200000000000006}]},{"osd":3,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67900000000000005,"15min":0.68999999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78000000000000003,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.79100000000000004,"15min":0.80900000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.056,"5min":1.24,"15min":1.24},"last":0.628},{"interface":"front","average":{"1min":0.747,"5min":0.82399999999999995,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1120000000000001,"5min":1.216,"15min":1.216},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.82199999999999995,"15min":0.84399999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95899999999999996,"5min":1.304,"15min":1.304},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.86799999999999999,"15min":0.88600000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.145,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Nov 16 16:46:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.92500000000000004},"min":{"1min":0.61899999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.111,"5min":1.282,"15min":1.282},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.98899999999999999,"15min":1.0229999999999999},"min":{"1min":0.57799999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0860000000000001,"5min":4.5650000000000004,"15min":4.5650000000000004},"last":0.67500000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1,"15min":1.0389999999999999},"min":{"1min":0.55300000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.163,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.93799999999999994,"15min":0.96699999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.077,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.71599999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.002,"15min":1.022},"min":{"1min":0.69699999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.135,"5min":4.4870000000000001,"15min":4.4870000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.86699999999999999,"15min":0.872},"min":{"1min":0.58799999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2130000000000001,"5min":1.343,"15min":1.343},"last":0.72799999999999998}]},{"osd":11,"last update":"Thu Nov 16 16:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.81100000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6540,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738612,"statfs":{"total":95995035648,"available":95988338688,"internally_reserved":0,"allocated":1052672,"data_stored":512952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.371,"5min":0.309,"15min":0.309},"max":{"1min":0.82199999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58399999999999996,"15min":0.59699999999999998},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77700000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.624}]},{"osd":1,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.47599999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.878}]},{"osd":2,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.46000000000000002,"15min":0.46500000000000002},"min":{"1min":0.32200000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72999999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46400000000000002,"15min":0.47399999999999998},"min":{"1min":0.34999999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.64300000000000002,"15min":0.67900000000000005},"min":{"1min":0.432,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78500000000000003,"5min":1.536,"15min":1.536},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.66600000000000004,"15min":0.69599999999999995},"min":{"1min":0.38800000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81399999999999995,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.90100000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.47899999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89600000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64200000000000002,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.56599999999999995},"min":{"1min":0.441,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76200000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58399999999999996,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.75800000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48599999999999999}]},{"osd":6,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.72499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94099999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.56000000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.87}]},{"osd":7,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.51500000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88200000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71899999999999997,"15min":0.73299999999999998},"min":{"1min":0.51400000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92800000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.68000000000000005}]},{"osd":9,"last update":"Thu Nov 16 16:47:03 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.91300000000000003}]},{"osd":10,"last update":"Thu Nov 16 16:47:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.74199999999999999,"15min":0.73699999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.745,"15min":0.72299999999999998},"min":{"1min":0.59099999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.89200000000000002}]},{"osd":11,"last update":"Thu Nov 16 16:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.73899999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7184,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93737968,"statfs":{"total":95995035648,"available":95987679232,"internally_reserved":0,"allocated":1646592,"data_stored":1108764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41299999999999998,"15min":0.40500000000000003},"min":{"1min":0.28199999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60899999999999999,"5min":0.624,"15min":0.624},"last":0.44},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.42599999999999999,"15min":0.41499999999999998},"min":{"1min":0.34000000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.60299999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58799999999999997,"15min":0.60799999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73899999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76800000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.85999999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84599999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66600000000000004,"15min":0.66400000000000003},"min":{"1min":0.48299999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.307,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72899999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68999999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.754,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.752},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.46400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71199999999999997,"15min":0.70399999999999996},"min":{"1min":0.55700000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84799999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.80100000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.624}]},{"osd":8,"last update":"Thu Nov 16 16:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73399999999999999,"15min":0.72199999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91400000000000003,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.872},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.754,"15min":0.73599999999999999},"min":{"1min":0.48599999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96599999999999997,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.73699999999999999}]},{"osd":9,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90600000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.60899999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93600000000000005,"5min":1.054,"15min":1.054},"last":0.80900000000000005}]},{"osd":10,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70999999999999996,"15min":0.70599999999999996},"min":{"1min":0.53600000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94899999999999995,"5min":1.002,"15min":1.002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55200000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.98299999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.78300000000000003}]},{"osd":11,"last update":"Thu Nov 16 16:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.83399999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":1036288,"data_stored":508539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.36799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81399999999999995,"5min":1.139,"15min":1.139},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.434,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.873,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.65000000000000002,"15min":0.624},"min":{"1min":0.46899999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Nov 16 16:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90300000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67300000000000004,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.69199999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.626,"5min":0.624,"15min":0.628},"min":{"1min":0.34999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.51000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86199999999999999,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72799999999999998,"15min":0.747},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98899999999999999,"5min":1.343,"15min":1.343},"last":0.73899999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.087,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.749,"15min":0.73999999999999999},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":0.98299999999999998,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.55400000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.0569999999999999,"5min":1.109,"15min":1.109},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0109999999999999,"5min":1.121,"15min":1.121},"last":0.89700000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78500000000000003,"15min":0.79200000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.151,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.60699999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84499999999999997}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.86699999999999999,"15min":0.877},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.089,"5min":1.165,"15min":1.165},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84299999999999997,"15min":0.84699999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.097,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.871}]},{"osd":10,"last update":"Thu Nov 16 16:47:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.85499999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.873,"15min":0.88500000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.133,"5min":1.2,"15min":1.2},"last":0.995}]},{"osd":11,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.92500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7612,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737540,"statfs":{"total":95995035648,"available":95987240960,"internally_reserved":0,"allocated":1101824,"data_stored":569286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52200000000000002,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59899999999999998,"5min":1.431,"15min":1.431},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57299999999999995,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76500000000000001,"5min":4.8170000000000002,"15min":4.8170000000000002},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Nov 16 16:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.436,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.58399999999999996,"15min":0.55600000000000005},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83699999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67700000000000005}]},{"osd":3,"last update":"Thu Nov 16 16:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.55100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91300000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60199999999999998,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.93200000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.40899999999999997,"15min":0.39200000000000002},"min":{"1min":0.32200000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.432,"15min":0.40300000000000002},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.58999999999999997,"15min":0.53800000000000003},"min":{"1min":0.44400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.82099999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.622},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61699999999999999,"15min":0.55700000000000005},"min":{"1min":0.52900000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84999999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63600000000000001,"15min":0.60499999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93700000000000006,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91500000000000004,"5min":0.995,"15min":0.995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66100000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.72399999999999998},"min":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.90000000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71999999999999997,"15min":0.70499999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96899999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60299999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94299999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.036,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.56499999999999995}]},{"osd":10,"last update":"Thu Nov 16 16:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.034,"5min":1.052,"15min":1.052},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77900000000000003,"15min":0.79000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88800000000000001,"5min":1.075,"15min":1.075},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Nov 16 16:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.79700000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7508,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737644,"statfs":{"total":95995035648,"available":95987347456,"internally_reserved":0,"allocated":1060864,"data_stored":509927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89100000000000001,"5min":1.01,"15min":1.01},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.61699999999999999,"15min":0.55900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Nov 16 16:47:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61499999999999999,"15min":0.55700000000000005},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.503},{"interface":"front","average":{"1min":0.624,"5min":0.60699999999999998,"15min":0.55900000000000005},"min":{"1min":0.432,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95099999999999996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.629}]},{"osd":3,"last update":"Thu Nov 16 16:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":1.0569999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:47:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64000000000000001,"15min":0.60799999999999998},"min":{"1min":0.44800000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.62,"15min":0.58399999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86299999999999999,"5min":0.871,"15min":0.871},"last":0.54500000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.65600000000000003,"15min":0.623},"min":{"1min":0.47599999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.52300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:46:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.90900000000000003,"5min":1.03,"15min":1.03},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Nov 16 16:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.83199999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.59699999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77900000000000003,"15min":0.78800000000000003},"min":{"1min":0.60399999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.79500000000000004}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.79000000000000004,"15min":0.80500000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81100000000000005,"15min":0.81299999999999994},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.121,"5min":1.145,"15min":1.145},"last":0.64500000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.042,"5min":1.075,"15min":1.075},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.876,"5min":0.84999999999999998,"15min":0.84599999999999997},"min":{"1min":0.60899999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.70199999999999996}]},{"osd":11,"last update":"Thu Nov 16 16:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.85799999999999998,"15min":0.84499999999999997},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.81699999999999995,"15min":0.80300000000000005},"min":{"1min":0.70699999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.755}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7700,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737452,"statfs":{"total":95995035648,"available":95987150848,"internally_reserved":0,"allocated":1126400,"data_stored":570588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.40200000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55700000000000005,"15min":0.53900000000000003},"min":{"1min":0.38200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68999999999999995,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Nov 16 16:47:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56100000000000005,"15min":0.53100000000000003},"min":{"1min":0.39700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80000000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57299999999999995,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81399999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.67100000000000004}]},{"osd":3,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.47699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.8399999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.66600000000000004,"15min":0.60999999999999999},"min":{"1min":0.46300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":4.8769999999999998,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":0.75900000000000001}]},{"osd":4,"last update":"Thu Nov 16 16:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.504}]},{"osd":5,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.48699999999999999,"15min":0.45500000000000002},"min":{"1min":0.441,"5min":0.314,"15min":0.314},"max":{"1min":0.86799999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.497,"15min":0.47099999999999997},"min":{"1min":0.42199999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73299999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.73399999999999999,"15min":0.65200000000000002},"min":{"1min":0.51800000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":7.2789999999999999,"5min":7.2789999999999999,"15min":7.2789999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.137,"5min":0.74299999999999999,"15min":0.66500000000000004},"min":{"1min":0.53800000000000003,"5min":0.437,"15min":0.437},"max":{"1min":4.9299999999999997,"5min":4.9299999999999997,"15min":4.9299999999999997},"last":0.90500000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70699999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.66500000000000004,"15min":0.65300000000000002},"min":{"1min":0.434,"5min":0.375,"15min":0.375},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.39900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71999999999999997}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.745,"15min":0.72999999999999998},"min":{"1min":0.63900000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.77500000000000002,"15min":0.76100000000000001},"min":{"1min":0.59099999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.86399999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:47:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.64200000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.749},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.72899999999999998,"15min":0.71499999999999997},"min":{"1min":0.70599999999999996,"5min":0.495,"15min":0.495},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.85399999999999998}]},{"osd":11,"last update":"Thu Nov 16 16:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.80700000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6792,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738360,"statfs":{"total":95995035648,"available":95988080640,"internally_reserved":0,"allocated":1048576,"data_stored":508733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.45500000000000002,"15min":0.39700000000000002},"min":{"1min":0.29299999999999998,"5min":0.246,"15min":0.246},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.47199999999999998},"min":{"1min":0.35199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.2110000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.435}]},{"osd":1,"last update":"Thu Nov 16 16:47:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.65000000000000002,"15min":0.60299999999999998},"min":{"1min":0.497,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80300000000000005,"5min":1.119,"15min":1.119},"last":0.32400000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:47:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.622},"min":{"1min":0.51000000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91500000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.70399999999999996,"15min":0.63200000000000001},"min":{"1min":0.51900000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Nov 16 16:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.57699999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.61199999999999999},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Nov 16 16:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67700000000000005,"15min":0.66100000000000003},"min":{"1min":0.56100000000000005,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.036,"5min":1.133,"15min":1.133},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.74099999999999999,"5min":1.149,"15min":1.149},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.498},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.36699999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.48399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.88900000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77100000000000002,"15min":0.76500000000000001},"min":{"1min":0.54600000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0609999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.61599999999999999}]},{"osd":8,"last update":"Thu Nov 16 16:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.753,"15min":0.753},"min":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1559999999999999,"5min":1.171,"15min":1.171},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76500000000000001,"15min":0.75800000000000001},"min":{"1min":0.60699999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.306}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.627},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81599999999999995,"15min":0.81899999999999995},"min":{"1min":0.64500000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.51100000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.81799999999999995,"15min":0.83199999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.58199999999999996}]},{"osd":11,"last update":"Thu Nov 16 16:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.63400000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6724,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738428,"statfs":{"total":95995035648,"available":95988150272,"internally_reserved":0,"allocated":1044480,"data_stored":513200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:46:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.52500000000000002,"15min":0.501},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.377,"15min":0.34799999999999998},"min":{"1min":0.33700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.35199999999999998},"min":{"1min":0.26500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.5,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.36799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.754,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.53900000000000003,"15min":0.51000000000000001},"min":{"1min":0.45200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72599999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.82399999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58399999999999996,"15min":0.53800000000000003},"min":{"1min":0.44400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75700000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82399999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68000000000000005}]},{"osd":5,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56200000000000006,"15min":0.53800000000000003},"min":{"1min":0.48599999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60799999999999998,"15min":0.56799999999999995},"min":{"1min":0.47499999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81699999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.89300000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.42699999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95299999999999996,"5min":1.097,"15min":1.097},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84699999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Nov 16 16:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82699999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.501,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83899999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.66000000000000003,"15min":0.63800000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.70999999999999996,"15min":0.67600000000000005},"min":{"1min":0.58599999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94399999999999995,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.94099999999999995}]},{"osd":9,"last update":"Thu Nov 16 16:47:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.80000000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.92900000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73099999999999998,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.878,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.878}]},{"osd":10,"last update":"Thu Nov 16 16:47:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.54700000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89700000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67000000000000004,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.76800000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.753}]},{"osd":11,"last update":"Thu Nov 16 16:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.96199999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7316,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737836,"statfs":{"total":95995035648,"available":95987544064,"internally_reserved":0,"allocated":1650688,"data_stored":1099914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76600000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.40300000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.55700000000000005,"15min":0.52000000000000002},"min":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77100000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.81699999999999995}]},{"osd":2,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.42299999999999999,"15min":0.39500000000000002},"min":{"1min":0.30599999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77700000000000002,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.44,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79400000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.436,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Nov 16 16:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.60799999999999998,"15min":0.55800000000000005},"min":{"1min":0.45900000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.0149999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.58599999999999997,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0229999999999999,"5min":1.046,"15min":1.046},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:47:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65700000000000003,"15min":0.63800000000000001},"min":{"1min":0.54500000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.88300000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.65700000000000003},"min":{"1min":0.57399999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.82799999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65900000000000003,"15min":0.64500000000000002},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.0069999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.63800000000000001,"15min":0.621},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95499999999999996,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.65100000000000002,"15min":0.63100000000000001},"min":{"1min":0.47299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.747},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.625,"15min":0.60499999999999998},"min":{"1min":0.52400000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90700000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.625}]},{"osd":9,"last update":"Thu Nov 16 16:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.75,"15min":0.746},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71999999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.877},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.754,"15min":0.748},"min":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0229999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.80500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:48:03.581 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-11-16T16:48:03.581 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-16T16:48:03.581 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-11-16T16:48:03.581 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph health --format=json 2023-11-16T16:48:04.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:03 smithi093 ceph-mon[97863]: pgmap v328: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-11-16T16:48:04.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:03 smithi093 ceph-mon[97863]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:48:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:03 smithi159 ceph-mon[131528]: pgmap v328: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-11-16T16:48:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:03 smithi159 ceph-mon[131528]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:48:04.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:03 smithi125 ceph-mon[131076]: pgmap v328: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-11-16T16:48:04.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:03 smithi125 ceph-mon[131076]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:48:04.430 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:48:05.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.965+0000 7f796bcc0700 1 -- 172.21.15.93:0/3008613118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7964075930 msgr2=0x7f7964110c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:05.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.965+0000 7f796bcc0700 1 --2- 172.21.15.93:0/3008613118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7964075930 0x7f7964110c70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7954009990 tx=0x7f7954009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:05.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.965+0000 7f796bcc0700 1 -- 172.21.15.93:0/3008613118 shutdown_connections 2023-11-16T16:48:05.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.965+0000 7f796bcc0700 1 --2- 172.21.15.93:0/3008613118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79641111b0 0x7f7964113590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:05.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.965+0000 7f796bcc0700 1 --2- 172.21.15.93:0/3008613118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964075020 0x7f79640753f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:05.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.965+0000 7f796bcc0700 1 --2- 172.21.15.93:0/3008613118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7964075930 0x7f7964110c70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:05.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.965+0000 7f796bcc0700 1 -- 172.21.15.93:0/3008613118 >> 172.21.15.93:0/3008613118 conn(0x7f79640fdfd0 msgr2=0x7f79641003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:05.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.965+0000 7f796bcc0700 1 -- 172.21.15.93:0/3008613118 shutdown_connections 2023-11-16T16:48:05.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.965+0000 7f796bcc0700 1 -- 172.21.15.93:0/3008613118 wait complete. 2023-11-16T16:48:05.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.966+0000 7f796bcc0700 1 Processor -- start 2023-11-16T16:48:05.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.966+0000 7f796bcc0700 1 -- start start 2023-11-16T16:48:05.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f796bcc0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964075020 0x7f79641a2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:05.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f796bcc0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7964075930 0x7f79641a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:05.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f796bcc0700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79641111b0 0x7f796419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:05.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f796bcc0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796419ce80 con 0x7f7964075930 2023-11-16T16:48:05.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f796bcc0700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796419cff0 con 0x7f79641111b0 2023-11-16T16:48:05.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f796bcc0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796419d1b0 con 0x7f7964075020 2023-11-16T16:48:05.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f796925b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7964075930 0x7f79641a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:05.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f796925b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7964075930 0x7f79641a2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40150/0 (socket says 172.21.15.93:40150) 2023-11-16T16:48:05.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f796925b700 1 -- 172.21.15.93:0/1831721065 learned_addr learned my addr 172.21.15.93:0/1831721065 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:05.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f7969a5c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964075020 0x7f79641a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:05.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f796a25d700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79641111b0 0x7f796419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:05.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f796925b700 1 -- 172.21.15.93:0/1831721065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964075020 msgr2=0x7f79641a2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:05.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.967+0000 7f796925b700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964075020 0x7f79641a2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:05.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.968+0000 7f796925b700 1 -- 172.21.15.93:0/1831721065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79641111b0 msgr2=0x7f796419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:05.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.968+0000 7f796925b700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79641111b0 0x7f796419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:05.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.968+0000 7f796925b700 1 -- 172.21.15.93:0/1831721065 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79540095f0 con 0x7f7964075930 2023-11-16T16:48:05.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.968+0000 7f7969a5c700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964075020 0x7f79641a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:05.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.968+0000 7f796925b700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7964075930 0x7f79641a2cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7954000c00 tx=0x7f7954004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:05.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.968+0000 7f795affd700 1 -- 172.21.15.93:0/1831721065 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795401d070 con 0x7f7964075930 2023-11-16T16:48:05.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.968+0000 7f795affd700 1 -- 172.21.15.93:0/1831721065 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7954021410 con 0x7f7964075930 2023-11-16T16:48:05.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.968+0000 7f795affd700 1 -- 172.21.15.93:0/1831721065 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954017650 con 0x7f7964075930 2023-11-16T16:48:05.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.968+0000 7f796bcc0700 1 -- 172.21.15.93:0/1831721065 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7964116960 con 0x7f7964075930 2023-11-16T16:48:05.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.968+0000 7f796bcc0700 1 -- 172.21.15.93:0/1831721065 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79641167d0 con 0x7f7964075930 2023-11-16T16:48:05.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.970+0000 7f796bcc0700 1 -- 172.21.15.93:0/1831721065 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7964116af0 con 0x7f7964075930 2023-11-16T16:48:05.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.971+0000 7f795affd700 1 -- 172.21.15.93:0/1831721065 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7954021580 con 0x7f7964075930 2023-11-16T16:48:05.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.972+0000 7f795affd700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f79500a1260 0x7f79500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:05.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.973+0000 7f795affd700 1 -- 172.21.15.93:0/1831721065 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f79540ce0d0 con 0x7f7964075930 2023-11-16T16:48:05.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.973+0000 7f7969a5c700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f79500a1260 0x7f79500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:05.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.973+0000 7f7969a5c700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f79500a1260 0x7f79500a3710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7960011490 tx=0x7f7960009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:05.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:05.977+0000 7f795affd700 1 -- 172.21.15.93:0/1831721065 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7954097320 con 0x7f7964075930 2023-11-16T16:48:06.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:05 smithi093 ceph-mon[97863]: pgmap v329: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2023-11-16T16:48:06.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.214+0000 7f796bcc0700 1 -- 172.21.15.93:0/1831721065 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7964066ea0 con 0x7f7964075930 2023-11-16T16:48:06.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.214+0000 7f795affd700 1 -- 172.21.15.93:0/1831721065 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f795409a940 con 0x7f7964075930 2023-11-16T16:48:06.216 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:06.216 INFO:teuthology.orchestra.run.smithi093.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-16T16:48:06.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.217+0000 7f796bcc0700 1 -- 172.21.15.93:0/1831721065 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f79500a1260 msgr2=0x7f79500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:06.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.217+0000 7f796bcc0700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f79500a1260 0x7f79500a3710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7960011490 tx=0x7f7960009260 comp rx=0 tx=0).stop 2023-11-16T16:48:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.217+0000 7f796bcc0700 1 -- 172.21.15.93:0/1831721065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7964075930 msgr2=0x7f79641a2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.217+0000 7f796bcc0700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7964075930 0x7f79641a2cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7954000c00 tx=0x7f7954004790 comp rx=0 tx=0).stop 2023-11-16T16:48:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.217+0000 7f796bcc0700 1 -- 172.21.15.93:0/1831721065 shutdown_connections 2023-11-16T16:48:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.217+0000 7f796bcc0700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f79500a1260 0x7f79500a3710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.217+0000 7f796bcc0700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79641111b0 0x7f796419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.217+0000 7f796bcc0700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964075020 0x7f79641a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.217+0000 7f796bcc0700 1 --2- 172.21.15.93:0/1831721065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7964075930 0x7f79641a2cd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.217+0000 7f796bcc0700 1 -- 172.21.15.93:0/1831721065 >> 172.21.15.93:0/1831721065 conn(0x7f79640fdfd0 msgr2=0x7f79641003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.217+0000 7f796bcc0700 1 -- 172.21.15.93:0/1831721065 shutdown_connections 2023-11-16T16:48:06.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.217+0000 7f796bcc0700 1 -- 172.21.15.93:0/1831721065 wait complete. 2023-11-16T16:48:06.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:05 smithi159 ceph-mon[131528]: pgmap v329: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2023-11-16T16:48:06.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:05 smithi125 ceph-mon[131076]: pgmap v329: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2023-11-16T16:48:06.725 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-11-16T16:48:06.726 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:48:06.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.927+0000 7fcfde9ff700 1 -- 172.21.15.93:0/3660150516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfd8100290 msgr2=0x7fcfd81006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:06.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.927+0000 7fcfde9ff700 1 --2- 172.21.15.93:0/3660150516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfd8100290 0x7fcfd81006a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009990 tx=0x7fcfc8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:06.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.928+0000 7fcfde9ff700 1 -- 172.21.15.93:0/3660150516 shutdown_connections 2023-11-16T16:48:06.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.928+0000 7fcfde9ff700 1 --2- 172.21.15.93:0/3660150516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfd8100290 0x7fcfd81006a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:06.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.928+0000 7fcfde9ff700 1 --2- 172.21.15.93:0/3660150516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd8101e10 0x7fcfd81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:06.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.928+0000 7fcfde9ff700 1 --2- 172.21.15.93:0/3660150516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd8101480 0x7fcfd81018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:06.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.928+0000 7fcfde9ff700 1 -- 172.21.15.93:0/3660150516 >> 172.21.15.93:0/3660150516 conn(0x7fcfd80fb870 msgr2=0x7fcfd80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:06.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.928+0000 7fcfde9ff700 1 -- 172.21.15.93:0/3660150516 shutdown_connections 2023-11-16T16:48:06.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.928+0000 7fcfde9ff700 1 -- 172.21.15.93:0/3660150516 wait complete. 2023-11-16T16:48:06.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.929+0000 7fcfde9ff700 1 Processor -- start 2023-11-16T16:48:06.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.929+0000 7fcfde9ff700 1 -- start start 2023-11-16T16:48:06.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfde9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd8100290 0x7fcfd8195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:06.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfde9ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfd8101480 0x7fcfd81964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:06.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfde9ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd8101e10 0x7fcfd819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:06.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfde9ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd819db30 con 0x7fcfd8101480 2023-11-16T16:48:06.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfde9ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd819dc70 con 0x7fcfd8101e10 2023-11-16T16:48:06.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfde9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd819de00 con 0x7fcfd8100290 2023-11-16T16:48:06.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfd77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfd8101480 0x7fcfd81964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:06.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfd7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd8100290 0x7fcfd8195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:06.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfd77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfd8101480 0x7fcfd81964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44422/0 (socket says 172.21.15.93:44422) 2023-11-16T16:48:06.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfd77fe700 1 -- 172.21.15.93:0/1606184553 learned_addr learned my addr 172.21.15.93:0/1606184553 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:06.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfdcf9c700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd8101e10 0x7fcfd819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:06.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfd77fe700 1 -- 172.21.15.93:0/1606184553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd8100290 msgr2=0x7fcfd8195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:06.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfd77fe700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd8100290 0x7fcfd8195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:06.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfd77fe700 1 -- 172.21.15.93:0/1606184553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd8101e10 msgr2=0x7fcfd819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:06.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfd77fe700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd8101e10 0x7fcfd819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:06.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.930+0000 7fcfd77fe700 1 -- 172.21.15.93:0/1606184553 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc80095f0 con 0x7fcfd8101480 2023-11-16T16:48:06.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.931+0000 7fcfd77fe700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfd8101480 0x7fcfd81964f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00e9a0 tx=0x7fcfcc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:06.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.931+0000 7fcfd57fa700 1 -- 172.21.15.93:0/1606184553 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc009790 con 0x7fcfd8101480 2023-11-16T16:48:06.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.931+0000 7fcfd57fa700 1 -- 172.21.15.93:0/1606184553 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfcc00cbd0 con 0x7fcfd8101480 2023-11-16T16:48:06.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.931+0000 7fcfde9ff700 1 -- 172.21.15.93:0/1606184553 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd810bc40 con 0x7fcfd8101480 2023-11-16T16:48:06.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.931+0000 7fcfd57fa700 1 -- 172.21.15.93:0/1606184553 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc018680 con 0x7fcfd8101480 2023-11-16T16:48:06.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.931+0000 7fcfde9ff700 1 -- 172.21.15.93:0/1606184553 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd810bab0 con 0x7fcfd8101480 2023-11-16T16:48:06.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.932+0000 7fcfde9ff700 1 -- 172.21.15.93:0/1606184553 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd8107880 con 0x7fcfd8101480 2023-11-16T16:48:06.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.934+0000 7fcfd57fa700 1 -- 172.21.15.93:0/1606184553 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fcfcc00cd40 con 0x7fcfd8101480 2023-11-16T16:48:06.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.935+0000 7fcfd57fa700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcfc00a11e0 0x7fcfc00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:06.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.935+0000 7fcfd57fa700 1 -- 172.21.15.93:0/1606184553 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fcfcc014070 con 0x7fcfd8101480 2023-11-16T16:48:06.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.935+0000 7fcfd7fff700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcfc00a11e0 0x7fcfc00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:06.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.936+0000 7fcfd7fff700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcfc00a11e0 0x7fcfc00a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800b2e0 tx=0x7fcfc8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:06.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:06.939+0000 7fcfd57fa700 1 -- 172.21.15.93:0/1606184553 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcfcc051730 con 0x7fcfd8101480 2023-11-16T16:48:07.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.139+0000 7fcfde9ff700 1 -- 172.21.15.93:0/1606184553 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfd802cfe0 con 0x7fcfd8101480 2023-11-16T16:48:07.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.141+0000 7fcfd57fa700 1 -- 172.21.15.93:0/1606184553 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24795 (secure 0 0 0) 0x7fcfcc0962e0 con 0x7fcfd8101480 2023-11-16T16:48:07.143 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:07.143 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":92,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:47:53.331398+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":1,"to":2}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:48:07.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.143+0000 7fcfde9ff700 1 -- 172.21.15.93:0/1606184553 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcfc00a11e0 msgr2=0x7fcfc00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:07.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.143+0000 7fcfde9ff700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcfc00a11e0 0x7fcfc00a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800b2e0 tx=0x7fcfc8005fb0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.143+0000 7fcfde9ff700 1 -- 172.21.15.93:0/1606184553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfd8101480 msgr2=0x7fcfd81964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:07.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.143+0000 7fcfde9ff700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfd8101480 0x7fcfd81964f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00e9a0 tx=0x7fcfcc00ecb0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.143+0000 7fcfde9ff700 1 -- 172.21.15.93:0/1606184553 shutdown_connections 2023-11-16T16:48:07.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.143+0000 7fcfde9ff700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcfc00a11e0 0x7fcfc00a3690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.143+0000 7fcfde9ff700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd8101e10 0x7fcfd819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.143+0000 7fcfde9ff700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd8100290 0x7fcfd8195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.143+0000 7fcfde9ff700 1 --2- 172.21.15.93:0/1606184553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfd8101480 0x7fcfd81964f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.143+0000 7fcfde9ff700 1 -- 172.21.15.93:0/1606184553 >> 172.21.15.93:0/1606184553 conn(0x7fcfd80fb870 msgr2=0x7fcfd81049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:07.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.143+0000 7fcfde9ff700 1 -- 172.21.15.93:0/1606184553 shutdown_connections 2023-11-16T16:48:07.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.143+0000 7fcfde9ff700 1 -- 172.21.15.93:0/1606184553 wait complete. 2023-11-16T16:48:07.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:06 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1831721065' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-16T16:48:07.164 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:42:13.029135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:46:35.301321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:46:36.552545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-16T16:46:39.628380+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-16T16:48:07.165 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:48:07.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:06 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1831721065' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-16T16:48:07.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:06 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1831721065' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-16T16:48:07.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.363+0000 7f52f23ff700 1 -- 172.21.15.93:0/1983550249 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52ec101e40 msgr2=0x7f52ec108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:07.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.363+0000 7f52f23ff700 1 --2- 172.21.15.93:0/1983550249 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52ec101e40 0x7f52ec108700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009f90 tx=0x7f52e400b330 comp rx=0 tx=0).stop 2023-11-16T16:48:07.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.363+0000 7f52f23ff700 1 -- 172.21.15.93:0/1983550249 shutdown_connections 2023-11-16T16:48:07.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.363+0000 7f52f23ff700 1 --2- 172.21.15.93:0/1983550249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52ec1002c0 0x7f52ec1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.364+0000 7f52f23ff700 1 --2- 172.21.15.93:0/1983550249 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52ec101e40 0x7f52ec108700 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.364+0000 7f52f23ff700 1 --2- 172.21.15.93:0/1983550249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52ec1014b0 0x7f52ec101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.364+0000 7f52f23ff700 1 -- 172.21.15.93:0/1983550249 >> 172.21.15.93:0/1983550249 conn(0x7f52ec0fb8a0 msgr2=0x7f52ec0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:07.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.364+0000 7f52f23ff700 1 -- 172.21.15.93:0/1983550249 shutdown_connections 2023-11-16T16:48:07.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.364+0000 7f52f23ff700 1 -- 172.21.15.93:0/1983550249 wait complete. 2023-11-16T16:48:07.368 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.365+0000 7f52f23ff700 1 Processor -- start 2023-11-16T16:48:07.368 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.365+0000 7f52f23ff700 1 -- start start 2023-11-16T16:48:07.368 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52f23ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52ec1002c0 0x7f52ec196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:07.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52f23ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52ec1014b0 0x7f52ec196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:07.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52f23ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52ec101e40 0x7f52ec19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:07.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52f23ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec19dcc0 con 0x7f52ec1014b0 2023-11-16T16:48:07.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52f23ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec19de00 con 0x7f52ec1002c0 2023-11-16T16:48:07.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52f23ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec19df90 con 0x7f52ec101e40 2023-11-16T16:48:07.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52ec1014b0 0x7f52ec196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:07.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52ebfff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52ec1002c0 0x7f52ec196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:07.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52ec1014b0 0x7f52ec196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44444/0 (socket says 172.21.15.93:44444) 2023-11-16T16:48:07.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52eb7fe700 1 -- 172.21.15.93:0/1543756026 learned_addr learned my addr 172.21.15.93:0/1543756026 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:07.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52f099c700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52ec101e40 0x7f52ec19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:07.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52ebfff700 1 -- 172.21.15.93:0/1543756026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52ec101e40 msgr2=0x7f52ec19d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:07.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52ebfff700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52ec101e40 0x7f52ec19d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52ebfff700 1 -- 172.21.15.93:0/1543756026 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52ec1014b0 msgr2=0x7f52ec196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:07.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52ebfff700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52ec1014b0 0x7f52ec196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52ebfff700 1 -- 172.21.15.93:0/1543756026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4009c40 con 0x7f52ec1002c0 2023-11-16T16:48:07.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.366+0000 7f52f099c700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52ec101e40 0x7f52ec19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:07.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.367+0000 7f52ebfff700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52ec1002c0 0x7f52ec196140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52dc005fd0 tx=0x7f52dc00eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:07.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.367+0000 7f52e97fa700 1 -- 172.21.15.93:0/1543756026 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc004220 con 0x7f52ec1002c0 2023-11-16T16:48:07.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.367+0000 7f52e97fa700 1 -- 172.21.15.93:0/1543756026 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52dc004380 con 0x7f52ec1002c0 2023-11-16T16:48:07.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.367+0000 7f52f23ff700 1 -- 172.21.15.93:0/1543756026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec10bc90 con 0x7f52ec1002c0 2023-11-16T16:48:07.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.367+0000 7f52e97fa700 1 -- 172.21.15.93:0/1543756026 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc003b00 con 0x7f52ec1002c0 2023-11-16T16:48:07.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.367+0000 7f52f23ff700 1 -- 172.21.15.93:0/1543756026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec10bb00 con 0x7f52ec1002c0 2023-11-16T16:48:07.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.368+0000 7f52f23ff700 1 -- 172.21.15.93:0/1543756026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ec107100 con 0x7f52ec1002c0 2023-11-16T16:48:07.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.370+0000 7f52e97fa700 1 -- 172.21.15.93:0/1543756026 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f52dc003d60 con 0x7f52ec1002c0 2023-11-16T16:48:07.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.371+0000 7f52e97fa700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f52d40a11e0 0x7f52d40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:07.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.371+0000 7f52e97fa700 1 -- 172.21.15.93:0/1543756026 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f52dc014070 con 0x7f52ec1002c0 2023-11-16T16:48:07.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.371+0000 7f52eb7fe700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f52d40a11e0 0x7f52d40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:07.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.372+0000 7f52eb7fe700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f52d40a11e0 0x7f52d40a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f52e0010e40 tx=0x7f52e0012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:07.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.373+0000 7f52e97fa700 1 -- 172.21.15.93:0/1543756026 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f52dc096870 con 0x7f52ec1002c0 2023-11-16T16:48:07.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.571+0000 7f52f23ff700 1 -- 172.21.15.93:0/1543756026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52ec061700 con 0x7f52ec1002c0 2023-11-16T16:48:07.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.571+0000 7f52e97fa700 1 -- 172.21.15.93:0/1543756026 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52dc099e90 con 0x7f52ec1002c0 2023-11-16T16:48:07.573 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-11-16T16:48:07.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.573+0000 7f52f23ff700 1 -- 172.21.15.93:0/1543756026 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f52d40a11e0 msgr2=0x7f52d40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:07.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.573+0000 7f52f23ff700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f52d40a11e0 0x7f52d40a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f52e0010e40 tx=0x7f52e0012040 comp rx=0 tx=0).stop 2023-11-16T16:48:07.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.574+0000 7f52f23ff700 1 -- 172.21.15.93:0/1543756026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52ec1002c0 msgr2=0x7f52ec196140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:07.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.574+0000 7f52f23ff700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52ec1002c0 0x7f52ec196140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52dc005fd0 tx=0x7f52dc00eca0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.574+0000 7f52f23ff700 1 -- 172.21.15.93:0/1543756026 shutdown_connections 2023-11-16T16:48:07.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.574+0000 7f52f23ff700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f52d40a11e0 0x7f52d40a3690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.574+0000 7f52f23ff700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52ec101e40 0x7f52ec19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.574+0000 7f52f23ff700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52ec1002c0 0x7f52ec196140 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.574+0000 7f52f23ff700 1 --2- 172.21.15.93:0/1543756026 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52ec1014b0 0x7f52ec196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.574+0000 7f52f23ff700 1 -- 172.21.15.93:0/1543756026 >> 172.21.15.93:0/1543756026 conn(0x7f52ec0fb8a0 msgr2=0x7f52ec0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:07.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.574+0000 7f52f23ff700 1 -- 172.21.15.93:0/1543756026 shutdown_connections 2023-11-16T16:48:07.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.574+0000 7f52f23ff700 1 -- 172.21.15.93:0/1543756026 wait complete. 2023-11-16T16:48:07.594 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:48:07.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.792+0000 7f09473ff700 1 -- 172.21.15.93:0/418961903 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0940101e40 msgr2=0x7f0940108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:07.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.792+0000 7f09473ff700 1 --2- 172.21.15.93:0/418961903 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0940101e40 0x7f0940108700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f093800b0a0 tx=0x7f093800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.792+0000 7f09473ff700 1 -- 172.21.15.93:0/418961903 shutdown_connections 2023-11-16T16:48:07.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.792+0000 7f09473ff700 1 --2- 172.21.15.93:0/418961903 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0940101e40 0x7f0940108700 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.792+0000 7f09473ff700 1 --2- 172.21.15.93:0/418961903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09401002c0 0x7f09401006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.792+0000 7f09473ff700 1 --2- 172.21.15.93:0/418961903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09401014b0 0x7f0940101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.792+0000 7f09473ff700 1 -- 172.21.15.93:0/418961903 >> 172.21.15.93:0/418961903 conn(0x7f09400fb8a0 msgr2=0x7f09400fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:07.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.792+0000 7f09473ff700 1 -- 172.21.15.93:0/418961903 shutdown_connections 2023-11-16T16:48:07.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.792+0000 7f09473ff700 1 -- 172.21.15.93:0/418961903 wait complete. 2023-11-16T16:48:07.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.793+0000 7f09473ff700 1 Processor -- start 2023-11-16T16:48:07.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.793+0000 7f09473ff700 1 -- start start 2023-11-16T16:48:07.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f09473ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09401002c0 0x7f0940198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:07.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f09473ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09401014b0 0x7f0940198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:07.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f09473ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0940101e40 0x7f094019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:07.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f09473ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094019fec0 con 0x7f09401014b0 2023-11-16T16:48:07.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f09473ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09401a0000 con 0x7f0940101e40 2023-11-16T16:48:07.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f09473ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09401a0190 con 0x7f09401002c0 2023-11-16T16:48:07.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f094519b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09401002c0 0x7f0940198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:07.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f094499a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09401014b0 0x7f0940198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:07.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f094519b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09401002c0 0x7f0940198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:58504/0 (socket says 172.21.15.93:58504) 2023-11-16T16:48:07.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f094519b700 1 -- 172.21.15.93:0/1538202013 learned_addr learned my addr 172.21.15.93:0/1538202013 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:07.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f094599c700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0940101e40 0x7f094019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:07.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f094499a700 1 -- 172.21.15.93:0/1538202013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09401002c0 msgr2=0x7f0940198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:07.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f094499a700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09401002c0 0x7f0940198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f094499a700 1 -- 172.21.15.93:0/1538202013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0940101e40 msgr2=0x7f094019f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:07.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f094499a700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0940101e40 0x7f094019f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:07.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f094499a700 1 -- 172.21.15.93:0/1538202013 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0930009520 con 0x7f09401014b0 2023-11-16T16:48:07.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f094519b700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09401002c0 0x7f0940198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:07.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.794+0000 7f094599c700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0940101e40 0x7f094019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:07.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.795+0000 7f094499a700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09401014b0 0x7f0940198880 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f093000e9f0 tx=0x7f093000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:07.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.795+0000 7f09367fc700 1 -- 172.21.15.93:0/1538202013 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000cb70 con 0x7f09401014b0 2023-11-16T16:48:07.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.795+0000 7f09367fc700 1 -- 172.21.15.93:0/1538202013 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f093000ee90 con 0x7f09401014b0 2023-11-16T16:48:07.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.795+0000 7f09367fc700 1 -- 172.21.15.93:0/1538202013 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930003cf0 con 0x7f09401014b0 2023-11-16T16:48:07.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.795+0000 7f09473ff700 1 -- 172.21.15.93:0/1538202013 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0938009d00 con 0x7f09401014b0 2023-11-16T16:48:07.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.795+0000 7f09473ff700 1 -- 172.21.15.93:0/1538202013 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094010bc90 con 0x7f09401014b0 2023-11-16T16:48:07.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.796+0000 7f09473ff700 1 -- 172.21.15.93:0/1538202013 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0940067b00 con 0x7f09401014b0 2023-11-16T16:48:07.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.798+0000 7f09367fc700 1 -- 172.21.15.93:0/1538202013 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f093000ccd0 con 0x7f09401014b0 2023-11-16T16:48:07.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.799+0000 7f09367fc700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f092c0a11e0 0x7f092c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:07.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.799+0000 7f09367fc700 1 -- 172.21.15.93:0/1538202013 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f0930014070 con 0x7f09401014b0 2023-11-16T16:48:07.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.799+0000 7f094519b700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f092c0a11e0 0x7f092c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:07.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.800+0000 7f094519b700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f092c0a11e0 0x7f092c0a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f093c005950 tx=0x7f093c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:07.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:07.803+0000 7f09367fc700 1 -- 172.21.15.93:0/1538202013 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0930096340 con 0x7f09401014b0 2023-11-16T16:48:08.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.004+0000 7f09473ff700 1 -- 172.21.15.93:0/1538202013 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09401a0da0 con 0x7f09401014b0 2023-11-16T16:48:08.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.005+0000 7f09367fc700 1 -- 172.21.15.93:0/1538202013 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0930099960 con 0x7f09401014b0 2023-11-16T16:48:08.007 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:48:08.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.007+0000 7f09473ff700 1 -- 172.21.15.93:0/1538202013 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f092c0a11e0 msgr2=0x7f092c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:08.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.007+0000 7f09473ff700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f092c0a11e0 0x7f092c0a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f093c005950 tx=0x7f093c009260 comp rx=0 tx=0).stop 2023-11-16T16:48:08.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.007+0000 7f09473ff700 1 -- 172.21.15.93:0/1538202013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09401014b0 msgr2=0x7f0940198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:08.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.007+0000 7f09473ff700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09401014b0 0x7f0940198880 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f093000e9f0 tx=0x7f093000ed00 comp rx=0 tx=0).stop 2023-11-16T16:48:08.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.007+0000 7f09473ff700 1 -- 172.21.15.93:0/1538202013 shutdown_connections 2023-11-16T16:48:08.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.007+0000 7f09473ff700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f092c0a11e0 0x7f092c0a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.007+0000 7f09473ff700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0940101e40 0x7f094019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.007+0000 7f09473ff700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09401002c0 0x7f0940198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.007+0000 7f09473ff700 1 --2- 172.21.15.93:0/1538202013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09401014b0 0x7f0940198880 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.007+0000 7f09473ff700 1 -- 172.21.15.93:0/1538202013 >> 172.21.15.93:0/1538202013 conn(0x7f09400fb8a0 msgr2=0x7f09400fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:08.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.007+0000 7f09473ff700 1 -- 172.21.15.93:0/1538202013 shutdown_connections 2023-11-16T16:48:08.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.007+0000 7f09473ff700 1 -- 172.21.15.93:0/1538202013 wait complete. 2023-11-16T16:48:08.027 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:48:08.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:07 smithi093 ceph-mon[97863]: pgmap v330: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:08.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:07 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1606184553' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:08.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:07 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1543756026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:48:08.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.228+0000 7f932fdff700 1 -- 172.21.15.93:0/3414202462 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93281014c0 msgr2=0x7f9328101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:08.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.228+0000 7f932fdff700 1 --2- 172.21.15.93:0/3414202462 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93281014c0 0x7f9328101910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93180098f0 tx=0x7f9318009c00 comp rx=0 tx=0).stop 2023-11-16T16:48:08.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.228+0000 7f932fdff700 1 -- 172.21.15.93:0/3414202462 shutdown_connections 2023-11-16T16:48:08.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.228+0000 7f932fdff700 1 --2- 172.21.15.93:0/3414202462 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93281002d0 0x7f93281006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.228+0000 7f932fdff700 1 --2- 172.21.15.93:0/3414202462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9328101e50 0x7f9328108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.228+0000 7f932fdff700 1 --2- 172.21.15.93:0/3414202462 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93281014c0 0x7f9328101910 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.228+0000 7f932fdff700 1 -- 172.21.15.93:0/3414202462 >> 172.21.15.93:0/3414202462 conn(0x7f93280fb8b0 msgr2=0x7f93280fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:08.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.228+0000 7f932fdff700 1 -- 172.21.15.93:0/3414202462 shutdown_connections 2023-11-16T16:48:08.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.228+0000 7f932fdff700 1 -- 172.21.15.93:0/3414202462 wait complete. 2023-11-16T16:48:08.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.229+0000 7f932fdff700 1 Processor -- start 2023-11-16T16:48:08.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.230+0000 7f932fdff700 1 -- start start 2023-11-16T16:48:08.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.230+0000 7f932fdff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93281002d0 0x7f93281960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:08.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.230+0000 7f932fdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93281014c0 0x7f9328196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:08.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.230+0000 7f932fdff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9328101e50 0x7f932819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:08.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.230+0000 7f932fdff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932819dc60 con 0x7f93281002d0 2023-11-16T16:48:08.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.230+0000 7f932fdff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932819dda0 con 0x7f9328101e50 2023-11-16T16:48:08.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.230+0000 7f932fdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932819df30 con 0x7f93281014c0 2023-11-16T16:48:08.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.230+0000 7f932db9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93281002d0 0x7f93281960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:08.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.230+0000 7f932db9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93281002d0 0x7f93281960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44472/0 (socket says 172.21.15.93:44472) 2023-11-16T16:48:08.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.230+0000 7f932db9b700 1 -- 172.21.15.93:0/2157159912 learned_addr learned my addr 172.21.15.93:0/2157159912 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:08.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.230+0000 7f932d39a700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93281014c0 0x7f9328196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:08.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.230+0000 7f932e39c700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9328101e50 0x7f932819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:08.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.231+0000 7f932db9b700 1 -- 172.21.15.93:0/2157159912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93281014c0 msgr2=0x7f9328196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:08.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.231+0000 7f932db9b700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93281014c0 0x7f9328196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.231+0000 7f932db9b700 1 -- 172.21.15.93:0/2157159912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9328101e50 msgr2=0x7f932819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:08.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.231+0000 7f932db9b700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9328101e50 0x7f932819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.231+0000 7f932db9b700 1 -- 172.21.15.93:0/2157159912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93240095f0 con 0x7f93281002d0 2023-11-16T16:48:08.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.231+0000 7f932db9b700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93281002d0 0x7f93281960e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f932400ebf0 tx=0x7f932400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:08.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.231+0000 7f931effd700 1 -- 172.21.15.93:0/2157159912 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324009a10 con 0x7f93281002d0 2023-11-16T16:48:08.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.231+0000 7f931effd700 1 -- 172.21.15.93:0/2157159912 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f932400cc90 con 0x7f93281002d0 2023-11-16T16:48:08.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.231+0000 7f932fdff700 1 -- 172.21.15.93:0/2157159912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9318009520 con 0x7f93281002d0 2023-11-16T16:48:08.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.231+0000 7f931effd700 1 -- 172.21.15.93:0/2157159912 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324018750 con 0x7f93281002d0 2023-11-16T16:48:08.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.231+0000 7f932fdff700 1 -- 172.21.15.93:0/2157159912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932810bc80 con 0x7f93281002d0 2023-11-16T16:48:08.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:07 smithi159 ceph-mon[131528]: pgmap v330: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:08.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:07 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1606184553' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:08.236 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:07 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1543756026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:48:08.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.232+0000 7f932fdff700 1 -- 172.21.15.93:0/2157159912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93281078e0 con 0x7f93281002d0 2023-11-16T16:48:08.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.234+0000 7f931effd700 1 -- 172.21.15.93:0/2157159912 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f93240188b0 con 0x7f93281002d0 2023-11-16T16:48:08.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.235+0000 7f931effd700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f93140a1260 0x7f93140a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:08.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.236+0000 7f931effd700 1 -- 172.21.15.93:0/2157159912 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f9324014070 con 0x7f93281002d0 2023-11-16T16:48:08.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.236+0000 7f932d39a700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f93140a1260 0x7f93140a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:08.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.236+0000 7f932d39a700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f93140a1260 0x7f93140a3710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9318005e90 tx=0x7f9318019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:08.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.239+0000 7f931effd700 1 -- 172.21.15.93:0/2157159912 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9324096600 con 0x7f93281002d0 2023-11-16T16:48:08.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:07 smithi125 ceph-mon[131076]: pgmap v330: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:08.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:07 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1606184553' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:08.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:07 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1543756026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:48:08.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.438+0000 7f932fdff700 1 -- 172.21.15.93:0/2157159912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9328067b00 con 0x7f93281002d0 2023-11-16T16:48:08.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.439+0000 7f931effd700 1 -- 172.21.15.93:0/2157159912 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9324099c20 con 0x7f93281002d0 2023-11-16T16:48:08.441 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-11-16T16:48:08.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.441+0000 7f932fdff700 1 -- 172.21.15.93:0/2157159912 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f93140a1260 msgr2=0x7f93140a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:08.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.441+0000 7f932fdff700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f93140a1260 0x7f93140a3710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9318005e90 tx=0x7f9318019040 comp rx=0 tx=0).stop 2023-11-16T16:48:08.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.441+0000 7f932fdff700 1 -- 172.21.15.93:0/2157159912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93281002d0 msgr2=0x7f93281960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:08.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.441+0000 7f932fdff700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93281002d0 0x7f93281960e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f932400ebf0 tx=0x7f932400ef00 comp rx=0 tx=0).stop 2023-11-16T16:48:08.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.442+0000 7f932fdff700 1 -- 172.21.15.93:0/2157159912 shutdown_connections 2023-11-16T16:48:08.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.442+0000 7f932fdff700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f93140a1260 0x7f93140a3710 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.442+0000 7f932fdff700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93281002d0 0x7f93281960e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.442+0000 7f932fdff700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9328101e50 0x7f932819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.442+0000 7f932fdff700 1 --2- 172.21.15.93:0/2157159912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93281014c0 0x7f9328196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.442+0000 7f932fdff700 1 -- 172.21.15.93:0/2157159912 >> 172.21.15.93:0/2157159912 conn(0x7f93280fb8b0 msgr2=0x7f9328104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:08.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.442+0000 7f932fdff700 1 -- 172.21.15.93:0/2157159912 shutdown_connections 2023-11-16T16:48:08.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.442+0000 7f932fdff700 1 -- 172.21.15.93:0/2157159912 wait complete. 2023-11-16T16:48:08.462 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-16T16:48:08.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.661+0000 7f4c4bbff700 1 -- 172.21.15.93:0/4287972972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c44101e40 msgr2=0x7f4c44108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:08.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.661+0000 7f4c4bbff700 1 --2- 172.21.15.93:0/4287972972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c44101e40 0x7f4c44108700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00b0a0 tx=0x7f4c3c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.662+0000 7f4c4bbff700 1 -- 172.21.15.93:0/4287972972 shutdown_connections 2023-11-16T16:48:08.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.662+0000 7f4c4bbff700 1 --2- 172.21.15.93:0/4287972972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c44101e40 0x7f4c44108700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.662+0000 7f4c4bbff700 1 --2- 172.21.15.93:0/4287972972 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c441002c0 0x7f4c441006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.662+0000 7f4c4bbff700 1 --2- 172.21.15.93:0/4287972972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c441014b0 0x7f4c44101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.662+0000 7f4c4bbff700 1 -- 172.21.15.93:0/4287972972 >> 172.21.15.93:0/4287972972 conn(0x7f4c440fb8a0 msgr2=0x7f4c440fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:08.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.662+0000 7f4c4bbff700 1 -- 172.21.15.93:0/4287972972 shutdown_connections 2023-11-16T16:48:08.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.662+0000 7f4c4bbff700 1 -- 172.21.15.93:0/4287972972 wait complete. 2023-11-16T16:48:08.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.663+0000 7f4c4bbff700 1 Processor -- start 2023-11-16T16:48:08.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.663+0000 7f4c4bbff700 1 -- start start 2023-11-16T16:48:08.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.663+0000 7f4c4bbff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c441002c0 0x7f4c44198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:08.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.663+0000 7f4c4bbff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c441014b0 0x7f4c441988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:08.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.663+0000 7f4c4bbff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c44101e40 0x7f4c4419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:08.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.663+0000 7f4c4bbff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4419ff00 con 0x7f4c441014b0 2023-11-16T16:48:08.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.663+0000 7f4c4bbff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c441a0040 con 0x7f4c44101e40 2023-11-16T16:48:08.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.663+0000 7f4c4bbff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c441a01d0 con 0x7f4c441002c0 2023-11-16T16:48:08.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.663+0000 7f4c4919a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c441014b0 0x7f4c441988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:08.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.663+0000 7f4c4919a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c441014b0 0x7f4c441988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44494/0 (socket says 172.21.15.93:44494) 2023-11-16T16:48:08.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c4919a700 1 -- 172.21.15.93:0/2704583633 learned_addr learned my addr 172.21.15.93:0/2704583633 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:08.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c4999b700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c441002c0 0x7f4c44198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:08.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c4a19c700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c44101e40 0x7f4c4419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:08.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c4919a700 1 -- 172.21.15.93:0/2704583633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c441002c0 msgr2=0x7f4c44198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:08.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c4919a700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c441002c0 0x7f4c44198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c4919a700 1 -- 172.21.15.93:0/2704583633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c44101e40 msgr2=0x7f4c4419f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:08.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c4919a700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c44101e40 0x7f4c4419f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c4919a700 1 -- 172.21.15.93:0/2704583633 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34009520 con 0x7f4c441014b0 2023-11-16T16:48:08.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c4919a700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c441014b0 0x7f4c441988c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400eb30 tx=0x7f4c3400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:08.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c3affd700 1 -- 172.21.15.93:0/2704583633 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34009880 con 0x7f4c441014b0 2023-11-16T16:48:08.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c3affd700 1 -- 172.21.15.93:0/2704583633 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c340044e0 con 0x7f4c441014b0 2023-11-16T16:48:08.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c3affd700 1 -- 172.21.15.93:0/2704583633 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34003d80 con 0x7f4c441014b0 2023-11-16T16:48:08.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.664+0000 7f4c4bbff700 1 -- 172.21.15.93:0/2704583633 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c009d00 con 0x7f4c441014b0 2023-11-16T16:48:08.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.665+0000 7f4c4bbff700 1 -- 172.21.15.93:0/2704583633 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4410bc90 con 0x7f4c441014b0 2023-11-16T16:48:08.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.666+0000 7f4c4bbff700 1 -- 172.21.15.93:0/2704583633 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c44067b00 con 0x7f4c441014b0 2023-11-16T16:48:08.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.667+0000 7f4c3affd700 1 -- 172.21.15.93:0/2704583633 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4c34003f20 con 0x7f4c441014b0 2023-11-16T16:48:08.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.669+0000 7f4c3affd700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4c300a11e0 0x7f4c300a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:08.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.669+0000 7f4c3affd700 1 -- 172.21.15.93:0/2704583633 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f4c34014070 con 0x7f4c441014b0 2023-11-16T16:48:08.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.669+0000 7f4c4999b700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4c300a11e0 0x7f4c300a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:08.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.669+0000 7f4c4999b700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4c300a11e0 0x7f4c300a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4c44101310 tx=0x7f4c40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:08.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.672+0000 7f4c3affd700 1 -- 172.21.15.93:0/2704583633 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4c340965c0 con 0x7f4c441014b0 2023-11-16T16:48:08.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.872+0000 7f4c4bbff700 1 -- 172.21.15.93:0/2704583633 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4c441a0de0 con 0x7f4c441014b0 2023-11-16T16:48:08.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.873+0000 7f4c3affd700 1 -- 172.21.15.93:0/2704583633 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4c34099be0 con 0x7f4c441014b0 2023-11-16T16:48:08.875 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:48:08.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.875+0000 7f4c4bbff700 1 -- 172.21.15.93:0/2704583633 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4c300a11e0 msgr2=0x7f4c300a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:08.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.875+0000 7f4c4bbff700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4c300a11e0 0x7f4c300a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4c44101310 tx=0x7f4c40009260 comp rx=0 tx=0).stop 2023-11-16T16:48:08.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.875+0000 7f4c4bbff700 1 -- 172.21.15.93:0/2704583633 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c441014b0 msgr2=0x7f4c441988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:08.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.875+0000 7f4c4bbff700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c441014b0 0x7f4c441988c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400eb30 tx=0x7f4c3400ee40 comp rx=0 tx=0).stop 2023-11-16T16:48:08.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.875+0000 7f4c4bbff700 1 -- 172.21.15.93:0/2704583633 shutdown_connections 2023-11-16T16:48:08.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.875+0000 7f4c4bbff700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4c300a11e0 0x7f4c300a3690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.875+0000 7f4c4bbff700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c44101e40 0x7f4c4419f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.875+0000 7f4c4bbff700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c441002c0 0x7f4c44198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.875+0000 7f4c4bbff700 1 --2- 172.21.15.93:0/2704583633 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c441014b0 0x7f4c441988c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:08.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.876+0000 7f4c4bbff700 1 -- 172.21.15.93:0/2704583633 >> 172.21.15.93:0/2704583633 conn(0x7f4c440fb8a0 msgr2=0x7f4c44106ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:08.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.876+0000 7f4c4bbff700 1 -- 172.21.15.93:0/2704583633 shutdown_connections 2023-11-16T16:48:08.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:08.876+0000 7f4c4bbff700 1 -- 172.21.15.93:0/2704583633 wait complete. 2023-11-16T16:48:08.897 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:48:09.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.094+0000 7fb33bdff700 1 -- 172.21.15.93:0/1475788339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb334101e10 msgr2=0x7fb3341086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:09.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.094+0000 7fb33bdff700 1 --2- 172.21.15.93:0/1475788339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb334101e10 0x7fb3341086d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b0a0 tx=0x7fb32c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.094+0000 7fb33bdff700 1 -- 172.21.15.93:0/1475788339 shutdown_connections 2023-11-16T16:48:09.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.094+0000 7fb33bdff700 1 --2- 172.21.15.93:0/1475788339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb334101e10 0x7fb3341086d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.094+0000 7fb33bdff700 1 --2- 172.21.15.93:0/1475788339 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb334100290 0x7fb3341006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.094+0000 7fb33bdff700 1 --2- 172.21.15.93:0/1475788339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb334101480 0x7fb3341018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.094+0000 7fb33bdff700 1 -- 172.21.15.93:0/1475788339 >> 172.21.15.93:0/1475788339 conn(0x7fb3340fb870 msgr2=0x7fb3340fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:09.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.094+0000 7fb33bdff700 1 -- 172.21.15.93:0/1475788339 shutdown_connections 2023-11-16T16:48:09.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.094+0000 7fb33bdff700 1 -- 172.21.15.93:0/1475788339 wait complete. 2023-11-16T16:48:09.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.095+0000 7fb33bdff700 1 Processor -- start 2023-11-16T16:48:09.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.095+0000 7fb33bdff700 1 -- start start 2023-11-16T16:48:09.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb33bdff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb334100290 0x7fb334198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:09.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb33bdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb334101480 0x7fb3341987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:09.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb33bdff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb334101e10 0x7fb33419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:09.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb33bdff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33419fe10 con 0x7fb334100290 2023-11-16T16:48:09.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb33bdff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33419ff50 con 0x7fb334101e10 2023-11-16T16:48:09.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb33bdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3341a00e0 con 0x7fb334101480 2023-11-16T16:48:09.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb339b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb334100290 0x7fb334198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:09.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb339b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb334100290 0x7fb334198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44518/0 (socket says 172.21.15.93:44518) 2023-11-16T16:48:09.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb339b9b700 1 -- 172.21.15.93:0/3872543111 learned_addr learned my addr 172.21.15.93:0/3872543111 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:09.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb33a39c700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb334101e10 0x7fb33419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:09.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb33939a700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb334101480 0x7fb3341987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:09.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb339b9b700 1 -- 172.21.15.93:0/3872543111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb334101480 msgr2=0x7fb3341987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:09.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb339b9b700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb334101480 0x7fb3341987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb339b9b700 1 -- 172.21.15.93:0/3872543111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb334101e10 msgr2=0x7fb33419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:09.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb339b9b700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb334101e10 0x7fb33419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb339b9b700 1 -- 172.21.15.93:0/3872543111 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c009d00 con 0x7fb334100290 2023-11-16T16:48:09.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.096+0000 7fb339b9b700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb334100290 0x7fb334198290 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e970 tx=0x7fb33000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:09.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.097+0000 7fb32affd700 1 -- 172.21.15.93:0/3872543111 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330009790 con 0x7fb334100290 2023-11-16T16:48:09.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.097+0000 7fb32affd700 1 -- 172.21.15.93:0/3872543111 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb33000c7e0 con 0x7fb334100290 2023-11-16T16:48:09.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.097+0000 7fb33bdff700 1 -- 172.21.15.93:0/3872543111 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33410bc70 con 0x7fb334100290 2023-11-16T16:48:09.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.097+0000 7fb32affd700 1 -- 172.21.15.93:0/3872543111 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330003c70 con 0x7fb334100290 2023-11-16T16:48:09.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.097+0000 7fb33bdff700 1 -- 172.21.15.93:0/3872543111 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33410bae0 con 0x7fb334100290 2023-11-16T16:48:09.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.098+0000 7fb33bdff700 1 -- 172.21.15.93:0/3872543111 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3341a07e0 con 0x7fb334100290 2023-11-16T16:48:09.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.100+0000 7fb32affd700 1 -- 172.21.15.93:0/3872543111 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb33000cc50 con 0x7fb334100290 2023-11-16T16:48:09.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.101+0000 7fb32affd700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb3200a11e0 0x7fb3200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:09.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.101+0000 7fb32affd700 1 -- 172.21.15.93:0/3872543111 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fb330014070 con 0x7fb334100290 2023-11-16T16:48:09.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.101+0000 7fb33939a700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb3200a11e0 0x7fb3200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:09.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.102+0000 7fb33939a700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb3200a11e0 0x7fb3200a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb324009c60 tx=0x7fb324009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:09.106 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.105+0000 7fb32affd700 1 -- 172.21.15.93:0/3872543111 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb330096220 con 0x7fb334100290 2023-11-16T16:48:09.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:08 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1538202013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:48:09.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:08 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2157159912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:48:09.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:08 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1538202013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:48:09.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:08 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2157159912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:48:09.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:08 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1538202013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:48:09.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:08 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2157159912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:48:09.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.352+0000 7fb33bdff700 1 -- 172.21.15.93:0/3872543111 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3341a0d90 con 0x7fb334100290 2023-11-16T16:48:09.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.353+0000 7fb32affd700 1 -- 172.21.15.93:0/3872543111 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10811 (secure 0 0 0) 0x7fb330003dd0 con 0x7fb334100290 2023-11-16T16:48:09.355 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:09.355 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3614592201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3614592201},{"type":"v1","addr":"172.21.15.93:6839","nonce":3614592201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4068789362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4068789362},{"type":"v1","addr":"172.21.15.93:6841","nonce":4068789362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:46:37.124338+0000","modified":"2023-11-16T16:47:10.311966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24541,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/778982589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":778982589},{"type":"v1","addr":"172.21.15.159:6837","nonce":778982589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6839/149300707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":149300707},{"type":"v1","addr":"172.21.15.159:6839","nonce":149300707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/3590027328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":3590027328},{"type":"v1","addr":"172.21.15.125:6839","nonce":3590027328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:48:09.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.356+0000 7fb33bdff700 1 -- 172.21.15.93:0/3872543111 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb3200a11e0 msgr2=0x7fb3200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:09.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.356+0000 7fb33bdff700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb3200a11e0 0x7fb3200a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb324009c60 tx=0x7fb324009260 comp rx=0 tx=0).stop 2023-11-16T16:48:09.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.356+0000 7fb33bdff700 1 -- 172.21.15.93:0/3872543111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb334100290 msgr2=0x7fb334198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:09.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.356+0000 7fb33bdff700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb334100290 0x7fb334198290 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e970 tx=0x7fb33000ec80 comp rx=0 tx=0).stop 2023-11-16T16:48:09.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.356+0000 7fb33bdff700 1 -- 172.21.15.93:0/3872543111 shutdown_connections 2023-11-16T16:48:09.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.356+0000 7fb33bdff700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb3200a11e0 0x7fb3200a3690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.356+0000 7fb33bdff700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb334100290 0x7fb334198290 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.356+0000 7fb33bdff700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb334101e10 0x7fb33419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.356+0000 7fb33bdff700 1 --2- 172.21.15.93:0/3872543111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb334101480 0x7fb3341987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.356+0000 7fb33bdff700 1 -- 172.21.15.93:0/3872543111 >> 172.21.15.93:0/3872543111 conn(0x7fb3340fb870 msgr2=0x7fb3340fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:09.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.356+0000 7fb33bdff700 1 -- 172.21.15.93:0/3872543111 shutdown_connections 2023-11-16T16:48:09.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.356+0000 7fb33bdff700 1 -- 172.21.15.93:0/3872543111 wait complete. 2023-11-16T16:48:09.359 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 23 2023-11-16T16:48:09.377 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:48:09.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.572+0000 7fcf27fff700 1 -- 172.21.15.93:0/4268285038 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf20101e30 msgr2=0x7fcf201086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:09.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.572+0000 7fcf27fff700 1 --2- 172.21.15.93:0/4268285038 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf20101e30 0x7fcf201086f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009f90 tx=0x7fcf1800b330 comp rx=0 tx=0).stop 2023-11-16T16:48:09.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.572+0000 7fcf27fff700 1 -- 172.21.15.93:0/4268285038 shutdown_connections 2023-11-16T16:48:09.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.572+0000 7fcf27fff700 1 --2- 172.21.15.93:0/4268285038 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf20101e30 0x7fcf201086f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.572+0000 7fcf27fff700 1 --2- 172.21.15.93:0/4268285038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf201002b0 0x7fcf201006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.572+0000 7fcf27fff700 1 --2- 172.21.15.93:0/4268285038 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf201014a0 0x7fcf201018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.572+0000 7fcf27fff700 1 -- 172.21.15.93:0/4268285038 >> 172.21.15.93:0/4268285038 conn(0x7fcf200fb890 msgr2=0x7fcf200fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:09.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.572+0000 7fcf27fff700 1 -- 172.21.15.93:0/4268285038 shutdown_connections 2023-11-16T16:48:09.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.572+0000 7fcf27fff700 1 -- 172.21.15.93:0/4268285038 wait complete. 2023-11-16T16:48:09.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.573+0000 7fcf27fff700 1 Processor -- start 2023-11-16T16:48:09.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.573+0000 7fcf27fff700 1 -- start start 2023-11-16T16:48:09.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.573+0000 7fcf27fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf201002b0 0x7fcf201982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:09.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.573+0000 7fcf27fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf201014a0 0x7fcf20198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:09.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.573+0000 7fcf27fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf20101e30 0x7fcf2019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:09.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.573+0000 7fcf27fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2019fe50 con 0x7fcf20101e30 2023-11-16T16:48:09.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.573+0000 7fcf27fff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2019ff90 con 0x7fcf201002b0 2023-11-16T16:48:09.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.573+0000 7fcf27fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf201a0120 con 0x7fcf201014a0 2023-11-16T16:48:09.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf2659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf20101e30 0x7fcf2019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:09.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf2659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf20101e30 0x7fcf2019f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44536/0 (socket says 172.21.15.93:44536) 2023-11-16T16:48:09.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf2659c700 1 -- 172.21.15.93:0/2295643675 learned_addr learned my addr 172.21.15.93:0/2295643675 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:09.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf25d9b700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf201002b0 0x7fcf201982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:09.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf2559a700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf201014a0 0x7fcf20198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:09.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf2659c700 1 -- 172.21.15.93:0/2295643675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf201014a0 msgr2=0x7fcf20198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:09.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf2659c700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf201014a0 0x7fcf20198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf2659c700 1 -- 172.21.15.93:0/2295643675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf201002b0 msgr2=0x7fcf201982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:09.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf2659c700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf201002b0 0x7fcf201982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf2659c700 1 -- 172.21.15.93:0/2295643675 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf10009520 con 0x7fcf20101e30 2023-11-16T16:48:09.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf2659c700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf20101e30 0x7fcf2019f880 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcf18007930 tx=0x7fcf18004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:09.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf16ffd700 1 -- 172.21.15.93:0/2295643675 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1800e040 con 0x7fcf20101e30 2023-11-16T16:48:09.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf16ffd700 1 -- 172.21.15.93:0/2295643675 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf1800bda0 con 0x7fcf20101e30 2023-11-16T16:48:09.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf27fff700 1 -- 172.21.15.93:0/2295643675 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf18009c40 con 0x7fcf20101e30 2023-11-16T16:48:09.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.574+0000 7fcf16ffd700 1 -- 172.21.15.93:0/2295643675 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf180052a0 con 0x7fcf20101e30 2023-11-16T16:48:09.580 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.575+0000 7fcf27fff700 1 -- 172.21.15.93:0/2295643675 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2010bc80 con 0x7fcf20101e30 2023-11-16T16:48:09.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.576+0000 7fcf27fff700 1 -- 172.21.15.93:0/2295643675 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2010be10 con 0x7fcf20101e30 2023-11-16T16:48:09.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.578+0000 7fcf16ffd700 1 -- 172.21.15.93:0/2295643675 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fcf1801d470 con 0x7fcf20101e30 2023-11-16T16:48:09.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.579+0000 7fcf16ffd700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcf0c0a1260 0x7fcf0c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:09.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.579+0000 7fcf16ffd700 1 -- 172.21.15.93:0/2295643675 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fcf180ce440 con 0x7fcf20101e30 2023-11-16T16:48:09.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.579+0000 7fcf25d9b700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcf0c0a1260 0x7fcf0c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:09.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.580+0000 7fcf25d9b700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcf0c0a1260 0x7fcf0c0a3710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcf20101300 tx=0x7fcf1c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:09.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.582+0000 7fcf16ffd700 1 -- 172.21.15.93:0/2295643675 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcf18097690 con 0x7fcf20101e30 2023-11-16T16:48:09.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.784+0000 7fcf27fff700 1 -- 172.21.15.93:0/2295643675 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf201a0ae0 con 0x7fcf20101e30 2023-11-16T16:48:09.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.786+0000 7fcf16ffd700 1 -- 172.21.15.93:0/2295643675 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24795 (secure 0 0 0) 0x7fcf18008f30 con 0x7fcf20101e30 2023-11-16T16:48:09.788 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:09.788 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":92,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:47:53.331398+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":1,"to":2}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:48:09.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.788+0000 7fcf27fff700 1 -- 172.21.15.93:0/2295643675 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcf0c0a1260 msgr2=0x7fcf0c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:09.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.788+0000 7fcf27fff700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcf0c0a1260 0x7fcf0c0a3710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcf20101300 tx=0x7fcf1c009310 comp rx=0 tx=0).stop 2023-11-16T16:48:09.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.788+0000 7fcf27fff700 1 -- 172.21.15.93:0/2295643675 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf20101e30 msgr2=0x7fcf2019f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:09.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.788+0000 7fcf27fff700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf20101e30 0x7fcf2019f880 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcf18007930 tx=0x7fcf18004c60 comp rx=0 tx=0).stop 2023-11-16T16:48:09.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.789+0000 7fcf27fff700 1 -- 172.21.15.93:0/2295643675 shutdown_connections 2023-11-16T16:48:09.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.789+0000 7fcf27fff700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fcf0c0a1260 0x7fcf0c0a3710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.789+0000 7fcf27fff700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf20101e30 0x7fcf2019f880 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.789+0000 7fcf27fff700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf201002b0 0x7fcf201982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.789+0000 7fcf27fff700 1 --2- 172.21.15.93:0/2295643675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf201014a0 0x7fcf20198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:09.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.789+0000 7fcf27fff700 1 -- 172.21.15.93:0/2295643675 >> 172.21.15.93:0/2295643675 conn(0x7fcf200fb890 msgr2=0x7fcf20106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:09.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.789+0000 7fcf27fff700 1 -- 172.21.15.93:0/2295643675 shutdown_connections 2023-11-16T16:48:09.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:09.789+0000 7fcf27fff700 1 -- 172.21.15.93:0/2295643675 wait complete. 2023-11-16T16:48:09.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:42:13.029135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:46:35.301321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:46:36.552545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-16T16:46:39.628380+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-16T16:48:09.811 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:48:10.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.011+0000 7fa12adff700 1 -- 172.21.15.93:0/2105477201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1241014c0 msgr2=0x7fa124101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.011+0000 7fa12adff700 1 --2- 172.21.15.93:0/2105477201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1241014c0 0x7fa124101910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa110009990 tx=0x7fa110009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.011+0000 7fa12adff700 1 -- 172.21.15.93:0/2105477201 shutdown_connections 2023-11-16T16:48:10.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.011+0000 7fa12adff700 1 --2- 172.21.15.93:0/2105477201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1241002d0 0x7fa1241006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.011+0000 7fa12adff700 1 --2- 172.21.15.93:0/2105477201 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa124101e50 0x7fa124108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.011+0000 7fa12adff700 1 --2- 172.21.15.93:0/2105477201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1241014c0 0x7fa124101910 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.011+0000 7fa12adff700 1 -- 172.21.15.93:0/2105477201 >> 172.21.15.93:0/2105477201 conn(0x7fa1240fb8b0 msgr2=0x7fa1240fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:10.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.012+0000 7fa12adff700 1 -- 172.21.15.93:0/2105477201 shutdown_connections 2023-11-16T16:48:10.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.012+0000 7fa12adff700 1 -- 172.21.15.93:0/2105477201 wait complete. 2023-11-16T16:48:10.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.012+0000 7fa12adff700 1 Processor -- start 2023-11-16T16:48:10.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.013+0000 7fa12adff700 1 -- start start 2023-11-16T16:48:10.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.013+0000 7fa12adff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1241002d0 0x7fa124196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:10.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.013+0000 7fa12adff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1241014c0 0x7fa124196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:10.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.013+0000 7fa12adff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa124101e50 0x7fa12419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:10.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.013+0000 7fa12adff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12419dbb0 con 0x7fa1241002d0 2023-11-16T16:48:10.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.013+0000 7fa12adff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12419dcf0 con 0x7fa124101e50 2023-11-16T16:48:10.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.013+0000 7fa12adff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12419de80 con 0x7fa1241014c0 2023-11-16T16:48:10.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.013+0000 7fa128b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1241002d0 0x7fa124196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:10.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.013+0000 7fa128b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1241002d0 0x7fa124196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44546/0 (socket says 172.21.15.93:44546) 2023-11-16T16:48:10.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.013+0000 7fa128b9b700 1 -- 172.21.15.93:0/44633696 learned_addr learned my addr 172.21.15.93:0/44633696 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:10.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.013+0000 7fa12939c700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa124101e50 0x7fa12419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:10.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.013+0000 7fa123fff700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1241014c0 0x7fa124196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:10.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.014+0000 7fa128b9b700 1 -- 172.21.15.93:0/44633696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1241014c0 msgr2=0x7fa124196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.014+0000 7fa128b9b700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1241014c0 0x7fa124196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.014+0000 7fa128b9b700 1 -- 172.21.15.93:0/44633696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa124101e50 msgr2=0x7fa12419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.014+0000 7fa128b9b700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa124101e50 0x7fa12419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.014+0000 7fa128b9b700 1 -- 172.21.15.93:0/44633696 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1100095f0 con 0x7fa1241002d0 2023-11-16T16:48:10.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.014+0000 7fa128b9b700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1241002d0 0x7fa124196030 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa11800e970 tx=0x7fa11800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:10.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.014+0000 7fa121ffb700 1 -- 172.21.15.93:0/44633696 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118009790 con 0x7fa1241002d0 2023-11-16T16:48:10.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.014+0000 7fa12adff700 1 -- 172.21.15.93:0/44633696 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12410bc30 con 0x7fa1241002d0 2023-11-16T16:48:10.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.014+0000 7fa121ffb700 1 -- 172.21.15.93:0/44633696 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa11800eec0 con 0x7fa1241002d0 2023-11-16T16:48:10.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.014+0000 7fa121ffb700 1 -- 172.21.15.93:0/44633696 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118003cf0 con 0x7fa1241002d0 2023-11-16T16:48:10.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.014+0000 7fa12adff700 1 -- 172.21.15.93:0/44633696 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12410baa0 con 0x7fa1241002d0 2023-11-16T16:48:10.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.015+0000 7fa12adff700 1 -- 172.21.15.93:0/44633696 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa124107200 con 0x7fa1241002d0 2023-11-16T16:48:10.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.017+0000 7fa121ffb700 1 -- 172.21.15.93:0/44633696 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa11800ccd0 con 0x7fa1241002d0 2023-11-16T16:48:10.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.018+0000 7fa121ffb700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa1140a11e0 0x7fa1140a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:10.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.019+0000 7fa121ffb700 1 -- 172.21.15.93:0/44633696 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fa118014070 con 0x7fa1241002d0 2023-11-16T16:48:10.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.019+0000 7fa123fff700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa1140a11e0 0x7fa1140a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:10.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.019+0000 7fa123fff700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa1140a11e0 0x7fa1140a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b2e0 tx=0x7fa110005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:10.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.022+0000 7fa121ffb700 1 -- 172.21.15.93:0/44633696 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa1180962e0 con 0x7fa1241002d0 2023-11-16T16:48:10.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:09 smithi093 ceph-mon[97863]: pgmap v331: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-16T16:48:10.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:09 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2704583633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:48:10.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:09 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3872543111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:10.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:09 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2295643675' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:10.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.221+0000 7fa12adff700 1 -- 172.21.15.93:0/44633696 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa124067b00 con 0x7fa1241002d0 2023-11-16T16:48:10.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.222+0000 7fa121ffb700 1 -- 172.21.15.93:0/44633696 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa118099900 con 0x7fa1241002d0 2023-11-16T16:48:10.223 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-11-16T16:48:10.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.224+0000 7fa12adff700 1 -- 172.21.15.93:0/44633696 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa1140a11e0 msgr2=0x7fa1140a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.224+0000 7fa12adff700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa1140a11e0 0x7fa1140a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b2e0 tx=0x7fa110005fb0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.224+0000 7fa12adff700 1 -- 172.21.15.93:0/44633696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1241002d0 msgr2=0x7fa124196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.224+0000 7fa12adff700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1241002d0 0x7fa124196030 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa11800e970 tx=0x7fa11800ec80 comp rx=0 tx=0).stop 2023-11-16T16:48:10.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.224+0000 7fa12adff700 1 -- 172.21.15.93:0/44633696 shutdown_connections 2023-11-16T16:48:10.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.224+0000 7fa12adff700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa1140a11e0 0x7fa1140a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.224+0000 7fa12adff700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1241002d0 0x7fa124196030 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.224+0000 7fa12adff700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa124101e50 0x7fa12419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.224+0000 7fa12adff700 1 --2- 172.21.15.93:0/44633696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1241014c0 0x7fa124196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.224+0000 7fa12adff700 1 -- 172.21.15.93:0/44633696 >> 172.21.15.93:0/44633696 conn(0x7fa1240fb8b0 msgr2=0x7fa1240fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:10.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.224+0000 7fa12adff700 1 -- 172.21.15.93:0/44633696 shutdown_connections 2023-11-16T16:48:10.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.224+0000 7fa12adff700 1 -- 172.21.15.93:0/44633696 wait complete. 2023-11-16T16:48:10.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:09 smithi159 ceph-mon[131528]: pgmap v331: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-16T16:48:10.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:09 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2704583633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:48:10.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:09 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3872543111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:10.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:09 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2295643675' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:10.245 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:48:10.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:09 smithi125 ceph-mon[131076]: pgmap v331: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-16T16:48:10.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:09 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2704583633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:48:10.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:09 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3872543111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:10.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:09 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2295643675' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:10.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.444+0000 7fdc435ff700 1 -- 172.21.15.93:0/2401914548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c1002d0 msgr2=0x7fdc3c1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.444+0000 7fdc435ff700 1 --2- 172.21.15.93:0/2401914548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c1002d0 0x7fdc3c1006e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009990 tx=0x7fdc38009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.444+0000 7fdc435ff700 1 -- 172.21.15.93:0/2401914548 shutdown_connections 2023-11-16T16:48:10.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.444+0000 7fdc435ff700 1 --2- 172.21.15.93:0/2401914548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c1002d0 0x7fdc3c1006e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.444+0000 7fdc435ff700 1 --2- 172.21.15.93:0/2401914548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc3c101e50 0x7fdc3c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.444+0000 7fdc435ff700 1 --2- 172.21.15.93:0/2401914548 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc3c1014c0 0x7fdc3c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.444+0000 7fdc435ff700 1 -- 172.21.15.93:0/2401914548 >> 172.21.15.93:0/2401914548 conn(0x7fdc3c0fb8b0 msgr2=0x7fdc3c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:10.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.444+0000 7fdc435ff700 1 -- 172.21.15.93:0/2401914548 shutdown_connections 2023-11-16T16:48:10.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.444+0000 7fdc435ff700 1 -- 172.21.15.93:0/2401914548 wait complete. 2023-11-16T16:48:10.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.445+0000 7fdc435ff700 1 Processor -- start 2023-11-16T16:48:10.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.446+0000 7fdc435ff700 1 -- start start 2023-11-16T16:48:10.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.446+0000 7fdc435ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc3c1002d0 0x7fdc3c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:10.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.446+0000 7fdc435ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c1014c0 0x7fdc3c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:10.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.446+0000 7fdc435ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc3c101e50 0x7fdc3c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:10.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.446+0000 7fdc435ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c073bc0 con 0x7fdc3c1014c0 2023-11-16T16:48:10.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.446+0000 7fdc435ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c073d30 con 0x7fdc3c1002d0 2023-11-16T16:48:10.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.446+0000 7fdc435ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c1a00b0 con 0x7fdc3c101e50 2023-11-16T16:48:10.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.446+0000 7fdc40b9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c1014c0 0x7fdc3c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:10.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.446+0000 7fdc40b9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c1014c0 0x7fdc3c072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44554/0 (socket says 172.21.15.93:44554) 2023-11-16T16:48:10.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.446+0000 7fdc40b9a700 1 -- 172.21.15.93:0/2991113311 learned_addr learned my addr 172.21.15.93:0/2991113311 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:10.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.446+0000 7fdc4139b700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc3c1002d0 0x7fdc3c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:10.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.446+0000 7fdc41b9c700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc3c101e50 0x7fdc3c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:10.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.447+0000 7fdc40b9a700 1 -- 172.21.15.93:0/2991113311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc3c101e50 msgr2=0x7fdc3c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.447+0000 7fdc40b9a700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc3c101e50 0x7fdc3c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.447+0000 7fdc40b9a700 1 -- 172.21.15.93:0/2991113311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc3c1002d0 msgr2=0x7fdc3c0748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.447+0000 7fdc40b9a700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc3c1002d0 0x7fdc3c0748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.447+0000 7fdc40b9a700 1 -- 172.21.15.93:0/2991113311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc380095f0 con 0x7fdc3c1014c0 2023-11-16T16:48:10.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.447+0000 7fdc41b9c700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc3c101e50 0x7fdc3c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:10.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.447+0000 7fdc40b9a700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c1014c0 0x7fdc3c072ee0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00d980 tx=0x7fdc2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:10.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.447+0000 7fdc327fc700 1 -- 172.21.15.93:0/2991113311 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c004220 con 0x7fdc3c1014c0 2023-11-16T16:48:10.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.447+0000 7fdc327fc700 1 -- 172.21.15.93:0/2991113311 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc2c007640 con 0x7fdc3c1014c0 2023-11-16T16:48:10.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.447+0000 7fdc435ff700 1 -- 172.21.15.93:0/2991113311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c10bc30 con 0x7fdc3c1014c0 2023-11-16T16:48:10.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.447+0000 7fdc327fc700 1 -- 172.21.15.93:0/2991113311 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c005610 con 0x7fdc3c1014c0 2023-11-16T16:48:10.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.447+0000 7fdc435ff700 1 -- 172.21.15.93:0/2991113311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c10baa0 con 0x7fdc3c1014c0 2023-11-16T16:48:10.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.448+0000 7fdc435ff700 1 -- 172.21.15.93:0/2991113311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3c107200 con 0x7fdc3c1014c0 2023-11-16T16:48:10.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.450+0000 7fdc327fc700 1 -- 172.21.15.93:0/2991113311 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdc2c004750 con 0x7fdc3c1014c0 2023-11-16T16:48:10.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.451+0000 7fdc327fc700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdc280a11e0 0x7fdc280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:10.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.452+0000 7fdc327fc700 1 -- 172.21.15.93:0/2991113311 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fdc2c0cd270 con 0x7fdc3c1014c0 2023-11-16T16:48:10.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.452+0000 7fdc4139b700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdc280a11e0 0x7fdc280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:10.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.452+0000 7fdc4139b700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdc280a11e0 0x7fdc280a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800b5c0 tx=0x7fdc38005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:10.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.454+0000 7fdc327fc700 1 -- 172.21.15.93:0/2991113311 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdc2c096430 con 0x7fdc3c1014c0 2023-11-16T16:48:10.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.653+0000 7fdc435ff700 1 -- 172.21.15.93:0/2991113311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdc3c061700 con 0x7fdc3c1014c0 2023-11-16T16:48:10.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.654+0000 7fdc327fc700 1 -- 172.21.15.93:0/2991113311 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdc2c016020 con 0x7fdc3c1014c0 2023-11-16T16:48:10.656 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:48:10.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.656+0000 7fdc435ff700 1 -- 172.21.15.93:0/2991113311 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdc280a11e0 msgr2=0x7fdc280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.656+0000 7fdc435ff700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdc280a11e0 0x7fdc280a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800b5c0 tx=0x7fdc38005960 comp rx=0 tx=0).stop 2023-11-16T16:48:10.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.656+0000 7fdc435ff700 1 -- 172.21.15.93:0/2991113311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c1014c0 msgr2=0x7fdc3c072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.656+0000 7fdc435ff700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c1014c0 0x7fdc3c072ee0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00d980 tx=0x7fdc2c00dc90 comp rx=0 tx=0).stop 2023-11-16T16:48:10.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.656+0000 7fdc435ff700 1 -- 172.21.15.93:0/2991113311 shutdown_connections 2023-11-16T16:48:10.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.656+0000 7fdc435ff700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdc280a11e0 0x7fdc280a3690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.656+0000 7fdc435ff700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc3c101e50 0x7fdc3c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.656+0000 7fdc435ff700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc3c1002d0 0x7fdc3c0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.656+0000 7fdc435ff700 1 --2- 172.21.15.93:0/2991113311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c1014c0 0x7fdc3c072ee0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.656+0000 7fdc435ff700 1 -- 172.21.15.93:0/2991113311 >> 172.21.15.93:0/2991113311 conn(0x7fdc3c0fb8b0 msgr2=0x7fdc3c0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:10.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.657+0000 7fdc435ff700 1 -- 172.21.15.93:0/2991113311 shutdown_connections 2023-11-16T16:48:10.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.657+0000 7fdc435ff700 1 -- 172.21.15.93:0/2991113311 wait complete. 2023-11-16T16:48:10.677 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:48:10.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.876+0000 7f62bf9ff700 1 -- 172.21.15.93:0/2707284084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62b81036c0 msgr2=0x7f62b8105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.876+0000 7f62bf9ff700 1 --2- 172.21.15.93:0/2707284084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62b81036c0 0x7f62b8105aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62a80098f0 tx=0x7f62a8009c00 comp rx=0 tx=0).stop 2023-11-16T16:48:10.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.876+0000 7f62bf9ff700 1 -- 172.21.15.93:0/2707284084 shutdown_connections 2023-11-16T16:48:10.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.876+0000 7f62bf9ff700 1 --2- 172.21.15.93:0/2707284084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62b8105fe0 0x7f62b81083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.876+0000 7f62bf9ff700 1 --2- 172.21.15.93:0/2707284084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8100da0 0x7f62b8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.876+0000 7f62bf9ff700 1 --2- 172.21.15.93:0/2707284084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62b81036c0 0x7f62b8105aa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.876+0000 7f62bf9ff700 1 -- 172.21.15.93:0/2707284084 >> 172.21.15.93:0/2707284084 conn(0x7f62b80fa7c0 msgr2=0x7f62b80fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:10.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.876+0000 7f62bf9ff700 1 -- 172.21.15.93:0/2707284084 shutdown_connections 2023-11-16T16:48:10.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.876+0000 7f62bf9ff700 1 -- 172.21.15.93:0/2707284084 wait complete. 2023-11-16T16:48:10.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.877+0000 7f62bf9ff700 1 Processor -- start 2023-11-16T16:48:10.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.877+0000 7f62bf9ff700 1 -- start start 2023-11-16T16:48:10.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.878+0000 7f62bf9ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62b8100da0 0x7f62b8100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:10.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.878+0000 7f62bf9ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62b81036c0 0x7f62b80fed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:10.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.878+0000 7f62bf9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8105fe0 0x7f62b80ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:10.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.878+0000 7f62bf9ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b80ffa30 con 0x7f62b8100da0 2023-11-16T16:48:10.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.878+0000 7f62bf9ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b80ffba0 con 0x7f62b81036c0 2023-11-16T16:48:10.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.878+0000 7f62bf9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b81a0130 con 0x7f62b8105fe0 2023-11-16T16:48:10.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.878+0000 7f62bd79b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62b8100da0 0x7f62b8100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:10.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.878+0000 7f62bcf9a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62b81036c0 0x7f62b80fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:10.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.878+0000 7f62bd79b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62b8100da0 0x7f62b8100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44578/0 (socket says 172.21.15.93:44578) 2023-11-16T16:48:10.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.878+0000 7f62bd79b700 1 -- 172.21.15.93:0/100718677 learned_addr learned my addr 172.21.15.93:0/100718677 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:10.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.878+0000 7f62bdf9c700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8105fe0 0x7f62b80ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:10.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.879+0000 7f62bd79b700 1 -- 172.21.15.93:0/100718677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8105fe0 msgr2=0x7f62b80ff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.879+0000 7f62bd79b700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8105fe0 0x7f62b80ff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.879+0000 7f62bd79b700 1 -- 172.21.15.93:0/100718677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62b81036c0 msgr2=0x7f62b80fed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:10.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.879+0000 7f62bd79b700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62b81036c0 0x7f62b80fed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:10.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.879+0000 7f62bd79b700 1 -- 172.21.15.93:0/100718677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a8009520 con 0x7f62b8100da0 2023-11-16T16:48:10.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.879+0000 7f62bd79b700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62b8100da0 0x7f62b8100710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62b400e9c0 tx=0x7f62b400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:10.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.881+0000 7f62ae7fc700 1 -- 172.21.15.93:0/100718677 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4004a30 con 0x7f62b8100da0 2023-11-16T16:48:10.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.881+0000 7f62ae7fc700 1 -- 172.21.15.93:0/100718677 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62b400cc80 con 0x7f62b8100da0 2023-11-16T16:48:10.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.881+0000 7f62bf9ff700 1 -- 172.21.15.93:0/100718677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b810bcd0 con 0x7f62b8100da0 2023-11-16T16:48:10.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.881+0000 7f62ae7fc700 1 -- 172.21.15.93:0/100718677 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b40054e0 con 0x7f62b8100da0 2023-11-16T16:48:10.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.881+0000 7f62bf9ff700 1 -- 172.21.15.93:0/100718677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b810bb40 con 0x7f62b8100da0 2023-11-16T16:48:10.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.882+0000 7f62bf9ff700 1 -- 172.21.15.93:0/100718677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b80fc330 con 0x7f62b8100da0 2023-11-16T16:48:10.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.884+0000 7f62ae7fc700 1 -- 172.21.15.93:0/100718677 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f62b4004b90 con 0x7f62b8100da0 2023-11-16T16:48:10.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.885+0000 7f62ae7fc700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f62a40a1260 0x7f62a40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:10.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.885+0000 7f62ae7fc700 1 -- 172.21.15.93:0/100718677 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f62b4014070 con 0x7f62b8100da0 2023-11-16T16:48:10.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.886+0000 7f62bcf9a700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f62a40a1260 0x7f62a40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:10.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.886+0000 7f62bcf9a700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f62a40a1260 0x7f62a40a3710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f62a8005e90 tx=0x7f62a8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:10.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:10.889+0000 7f62ae7fc700 1 -- 172.21.15.93:0/100718677 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f62b40960b0 con 0x7f62b8100da0 2023-11-16T16:48:11.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.090+0000 7f62bf9ff700 1 -- 172.21.15.93:0/100718677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62b8061700 con 0x7f62b8100da0 2023-11-16T16:48:11.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.090+0000 7f62ae7fc700 1 -- 172.21.15.93:0/100718677 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62b40996d0 con 0x7f62b8100da0 2023-11-16T16:48:11.092 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-11-16T16:48:11.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.092+0000 7f62bf9ff700 1 -- 172.21.15.93:0/100718677 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f62a40a1260 msgr2=0x7f62a40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:11.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.092+0000 7f62bf9ff700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f62a40a1260 0x7f62a40a3710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f62a8005e90 tx=0x7f62a8019040 comp rx=0 tx=0).stop 2023-11-16T16:48:11.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.092+0000 7f62bf9ff700 1 -- 172.21.15.93:0/100718677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62b8100da0 msgr2=0x7f62b8100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:11.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.092+0000 7f62bf9ff700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62b8100da0 0x7f62b8100710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62b400e9c0 tx=0x7f62b400ecd0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.092+0000 7f62bf9ff700 1 -- 172.21.15.93:0/100718677 shutdown_connections 2023-11-16T16:48:11.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.092+0000 7f62bf9ff700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f62a40a1260 0x7f62a40a3710 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.092+0000 7f62bf9ff700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62b8100da0 0x7f62b8100710 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.092+0000 7f62bf9ff700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8105fe0 0x7f62b80ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.092+0000 7f62bf9ff700 1 --2- 172.21.15.93:0/100718677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62b81036c0 0x7f62b80fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.092+0000 7f62bf9ff700 1 -- 172.21.15.93:0/100718677 >> 172.21.15.93:0/100718677 conn(0x7f62b80fa7c0 msgr2=0x7f62b8101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:11.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.093+0000 7f62bf9ff700 1 -- 172.21.15.93:0/100718677 shutdown_connections 2023-11-16T16:48:11.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.093+0000 7f62bf9ff700 1 -- 172.21.15.93:0/100718677 wait complete. 2023-11-16T16:48:11.111 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:10 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/44633696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:48:11.111 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:10 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2991113311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:48:11.113 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-16T16:48:11.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:10 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/44633696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:48:11.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:10 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2991113311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:48:11.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.313+0000 7efdc41ff700 1 -- 172.21.15.93:0/3044164737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdbc1014c0 msgr2=0x7efdbc101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:11.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.313+0000 7efdc41ff700 1 --2- 172.21.15.93:0/3044164737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdbc1014c0 0x7efdbc101910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efdac009990 tx=0x7efdac009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.313+0000 7efdc41ff700 1 -- 172.21.15.93:0/3044164737 shutdown_connections 2023-11-16T16:48:11.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.313+0000 7efdc41ff700 1 --2- 172.21.15.93:0/3044164737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc101e50 0x7efdbc108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.313+0000 7efdc41ff700 1 --2- 172.21.15.93:0/3044164737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdbc1002d0 0x7efdbc1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.313+0000 7efdc41ff700 1 --2- 172.21.15.93:0/3044164737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdbc1014c0 0x7efdbc101910 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.313+0000 7efdc41ff700 1 -- 172.21.15.93:0/3044164737 >> 172.21.15.93:0/3044164737 conn(0x7efdbc0fb8b0 msgr2=0x7efdbc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:11.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.313+0000 7efdc41ff700 1 -- 172.21.15.93:0/3044164737 shutdown_connections 2023-11-16T16:48:11.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.313+0000 7efdc41ff700 1 -- 172.21.15.93:0/3044164737 wait complete. 2023-11-16T16:48:11.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.314+0000 7efdc41ff700 1 Processor -- start 2023-11-16T16:48:11.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.314+0000 7efdc41ff700 1 -- start start 2023-11-16T16:48:11.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc41ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc1002d0 0x7efdbc0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:11.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc41ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdbc1014c0 0x7efdbc072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:11.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc41ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdbc101e50 0x7efdbc073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:11.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc41ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc073bc0 con 0x7efdbc1014c0 2023-11-16T16:48:11.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc41ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc073d30 con 0x7efdbc101e50 2023-11-16T16:48:11.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc41ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc1a00e0 con 0x7efdbc1002d0 2023-11-16T16:48:11.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc1f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc1002d0 0x7efdbc0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:11.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc1f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc1002d0 0x7efdbc0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:58644/0 (socket says 172.21.15.93:58644) 2023-11-16T16:48:11.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc1f9b700 1 -- 172.21.15.93:0/3396382309 learned_addr learned my addr 172.21.15.93:0/3396382309 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc179a700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdbc1014c0 0x7efdbc072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc279c700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdbc101e50 0x7efdbc073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc1f9b700 1 -- 172.21.15.93:0/3396382309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdbc101e50 msgr2=0x7efdbc073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc1f9b700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdbc101e50 0x7efdbc073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc1f9b700 1 -- 172.21.15.93:0/3396382309 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdbc1014c0 msgr2=0x7efdbc072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc1f9b700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdbc1014c0 0x7efdbc072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.315+0000 7efdc1f9b700 1 -- 172.21.15.93:0/3396382309 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac0095f0 con 0x7efdbc1002d0 2023-11-16T16:48:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.316+0000 7efdc1f9b700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc1002d0 0x7efdbc0748a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efdb800e970 tx=0x7efdb800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.316+0000 7efdb2ffd700 1 -- 172.21.15.93:0/3396382309 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8009790 con 0x7efdbc1002d0 2023-11-16T16:48:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.316+0000 7efdc41ff700 1 -- 172.21.15.93:0/3396382309 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc10bc80 con 0x7efdbc1002d0 2023-11-16T16:48:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.316+0000 7efdb2ffd700 1 -- 172.21.15.93:0/3396382309 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb800c7e0 con 0x7efdbc1002d0 2023-11-16T16:48:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.316+0000 7efdb2ffd700 1 -- 172.21.15.93:0/3396382309 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8003c70 con 0x7efdbc1002d0 2023-11-16T16:48:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.316+0000 7efdc41ff700 1 -- 172.21.15.93:0/3396382309 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc10baf0 con 0x7efdbc1002d0 2023-11-16T16:48:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.317+0000 7efdc41ff700 1 -- 172.21.15.93:0/3396382309 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc1a0880 con 0x7efdbc1002d0 2023-11-16T16:48:11.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.319+0000 7efdb2ffd700 1 -- 172.21.15.93:0/3396382309 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efdb800cc50 con 0x7efdbc1002d0 2023-11-16T16:48:11.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.320+0000 7efdb2ffd700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efda80a1260 0x7efda80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:11.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.320+0000 7efdb2ffd700 1 -- 172.21.15.93:0/3396382309 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7efdb8014070 con 0x7efdbc1002d0 2023-11-16T16:48:11.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.320+0000 7efdc179a700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efda80a1260 0x7efda80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:11.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.321+0000 7efdc179a700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efda80a1260 0x7efda80a3710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efdac00b2e0 tx=0x7efdac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:11.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.324+0000 7efdb2ffd700 1 -- 172.21.15.93:0/3396382309 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efdb80961d0 con 0x7efdbc1002d0 2023-11-16T16:48:11.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:10 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/44633696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:48:11.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:10 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2991113311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:48:11.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.527+0000 7efdc41ff700 1 -- 172.21.15.93:0/3396382309 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efdbc107200 con 0x7efdbc1002d0 2023-11-16T16:48:11.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.527+0000 7efdb2ffd700 1 -- 172.21.15.93:0/3396382309 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7efdb8016330 con 0x7efdbc1002d0 2023-11-16T16:48:11.529 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:48:11.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.530+0000 7efdc41ff700 1 -- 172.21.15.93:0/3396382309 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efda80a1260 msgr2=0x7efda80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:11.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.530+0000 7efdc41ff700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efda80a1260 0x7efda80a3710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efdac00b2e0 tx=0x7efdac005fb0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.530+0000 7efdc41ff700 1 -- 172.21.15.93:0/3396382309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc1002d0 msgr2=0x7efdbc0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:11.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.530+0000 7efdc41ff700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc1002d0 0x7efdbc0748a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efdb800e970 tx=0x7efdb800ec80 comp rx=0 tx=0).stop 2023-11-16T16:48:11.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.530+0000 7efdc41ff700 1 -- 172.21.15.93:0/3396382309 shutdown_connections 2023-11-16T16:48:11.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.530+0000 7efdc41ff700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efda80a1260 0x7efda80a3710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.530+0000 7efdc41ff700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdbc101e50 0x7efdbc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.530+0000 7efdc41ff700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc1002d0 0x7efdbc0748a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.530+0000 7efdc41ff700 1 --2- 172.21.15.93:0/3396382309 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdbc1014c0 0x7efdbc072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.530+0000 7efdc41ff700 1 -- 172.21.15.93:0/3396382309 >> 172.21.15.93:0/3396382309 conn(0x7efdbc0fb8b0 msgr2=0x7efdbc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:11.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.530+0000 7efdc41ff700 1 -- 172.21.15.93:0/3396382309 shutdown_connections 2023-11-16T16:48:11.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.530+0000 7efdc41ff700 1 -- 172.21.15.93:0/3396382309 wait complete. 2023-11-16T16:48:11.551 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:48:11.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.745+0000 7f49313ff700 1 -- 172.21.15.93:0/4261113983 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f492c10c860 msgr2=0x7f492c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:11.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.745+0000 7f49313ff700 1 --2- 172.21.15.93:0/4261113983 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f492c10c860 0x7f492c10ec40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f492400b0a0 tx=0x7f492400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.745+0000 7f49313ff700 1 -- 172.21.15.93:0/4261113983 shutdown_connections 2023-11-16T16:48:11.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.745+0000 7f49313ff700 1 --2- 172.21.15.93:0/4261113983 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f492c10c860 0x7f492c10ec40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.745+0000 7f49313ff700 1 --2- 172.21.15.93:0/4261113983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f492c074b30 0x7f492c072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.745+0000 7f49313ff700 1 --2- 172.21.15.93:0/4261113983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f492c073550 0x7f492c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.745+0000 7f49313ff700 1 -- 172.21.15.93:0/4261113983 >> 172.21.15.93:0/4261113983 conn(0x7f492c0fb870 msgr2=0x7f492c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:11.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.745+0000 7f49313ff700 1 -- 172.21.15.93:0/4261113983 shutdown_connections 2023-11-16T16:48:11.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.745+0000 7f49313ff700 1 -- 172.21.15.93:0/4261113983 wait complete. 2023-11-16T16:48:11.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.747+0000 7f49313ff700 1 Processor -- start 2023-11-16T16:48:11.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.747+0000 7f49313ff700 1 -- start start 2023-11-16T16:48:11.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.747+0000 7f49313ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f492c073550 0x7f492c19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:11.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.747+0000 7f49313ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f492c074b30 0x7f492c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:11.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f49313ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f492c10c860 0x7f492c1a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:11.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f49313ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c1a4230 con 0x7f492c10c860 2023-11-16T16:48:11.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f49313ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c1a4370 con 0x7f492c073550 2023-11-16T16:48:11.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f49313ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c1a4500 con 0x7f492c074b30 2023-11-16T16:48:11.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f492affd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f492c073550 0x7f492c19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:11.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f492affd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f492c073550 0x7f492c19c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:45696/0 (socket says 172.21.15.93:45696) 2023-11-16T16:48:11.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f492affd700 1 -- 172.21.15.93:0/4272705971 learned_addr learned my addr 172.21.15.93:0/4272705971 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:11.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f492b7fe700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f492c10c860 0x7f492c1a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:11.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f492a7fc700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f492c074b30 0x7f492c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:11.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f492affd700 1 -- 172.21.15.93:0/4272705971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f492c074b30 msgr2=0x7f492c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:11.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f492affd700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f492c074b30 0x7f492c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f492affd700 1 -- 172.21.15.93:0/4272705971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f492c10c860 msgr2=0x7f492c1a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:11.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f492affd700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f492c10c860 0x7f492c1a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:11.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f492affd700 1 -- 172.21.15.93:0/4272705971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4924009d00 con 0x7f492c073550 2023-11-16T16:48:11.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f492a7fc700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f492c074b30 0x7f492c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:11.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.748+0000 7f492affd700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f492c073550 0x7f492c19c6b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f491c00e970 tx=0x7f491c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:11.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.749+0000 7f4913fff700 1 -- 172.21.15.93:0/4272705971 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c009790 con 0x7f492c073550 2023-11-16T16:48:11.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.749+0000 7f4913fff700 1 -- 172.21.15.93:0/4272705971 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f491c00eec0 con 0x7f492c073550 2023-11-16T16:48:11.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.749+0000 7f49313ff700 1 -- 172.21.15.93:0/4272705971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c112270 con 0x7f492c073550 2023-11-16T16:48:11.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.749+0000 7f4913fff700 1 -- 172.21.15.93:0/4272705971 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c005590 con 0x7f492c073550 2023-11-16T16:48:11.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.749+0000 7f49313ff700 1 -- 172.21.15.93:0/4272705971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c1120e0 con 0x7f492c073550 2023-11-16T16:48:11.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.750+0000 7f49313ff700 1 -- 172.21.15.93:0/4272705971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c1023b0 con 0x7f492c073550 2023-11-16T16:48:11.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.752+0000 7f4913fff700 1 -- 172.21.15.93:0/4272705971 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f491c00cbd0 con 0x7f492c073550 2023-11-16T16:48:11.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.753+0000 7f4913fff700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f49140a11e0 0x7f49140a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:11.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.753+0000 7f4913fff700 1 -- 172.21.15.93:0/4272705971 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f491c014070 con 0x7f492c073550 2023-11-16T16:48:11.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.753+0000 7f492a7fc700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f49140a11e0 0x7f49140a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:11.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.754+0000 7f492a7fc700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f49140a11e0 0x7f49140a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f492c074850 tx=0x7f4920009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:11.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:11.757+0000 7f4913fff700 1 -- 172.21.15.93:0/4272705971 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f491c0962d0 con 0x7f492c073550 2023-11-16T16:48:12.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.002+0000 7f49313ff700 1 -- 172.21.15.93:0/4272705971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f492c02cfe0 con 0x7f492c073550 2023-11-16T16:48:12.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.003+0000 7f4913fff700 1 -- 172.21.15.93:0/4272705971 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10811 (secure 0 0 0) 0x7f491c004ef0 con 0x7f492c073550 2023-11-16T16:48:12.005 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:12.005 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3614592201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3614592201},{"type":"v1","addr":"172.21.15.93:6839","nonce":3614592201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4068789362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4068789362},{"type":"v1","addr":"172.21.15.93:6841","nonce":4068789362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:46:37.124338+0000","modified":"2023-11-16T16:47:10.311966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24541,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/778982589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":778982589},{"type":"v1","addr":"172.21.15.159:6837","nonce":778982589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6839/149300707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":149300707},{"type":"v1","addr":"172.21.15.159:6839","nonce":149300707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/3590027328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":3590027328},{"type":"v1","addr":"172.21.15.125:6839","nonce":3590027328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:48:12.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.006+0000 7f49313ff700 1 -- 172.21.15.93:0/4272705971 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f49140a11e0 msgr2=0x7f49140a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:12.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.006+0000 7f49313ff700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f49140a11e0 0x7f49140a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f492c074850 tx=0x7f4920009260 comp rx=0 tx=0).stop 2023-11-16T16:48:12.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.006+0000 7f49313ff700 1 -- 172.21.15.93:0/4272705971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f492c073550 msgr2=0x7f492c19c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:12.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.006+0000 7f49313ff700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f492c073550 0x7f492c19c6b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f491c00e970 tx=0x7f491c00ec80 comp rx=0 tx=0).stop 2023-11-16T16:48:12.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.006+0000 7f49313ff700 1 -- 172.21.15.93:0/4272705971 shutdown_connections 2023-11-16T16:48:12.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.006+0000 7f49313ff700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f49140a11e0 0x7f49140a3690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.006+0000 7f49313ff700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f492c10c860 0x7f492c1a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.006+0000 7f49313ff700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f492c073550 0x7f492c19c6b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.006+0000 7f49313ff700 1 --2- 172.21.15.93:0/4272705971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f492c074b30 0x7f492c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.006+0000 7f49313ff700 1 -- 172.21.15.93:0/4272705971 >> 172.21.15.93:0/4272705971 conn(0x7f492c0fb870 msgr2=0x7f492c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:12.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.006+0000 7f49313ff700 1 -- 172.21.15.93:0/4272705971 shutdown_connections 2023-11-16T16:48:12.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.006+0000 7f49313ff700 1 -- 172.21.15.93:0/4272705971 wait complete. 2023-11-16T16:48:12.009 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 23 2023-11-16T16:48:12.027 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:48:12.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:11 smithi093 ceph-mon[97863]: pgmap v332: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:12.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:11 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/100718677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:48:12.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:11 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3396382309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:48:12.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.222+0000 7fe89ddff700 1 -- 172.21.15.93:0/2891989970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe898100290 msgr2=0x7fe8981006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:12.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.222+0000 7fe89ddff700 1 --2- 172.21.15.93:0/2891989970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe898100290 0x7fe8981006a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe8880098f0 tx=0x7fe888009c00 comp rx=0 tx=0).stop 2023-11-16T16:48:12.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.222+0000 7fe89ddff700 1 -- 172.21.15.93:0/2891989970 shutdown_connections 2023-11-16T16:48:12.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.222+0000 7fe89ddff700 1 --2- 172.21.15.93:0/2891989970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe898100290 0x7fe8981006a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.222+0000 7fe89ddff700 1 --2- 172.21.15.93:0/2891989970 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe898101e10 0x7fe8981086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.222+0000 7fe89ddff700 1 --2- 172.21.15.93:0/2891989970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898101480 0x7fe8981018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.222+0000 7fe89ddff700 1 -- 172.21.15.93:0/2891989970 >> 172.21.15.93:0/2891989970 conn(0x7fe8980fb870 msgr2=0x7fe8980fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:12.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.222+0000 7fe89ddff700 1 -- 172.21.15.93:0/2891989970 shutdown_connections 2023-11-16T16:48:12.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.222+0000 7fe89ddff700 1 -- 172.21.15.93:0/2891989970 wait complete. 2023-11-16T16:48:12.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.223+0000 7fe89ddff700 1 Processor -- start 2023-11-16T16:48:12.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.223+0000 7fe89ddff700 1 -- start start 2023-11-16T16:48:12.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.223+0000 7fe89ddff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898100290 0x7fe898198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:12.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.223+0000 7fe89ddff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe898101480 0x7fe898198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:12.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe89ddff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe898101e10 0x7fe89819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:12.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe89ddff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89819fdb0 con 0x7fe898101e10 2023-11-16T16:48:12.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe89ddff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89819fef0 con 0x7fe898101480 2023-11-16T16:48:12.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe89ddff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981a0080 con 0x7fe898100290 2023-11-16T16:48:12.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe897fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe898101e10 0x7fe89819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:12.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe8977fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898100290 0x7fe898198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:12.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe8977fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898100290 0x7fe898198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:58680/0 (socket says 172.21.15.93:58680) 2023-11-16T16:48:12.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe8977fe700 1 -- 172.21.15.93:0/3720675718 learned_addr learned my addr 172.21.15.93:0/3720675718 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:12.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe896ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe898101480 0x7fe898198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:12.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe897fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe898101e10 0x7fe89819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44636/0 (socket says 172.21.15.93:44636) 2023-11-16T16:48:12.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe8977fe700 1 -- 172.21.15.93:0/3720675718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe898101480 msgr2=0x7fe898198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:12.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe8977fe700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe898101480 0x7fe898198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe8977fe700 1 -- 172.21.15.93:0/3720675718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe898101e10 msgr2=0x7fe89819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:12.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe8977fe700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe898101e10 0x7fe89819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe8977fe700 1 -- 172.21.15.93:0/3720675718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe888009520 con 0x7fe898100290 2023-11-16T16:48:12.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe897fff700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe898101e10 0x7fe89819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:12.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.224+0000 7fe8977fe700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898100290 0x7fe898198230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe888005e90 tx=0x7fe88800b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:12.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.225+0000 7fe894ff9700 1 -- 172.21.15.93:0/3720675718 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88801c070 con 0x7fe898100290 2023-11-16T16:48:12.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.225+0000 7fe89ddff700 1 -- 172.21.15.93:0/3720675718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89810bc40 con 0x7fe898100290 2023-11-16T16:48:12.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.225+0000 7fe894ff9700 1 -- 172.21.15.93:0/3720675718 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe88800f7f0 con 0x7fe898100290 2023-11-16T16:48:12.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.225+0000 7fe894ff9700 1 -- 172.21.15.93:0/3720675718 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888003990 con 0x7fe898100290 2023-11-16T16:48:12.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.225+0000 7fe89ddff700 1 -- 172.21.15.93:0/3720675718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89810bab0 con 0x7fe898100290 2023-11-16T16:48:12.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.226+0000 7fe89ddff700 1 -- 172.21.15.93:0/3720675718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8981928d0 con 0x7fe898100290 2023-11-16T16:48:12.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.228+0000 7fe894ff9700 1 -- 172.21.15.93:0/3720675718 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe888020030 con 0x7fe898100290 2023-11-16T16:48:12.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.229+0000 7fe894ff9700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe8800a1260 0x7fe8800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:12.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.229+0000 7fe894ff9700 1 -- 172.21.15.93:0/3720675718 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe8880ce1f0 con 0x7fe898100290 2023-11-16T16:48:12.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.230+0000 7fe896ffd700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe8800a1260 0x7fe8800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:12.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.230+0000 7fe896ffd700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe8800a1260 0x7fe8800a3710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe88c005fd0 tx=0x7fe88c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:12.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.231+0000 7fe894ff9700 1 -- 172.21.15.93:0/3720675718 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe888097440 con 0x7fe898100290 2023-11-16T16:48:12.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:11 smithi159 ceph-mon[131528]: pgmap v332: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:12.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:11 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/100718677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:48:12.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:11 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3396382309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:48:12.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:11 smithi125 ceph-mon[131076]: pgmap v332: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:12.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:11 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/100718677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:48:12.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:11 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3396382309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:48:12.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.430+0000 7fe89ddff700 1 -- 172.21.15.93:0/3720675718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe898067a60 con 0x7fe898100290 2023-11-16T16:48:12.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.432+0000 7fe894ff9700 1 -- 172.21.15.93:0/3720675718 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24795 (secure 0 0 0) 0x7fe88809aa60 con 0x7fe898100290 2023-11-16T16:48:12.434 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:12.434 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":92,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:47:53.331398+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":1,"to":2}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:48:12.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.434+0000 7fe89ddff700 1 -- 172.21.15.93:0/3720675718 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe8800a1260 msgr2=0x7fe8800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:12.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.434+0000 7fe89ddff700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe8800a1260 0x7fe8800a3710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe88c005fd0 tx=0x7fe88c009310 comp rx=0 tx=0).stop 2023-11-16T16:48:12.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.434+0000 7fe89ddff700 1 -- 172.21.15.93:0/3720675718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898100290 msgr2=0x7fe898198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.434+0000 7fe89ddff700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898100290 0x7fe898198230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe888005e90 tx=0x7fe88800b730 comp rx=0 tx=0).stop 2023-11-16T16:48:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.434+0000 7fe89ddff700 1 -- 172.21.15.93:0/3720675718 shutdown_connections 2023-11-16T16:48:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.434+0000 7fe89ddff700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe8800a1260 0x7fe8800a3710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.434+0000 7fe89ddff700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe898101e10 0x7fe89819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.434+0000 7fe89ddff700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898100290 0x7fe898198230 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.434+0000 7fe89ddff700 1 --2- 172.21.15.93:0/3720675718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe898101480 0x7fe898198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.434+0000 7fe89ddff700 1 -- 172.21.15.93:0/3720675718 >> 172.21.15.93:0/3720675718 conn(0x7fe8980fb870 msgr2=0x7fe898106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.434+0000 7fe89ddff700 1 -- 172.21.15.93:0/3720675718 shutdown_connections 2023-11-16T16:48:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.434+0000 7fe89ddff700 1 -- 172.21.15.93:0/3720675718 wait complete. 2023-11-16T16:48:12.456 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:48:12.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.655+0000 7f8f56dff700 1 -- 172.21.15.93:0/245717522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f50101e30 msgr2=0x7f8f501086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:12.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.655+0000 7f8f56dff700 1 --2- 172.21.15.93:0/245717522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f50101e30 0x7f8f501086f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800b0a0 tx=0x7f8f4800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.655+0000 7f8f56dff700 1 -- 172.21.15.93:0/245717522 shutdown_connections 2023-11-16T16:48:12.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.655+0000 7f8f56dff700 1 --2- 172.21.15.93:0/245717522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f50101e30 0x7f8f501086f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.655+0000 7f8f56dff700 1 --2- 172.21.15.93:0/245717522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f501002b0 0x7f8f501006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.655+0000 7f8f56dff700 1 --2- 172.21.15.93:0/245717522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f501014a0 0x7f8f501018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.655+0000 7f8f56dff700 1 -- 172.21.15.93:0/245717522 >> 172.21.15.93:0/245717522 conn(0x7f8f500fb890 msgr2=0x7f8f500fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:12.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.655+0000 7f8f56dff700 1 -- 172.21.15.93:0/245717522 shutdown_connections 2023-11-16T16:48:12.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.655+0000 7f8f56dff700 1 -- 172.21.15.93:0/245717522 wait complete. 2023-11-16T16:48:12.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.656+0000 7f8f56dff700 1 Processor -- start 2023-11-16T16:48:12.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.656+0000 7f8f56dff700 1 -- start start 2023-11-16T16:48:12.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.656+0000 7f8f56dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f501002b0 0x7f8f500748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:12.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.656+0000 7f8f56dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f501014a0 0x7f8f50072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:12.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.656+0000 7f8f56dff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f50101e30 0x7f8f50073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:12.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.656+0000 7f8f56dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50073bc0 con 0x7f8f501002b0 2023-11-16T16:48:12.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.656+0000 7f8f56dff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50073d30 con 0x7f8f50101e30 2023-11-16T16:48:12.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.656+0000 7f8f56dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f501a0080 con 0x7f8f501014a0 2023-11-16T16:48:12.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.657+0000 7f8f54b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f501002b0 0x7f8f500748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:12.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.657+0000 7f8f4ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f501014a0 0x7f8f50072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:12.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.657+0000 7f8f4ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f501014a0 0x7f8f50072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:58692/0 (socket says 172.21.15.93:58692) 2023-11-16T16:48:12.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.657+0000 7f8f4ffff700 1 -- 172.21.15.93:0/1865228119 learned_addr learned my addr 172.21.15.93:0/1865228119 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:12.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.657+0000 7f8f5539c700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f50101e30 0x7f8f50073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:12.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.657+0000 7f8f4ffff700 1 -- 172.21.15.93:0/1865228119 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f50101e30 msgr2=0x7f8f50073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:12.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.657+0000 7f8f4ffff700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f50101e30 0x7f8f50073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.657+0000 7f8f4ffff700 1 -- 172.21.15.93:0/1865228119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f501002b0 msgr2=0x7f8f500748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:12.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.657+0000 7f8f4ffff700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f501002b0 0x7f8f500748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.657+0000 7f8f4ffff700 1 -- 172.21.15.93:0/1865228119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f48009d00 con 0x7f8f501014a0 2023-11-16T16:48:12.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.657+0000 7f8f54b9b700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f501002b0 0x7f8f500748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:12.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.657+0000 7f8f4ffff700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f501014a0 0x7f8f50072ee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00e9a0 tx=0x7f8f3c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:12.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.658+0000 7f8f4dffb700 1 -- 172.21.15.93:0/1865228119 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c009790 con 0x7f8f501014a0 2023-11-16T16:48:12.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.658+0000 7f8f4dffb700 1 -- 172.21.15.93:0/1865228119 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f3c010430 con 0x7f8f501014a0 2023-11-16T16:48:12.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.658+0000 7f8f56dff700 1 -- 172.21.15.93:0/1865228119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5010bc80 con 0x7f8f501014a0 2023-11-16T16:48:12.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.658+0000 7f8f4dffb700 1 -- 172.21.15.93:0/1865228119 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c018680 con 0x7f8f501014a0 2023-11-16T16:48:12.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.658+0000 7f8f56dff700 1 -- 172.21.15.93:0/1865228119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5010baf0 con 0x7f8f501014a0 2023-11-16T16:48:12.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.659+0000 7f8f56dff700 1 -- 172.21.15.93:0/1865228119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f501077d0 con 0x7f8f501014a0 2023-11-16T16:48:12.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.661+0000 7f8f4dffb700 1 -- 172.21.15.93:0/1865228119 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8f3c0105a0 con 0x7f8f501014a0 2023-11-16T16:48:12.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.662+0000 7f8f4dffb700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f400a11e0 0x7f8f400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:12.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.662+0000 7f8f4dffb700 1 -- 172.21.15.93:0/1865228119 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f8f3c014070 con 0x7f8f501014a0 2023-11-16T16:48:12.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.663+0000 7f8f54b9b700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f400a11e0 0x7f8f400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:12.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.663+0000 7f8f54b9b700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f400a11e0 0x7f8f400a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f50101300 tx=0x7f8f44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:12.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.664+0000 7f8f4dffb700 1 -- 172.21.15.93:0/1865228119 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8f3c096440 con 0x7f8f501014a0 2023-11-16T16:48:12.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.909+0000 7f8f56dff700 1 -- 172.21.15.93:0/1865228119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f5002cfe0 con 0x7f8f501014a0 2023-11-16T16:48:12.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.911+0000 7f8f4dffb700 1 -- 172.21.15.93:0/1865228119 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10811 (secure 0 0 0) 0x7f8f3c026090 con 0x7f8f501014a0 2023-11-16T16:48:12.912 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:12.912 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3614592201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3614592201},{"type":"v1","addr":"172.21.15.93:6839","nonce":3614592201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4068789362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4068789362},{"type":"v1","addr":"172.21.15.93:6841","nonce":4068789362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:46:37.124338+0000","modified":"2023-11-16T16:47:10.311966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24541,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/778982589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":778982589},{"type":"v1","addr":"172.21.15.159:6837","nonce":778982589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6839/149300707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":149300707},{"type":"v1","addr":"172.21.15.159:6839","nonce":149300707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/3590027328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":3590027328},{"type":"v1","addr":"172.21.15.125:6839","nonce":3590027328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:48:12.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.913+0000 7f8f56dff700 1 -- 172.21.15.93:0/1865228119 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f400a11e0 msgr2=0x7f8f400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.913+0000 7f8f56dff700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f400a11e0 0x7f8f400a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f50101300 tx=0x7f8f44009260 comp rx=0 tx=0).stop 2023-11-16T16:48:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.913+0000 7f8f56dff700 1 -- 172.21.15.93:0/1865228119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f501014a0 msgr2=0x7f8f50072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.913+0000 7f8f56dff700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f501014a0 0x7f8f50072ee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00e9a0 tx=0x7f8f3c00ecb0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.913+0000 7f8f56dff700 1 -- 172.21.15.93:0/1865228119 shutdown_connections 2023-11-16T16:48:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.913+0000 7f8f56dff700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f400a11e0 0x7f8f400a3690 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.913+0000 7f8f56dff700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f501002b0 0x7f8f500748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.913+0000 7f8f56dff700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f50101e30 0x7f8f50073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.913+0000 7f8f56dff700 1 --2- 172.21.15.93:0/1865228119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f501014a0 0x7f8f50072ee0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.913+0000 7f8f56dff700 1 -- 172.21.15.93:0/1865228119 >> 172.21.15.93:0/1865228119 conn(0x7f8f500fb890 msgr2=0x7f8f50104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.913+0000 7f8f56dff700 1 -- 172.21.15.93:0/1865228119 shutdown_connections 2023-11-16T16:48:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:12.913+0000 7f8f56dff700 1 -- 172.21.15.93:0/1865228119 wait complete. 2023-11-16T16:48:12.916 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 23 2023-11-16T16:48:12.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-16T16:46:37.124338+0000', 'modified': '2023-11-16T16:47:10.311966+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24566, 'mds_1': 24541, 'mds_2': 24533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24506': {'gid': 24506, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.125:6835/2117404468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6834', 'nonce': 2117404468}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 2117404468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/3072074959', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 3072074959}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3072074959}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6837/778982589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 778982589}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 778982589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24533': {'gid': 24533, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.125:6837/1957598038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6836', 'nonce': 1957598038}, {'type': 'v1', 'addr': '172.21.15.125:6837', 'nonce': 1957598038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.159:6839/149300707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 149300707}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 149300707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6839/3590027328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6838', 'nonce': 3590027328}, {'type': 'v1', 'addr': '172.21.15.125:6839', 'nonce': 3590027328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-16T16:48:12.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-16T16:48:12.934 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-11-16T16:48:13.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.129+0000 7f46c13ff700 1 -- 172.21.15.93:0/3213767614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46bc101dc0 msgr2=0x7f46bc108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.129+0000 7f46c13ff700 1 --2- 172.21.15.93:0/3213767614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46bc101dc0 0x7f46bc108680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f46b400b0a0 tx=0x7f46b400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.129+0000 7f46c13ff700 1 -- 172.21.15.93:0/3213767614 shutdown_connections 2023-11-16T16:48:13.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.129+0000 7f46c13ff700 1 --2- 172.21.15.93:0/3213767614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46bc101dc0 0x7f46bc108680 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.129+0000 7f46c13ff700 1 --2- 172.21.15.93:0/3213767614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46bc100240 0x7f46bc100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.129+0000 7f46c13ff700 1 --2- 172.21.15.93:0/3213767614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46bc101430 0x7f46bc101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.129+0000 7f46c13ff700 1 -- 172.21.15.93:0/3213767614 >> 172.21.15.93:0/3213767614 conn(0x7f46bc0fb820 msgr2=0x7f46bc0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.129+0000 7f46c13ff700 1 -- 172.21.15.93:0/3213767614 shutdown_connections 2023-11-16T16:48:13.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.129+0000 7f46c13ff700 1 -- 172.21.15.93:0/3213767614 wait complete. 2023-11-16T16:48:13.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.130+0000 7f46c13ff700 1 Processor -- start 2023-11-16T16:48:13.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.130+0000 7f46c13ff700 1 -- start start 2023-11-16T16:48:13.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46c13ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46bc100240 0x7f46bc1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46c13ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46bc101430 0x7f46bc198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46c13ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46bc101dc0 0x7f46bc19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46c13ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc19fe50 con 0x7f46bc100240 2023-11-16T16:48:13.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46c13ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc19ff90 con 0x7f46bc101dc0 2023-11-16T16:48:13.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46c13ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc1a0120 con 0x7f46bc101430 2023-11-16T16:48:13.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46ba7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46bc101430 0x7f46bc198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46baffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46bc100240 0x7f46bc1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46ba7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46bc101430 0x7f46bc198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:58706/0 (socket says 172.21.15.93:58706) 2023-11-16T16:48:13.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46ba7fc700 1 -- 172.21.15.93:0/3989935868 learned_addr learned my addr 172.21.15.93:0/3989935868 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46bb7fe700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46bc101dc0 0x7f46bc19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46baffd700 1 -- 172.21.15.93:0/3989935868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46bc101430 msgr2=0x7f46bc198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46baffd700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46bc101430 0x7f46bc198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46baffd700 1 -- 172.21.15.93:0/3989935868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46bc101dc0 msgr2=0x7f46bc19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46baffd700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46bc101dc0 0x7f46bc19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.131+0000 7f46baffd700 1 -- 172.21.15.93:0/3989935868 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b4009d00 con 0x7f46bc100240 2023-11-16T16:48:13.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.132+0000 7f46baffd700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46bc100240 0x7f46bc1982d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00e9c0 tx=0x7f46ac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.132+0000 7f46a3fff700 1 -- 172.21.15.93:0/3989935868 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac004220 con 0x7f46bc100240 2023-11-16T16:48:13.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.132+0000 7f46a3fff700 1 -- 172.21.15.93:0/3989935868 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ac004380 con 0x7f46bc100240 2023-11-16T16:48:13.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.132+0000 7f46c13ff700 1 -- 172.21.15.93:0/3989935868 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46bc10bc20 con 0x7f46bc100240 2023-11-16T16:48:13.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.132+0000 7f46a3fff700 1 -- 172.21.15.93:0/3989935868 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac003be0 con 0x7f46bc100240 2023-11-16T16:48:13.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.132+0000 7f46c13ff700 1 -- 172.21.15.93:0/3989935868 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46bc10ba90 con 0x7f46bc100240 2023-11-16T16:48:13.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.135+0000 7f46a3fff700 1 -- 172.21.15.93:0/3989935868 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f46ac003d40 con 0x7f46bc100240 2023-11-16T16:48:13.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.136+0000 7f46d4d14b80 1 Processor -- start 2023-11-16T16:48:13.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.136+0000 7f46d4d14b80 1 -- start start 2023-11-16T16:48:13.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.136+0000 7f46a3fff700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f46a40a11e0 0x7f46a40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46a3fff700 1 -- 172.21.15.93:0/3989935868 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f46ac014070 con 0x7f46bc100240 2023-11-16T16:48:13.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46d4d14b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b8bb0 0x55acc36b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46d4d14b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b9500 0x55acc36b9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46bb7fe700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f46a40a11e0 0x7f46a40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46d4d14b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36ba0c0 0x55acc36ba530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46d4d14b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc35be010 con 0x55acc36ba0c0 2023-11-16T16:48:13.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46d4d14b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc3610850 con 0x55acc36b9500 2023-11-16T16:48:13.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46d4d14b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc3609ce0 con 0x55acc36b8bb0 2023-11-16T16:48:13.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46baffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b8bb0 0x55acc36b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46baffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b8bb0 0x55acc36b8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:58710/0 (socket says 172.21.15.93:58710) 2023-11-16T16:48:13.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46baffd700 1 -- 172.21.15.93:0/1233384919 learned_addr learned my addr 172.21.15.93:0/1233384919 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46bb7fe700 1 --2- 172.21.15.93:0/1233384919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36ba0c0 0x55acc36ba530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46ba7fc700 1 --2- 172.21.15.93:0/1233384919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b9500 0x55acc36b9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46baffd700 1 -- 172.21.15.93:0/1233384919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b9500 msgr2=0x55acc36b9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46baffd700 1 --2- 172.21.15.93:0/1233384919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b9500 0x55acc36b9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46baffd700 1 -- 172.21.15.93:0/1233384919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36ba0c0 msgr2=0x55acc36ba530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46baffd700 1 --2- 172.21.15.93:0/1233384919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36ba0c0 0x55acc36ba530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46baffd700 1 -- 172.21.15.93:0/1233384919 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acc348b5e0 con 0x55acc36b8bb0 2023-11-16T16:48:13.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.138+0000 7f46bb7fe700 1 --2- 172.21.15.93:0/1233384919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36ba0c0 0x55acc36ba530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:13.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.137+0000 7f46ba7fc700 1 --2- 172.21.15.93:0/1233384919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b9500 0x55acc36b9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:13.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.138+0000 7f46bb7fe700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f46a40a11e0 0x7f46a40a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f46b400b780 tx=0x7f46b4007e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.138+0000 7f46baffd700 1 --2- 172.21.15.93:0/1233384919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b8bb0 0x55acc36b8fc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f46ac096a50 tx=0x7f46ac099a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d88f0e93cde96f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.138+0000 7f46a17fa700 1 -- 172.21.15.93:0/1233384919 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac09e010 con 0x55acc36b8bb0 2023-11-16T16:48:13.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.138+0000 7f46a17fa700 1 -- 172.21.15.93:0/1233384919 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ac0a2430 con 0x55acc36b8bb0 2023-11-16T16:48:13.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.138+0000 7f46a17fa700 1 -- 172.21.15.93:0/1233384919 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0994d0 con 0x55acc36b8bb0 2023-11-16T16:48:13.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.139+0000 7f46d4d14b80 1 -- 172.21.15.93:0/1233384919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b8bb0 msgr2=0x55acc36b8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.139+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/1233384919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b8bb0 0x55acc36b8fc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f46ac096a50 tx=0x7f46ac099a40 comp rx=0 tx=0).stop 2023-11-16T16:48:13.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.139+0000 7f46d4d14b80 1 -- 172.21.15.93:0/1233384919 shutdown_connections 2023-11-16T16:48:13.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.139+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/1233384919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36ba0c0 0x55acc36ba530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.139+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/1233384919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b8bb0 0x55acc36b8fc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.139+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/1233384919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b9500 0x55acc36b9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.139+0000 7f46d4d14b80 1 -- 172.21.15.93:0/1233384919 >> 172.21.15.93:0/1233384919 conn(0x55acc36ae310 msgr2=0x55acc36b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.139+0000 7f46d4d14b80 1 -- 172.21.15.93:0/1233384919 shutdown_connections 2023-11-16T16:48:13.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.139+0000 7f46d4d14b80 1 -- 172.21.15.93:0/1233384919 wait complete. 2023-11-16T16:48:13.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.139+0000 7f46d4d14b80 1 -- start start 2023-11-16T16:48:13.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46d4d14b80 1 Processor -- start 2023-11-16T16:48:13.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46d4d14b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 0x55acc36c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46d4d14b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b9500 0x55acc36c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46d4d14b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36ba0c0 0x55acc36c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46d4d14b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc35ab890 con 0x55acc36b8bb0 2023-11-16T16:48:13.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46d4d14b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc35ab530 con 0x55acc36ba0c0 2023-11-16T16:48:13.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46d4d14b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc348d8d0 con 0x55acc36b9500 2023-11-16T16:48:13.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46d4d14b80 10 client.0 _ll_register_callbacks cb 0x55acc35c9730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:13.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46baffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 0x55acc36c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46baffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 0x55acc36c25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44696/0 (socket says 172.21.15.93:44696) 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46baffd700 1 -- 172.21.15.93:0/3816959188 learned_addr learned my addr 172.21.15.93:0/3816959188 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46ba7fc700 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b9500 0x55acc36c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.140+0000 7f46bb7fe700 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36ba0c0 0x55acc36c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f46baffd700 1 -- 172.21.15.93:0/3816959188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b9500 msgr2=0x55acc36c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f46baffd700 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b9500 0x55acc36c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f46baffd700 1 -- 172.21.15.93:0/3816959188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36ba0c0 msgr2=0x55acc36c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f46baffd700 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36ba0c0 0x55acc36c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f46baffd700 1 -- 172.21.15.93:0/3816959188 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46bc10ba90 con 0x55acc36b8bb0 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f46ba7fc700 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b9500 0x55acc36c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f46baffd700 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 0x55acc36c25e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00f010 tx=0x7f46ac00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa5954c0146d3ec server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f469eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f469eff5700 1 -- 172.21.15.93:0/3816959188 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0b8070 con 0x55acc36b8bb0 2023-11-16T16:48:13.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f469eff5700 1 -- 172.21.15.93:0/3816959188 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f46ac0b83d0 con 0x55acc36b8bb0 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f46d4d14b80 1 -- 172.21.15.93:0/3816959188 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acc36abad0 con 0x55acc36b8bb0 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f469eff5700 1 -- 172.21.15.93:0/3816959188 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46ac05fea0 con 0x55acc36b8bb0 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f46d4d14b80 10 client.15093 fetch_fsmap learned FSMap version 23 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.141+0000 7f46d4d14b80 1 -- 172.21.15.93:0/3816959188 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55acc35d0140 con 0x55acc36b8bb0 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.142+0000 7f469eff5700 1 -- 172.21.15.93:0/3816959188 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55acc36abad0 con 0x55acc36b8bb0 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.142+0000 7f469eff5700 1 -- 172.21.15.93:0/3816959188 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46ac095dd0 con 0x55acc36b8bb0 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.142+0000 7f46d4d14b80 10 client.15093 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.142+0000 7f46d4d14b80 10 client.15093 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.142+0000 7f46d4d14b80 20 client.15093 populate_metadata read hostname 'smithi093' 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.142+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55acc36cd900 0x55acc36cca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.142+0000 7f46d4d14b80 4 client.15093 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.142+0000 7f46d4d14b80 1 -- 172.21.15.93:0/3816959188 --> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acc35cc050 con 0x55acc36cd900 2023-11-16T16:48:13.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.142+0000 7f46ba7fc700 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55acc36cd900 0x55acc36cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.143+0000 7f46ba7fc700 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55acc36cd900 0x55acc36cca90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c5a8dd680c1f934d server_cookie=f64d7a3ed7f761fe in_seq=0 out_seq=0 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.143+0000 7f469eff5700 10 client.15093 ms_handle_connect on v2:172.21.15.125:6836/1957598038 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f469eff5700 1 -- 172.21.15.93:0/3816959188 <== mds.2 v2:172.21.15.125:6836/1957598038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55acc35cc050 con 0x55acc36cd900 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f469eff5700 10 client.15093 handle_command_reply: tid=0 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f46d4d14b80 1 client.15093 shutdown 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f46d4d14b80 1 -- 172.21.15.93:0/3816959188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 msgr2=0x55acc36c25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 0x55acc36c25e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00f010 tx=0x7f46ac00cf10 comp rx=0 tx=0).stop 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f46d4d14b80 1 -- 172.21.15.93:0/3816959188 shutdown_connections 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55acc36cd900 0x55acc36cca90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 0x55acc36c25e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36ba0c0 0x55acc36c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f469eff5700 0 client.15093 ms_handle_reset on v2:172.21.15.125:6836/1957598038 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/3816959188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b9500 0x55acc36c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f46d4d14b80 1 -- 172.21.15.93:0/3816959188 >> 172.21.15.93:0/3816959188 conn(0x55acc36ae310 msgr2=0x55acc36b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f46d4d14b80 1 -- 172.21.15.93:0/3816959188 shutdown_connections 2023-11-16T16:48:13.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f46d4d14b80 1 -- 172.21.15.93:0/3816959188 wait complete. 2023-11-16T16:48:13.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.144+0000 7f46d4d14b80 20 client.15093 trim_cache size 0 max 16384 2023-11-16T16:48:13.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:12 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/4272705971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:13.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:12 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3720675718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:13.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.166+0000 7f46d4d14b80 1 Processor -- start 2023-11-16T16:48:13.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.166+0000 7f46d4d14b80 1 -- start start 2023-11-16T16:48:13.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.166+0000 7f46d4d14b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b8bb0 0x55acc36bd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.166+0000 7f46d4d14b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b9500 0x55acc36bd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.166+0000 7f46d4d14b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36ba0c0 0x55acc36c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.166+0000 7f46d4d14b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc36ba9b0 con 0x55acc36b9500 2023-11-16T16:48:13.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.166+0000 7f46d4d14b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc36adeb0 con 0x55acc36b8bb0 2023-11-16T16:48:13.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.166+0000 7f46d4d14b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc36c0dd0 con 0x55acc36ba0c0 2023-11-16T16:48:13.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46baffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b8bb0 0x55acc36bd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46baffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b8bb0 0x55acc36bd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:45766/0 (socket says 172.21.15.93:45766) 2023-11-16T16:48:13.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46ba7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b9500 0x55acc36bd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46baffd700 1 -- 172.21.15.93:0/2167218652 learned_addr learned my addr 172.21.15.93:0/2167218652 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46bb7fe700 1 --2- 172.21.15.93:0/2167218652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36ba0c0 0x55acc36c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46baffd700 1 -- 172.21.15.93:0/2167218652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36ba0c0 msgr2=0x55acc36c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46baffd700 1 --2- 172.21.15.93:0/2167218652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36ba0c0 0x55acc36c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46baffd700 1 -- 172.21.15.93:0/2167218652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b9500 msgr2=0x55acc36bd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.170 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46baffd700 1 --2- 172.21.15.93:0/2167218652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b9500 0x55acc36bd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.170 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46baffd700 1 -- 172.21.15.93:0/2167218652 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acc35cfe90 con 0x55acc36b8bb0 2023-11-16T16:48:13.170 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46baffd700 1 --2- 172.21.15.93:0/2167218652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b8bb0 0x55acc36bd070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f46ac098330 tx=0x7f46ac09a240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18e61cb4f3354683 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46a1ffb700 1 -- 172.21.15.93:0/2167218652 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac05fea0 con 0x55acc36b8bb0 2023-11-16T16:48:13.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46a1ffb700 1 -- 172.21.15.93:0/2167218652 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ac099940 con 0x55acc36b8bb0 2023-11-16T16:48:13.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.167+0000 7f46a1ffb700 1 -- 172.21.15.93:0/2167218652 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0b8040 con 0x55acc36b8bb0 2023-11-16T16:48:13.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.168+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2167218652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b8bb0 msgr2=0x55acc36bd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.168+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/2167218652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b8bb0 0x55acc36bd070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f46ac098330 tx=0x7f46ac09a240 comp rx=0 tx=0).stop 2023-11-16T16:48:13.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.168+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2167218652 shutdown_connections 2023-11-16T16:48:13.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.168+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/2167218652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36ba0c0 0x55acc36c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.168+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/2167218652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36b8bb0 0x55acc36bd070 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.168+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/2167218652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b9500 0x55acc36bd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.168+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2167218652 >> 172.21.15.93:0/2167218652 conn(0x55acc36ae310 msgr2=0x55acc36c12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.168+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2167218652 shutdown_connections 2023-11-16T16:48:13.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.168+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2167218652 wait complete. 2023-11-16T16:48:13.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.169+0000 7f46d4d14b80 1 -- start start 2023-11-16T16:48:13.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.169+0000 7f46d4d14b80 1 Processor -- start 2023-11-16T16:48:13.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.169+0000 7f46d4d14b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 0x55acc36c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.169+0000 7f46d4d14b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b9500 0x55acc36c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.169+0000 7f46d4d14b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36ba0c0 0x55acc3721190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.169+0000 7f46d4d14b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc36c9d00 con 0x55acc36b8bb0 2023-11-16T16:48:13.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.169+0000 7f46d4d14b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc36c9e70 con 0x55acc36ba0c0 2023-11-16T16:48:13.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.169+0000 7f46d4d14b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc36ca030 con 0x55acc36b9500 2023-11-16T16:48:13.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.169+0000 7f46d4d14b80 10 client.0 _ll_register_callbacks cb 0x7f46ac099ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:13.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.169+0000 7f46bb7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36ba0c0 0x55acc3721190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f46baffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 0x55acc36c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f46bb7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36ba0c0 0x55acc3721190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:45770/0 (socket says 172.21.15.93:45770) 2023-11-16T16:48:13.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f46bb7fe700 1 -- 172.21.15.93:0/2792448041 learned_addr learned my addr 172.21.15.93:0/2792448041 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f46baffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 0x55acc36c67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44722/0 (socket says 172.21.15.93:44722) 2023-11-16T16:48:13.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f46ba7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b9500 0x55acc36c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f46baffd700 1 -- 172.21.15.93:0/2792448041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b9500 msgr2=0x55acc36c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f46baffd700 1 --2- 172.21.15.93:0/2792448041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b9500 0x55acc36c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f46baffd700 1 -- 172.21.15.93:0/2792448041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36ba0c0 msgr2=0x55acc3721190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f46baffd700 1 --2- 172.21.15.93:0/2792448041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36ba0c0 0x55acc3721190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f46baffd700 1 -- 172.21.15.93:0/2792448041 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acc35d0140 con 0x55acc36b8bb0 2023-11-16T16:48:13.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f46baffd700 1 --2- 172.21.15.93:0/2792448041 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 0x55acc36c67a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55acc36ced00 tx=0x7f46ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3eda8669fe1e6e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f469eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:48:13.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f469eff5700 1 -- 172.21.15.93:0/2792448041 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac05fea0 con 0x55acc36b8bb0 2023-11-16T16:48:13.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f469eff5700 1 -- 172.21.15.93:0/2792448041 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f46ac0994d0 con 0x55acc36b8bb0 2023-11-16T16:48:13.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.170+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2792448041 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46ac0a2430 con 0x55acc36b8bb0 2023-11-16T16:48:13.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.171+0000 7f469eff5700 1 -- 172.21.15.93:0/2792448041 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46ac099200 con 0x55acc36b8bb0 2023-11-16T16:48:13.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.171+0000 7f46d4d14b80 10 client.15099 fetch_fsmap learned FSMap version 23 2023-11-16T16:48:13.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.171+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2792448041 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55acc3420c10 con 0x55acc36b8bb0 2023-11-16T16:48:13.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.171+0000 7f469eff5700 1 -- 172.21.15.93:0/2792448041 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ac0a2430 con 0x55acc36b8bb0 2023-11-16T16:48:13.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.171+0000 7f469eff5700 1 -- 172.21.15.93:0/2792448041 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46ac09af10 con 0x55acc36b8bb0 2023-11-16T16:48:13.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.171+0000 7f46d4d14b80 10 client.15099 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:48:13.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.171+0000 7f46d4d14b80 10 client.15099 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-16T16:48:13.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.171+0000 7f46d4d14b80 20 client.15099 populate_metadata read hostname 'smithi093' 2023-11-16T16:48:13.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.171+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/2792448041 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55acc3725b30 0x55acc3722790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.171+0000 7f46d4d14b80 4 client.15099 mds_command: new command op to 24533 tid=0[{"prefix": "status"}] 2023-11-16T16:48:13.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.171+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2792448041 --> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55acc35d8610 con 0x55acc3725b30 2023-11-16T16:48:13.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.172+0000 7f46ba7fc700 1 --2- 172.21.15.93:0/2792448041 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55acc3725b30 0x55acc3722790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.172+0000 7f469eff5700 10 client.15099 ms_handle_connect on v2:172.21.15.125:6836/1957598038 2023-11-16T16:48:13.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.172+0000 7f46ba7fc700 1 --2- 172.21.15.93:0/2792448041 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55acc3725b30 0x55acc3722790 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e94f2dc6f5559d9 server_cookie=26e5b522ca17aeab in_seq=0 out_seq=0 2023-11-16T16:48:13.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.172+0000 7f469eff5700 1 -- 172.21.15.93:0/2792448041 <== mds.2 v2:172.21.15.125:6836/1957598038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55acc35d8610 con 0x55acc3725b30 2023-11-16T16:48:13.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.172+0000 7f469eff5700 10 client.15099 handle_command_reply: tid=0 2023-11-16T16:48:13.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.172+0000 7f46d4d14b80 1 client.15099 shutdown 2023-11-16T16:48:13.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2792448041 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 msgr2=0x55acc36c67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/2792448041 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 0x55acc36c67a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55acc36ced00 tx=0x7f46ac000bc0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2792448041 shutdown_connections 2023-11-16T16:48:13.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/2792448041 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55acc3725b30 0x55acc3722790 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/2792448041 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55acc36b8bb0 0x55acc36c67a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/2792448041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acc36ba0c0 0x55acc3721190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f46d4d14b80 1 --2- 172.21.15.93:0/2792448041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55acc36b9500 0x55acc36c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f469eff5700 0 client.15099 ms_handle_reset on v2:172.21.15.125:6836/1957598038 2023-11-16T16:48:13.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2792448041 >> 172.21.15.93:0/2792448041 conn(0x55acc36ae310 msgr2=0x55acc36bd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2792448041 shutdown_connections 2023-11-16T16:48:13.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f46d4d14b80 1 -- 172.21.15.93:0/2792448041 wait complete. 2023-11-16T16:48:13.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f46d4d14b80 20 client.15099 trim_cache size 0 max 16384 2023-11-16T16:48:13.180 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:48:13.180 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "6a59c1a8-849e-11ee-8dbe-212e2dc638e7", 2023-11-16T16:48:13.180 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 2, 2023-11-16T16:48:13.180 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24533, 2023-11-16T16:48:13.180 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-11-16T16:48:13.180 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-11-16T16:48:13.180 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-11-16T16:48:13.180 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 63.533111935999997, 2023-11-16T16:48:13.180 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 23, 2023-11-16T16:48:13.181 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 92, 2023-11-16T16:48:13.181 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 90, 2023-11-16T16:48:13.181 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 114.699110819 2023-11-16T16:48:13.181 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:48:13.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.173+0000 7f46c13ff700 1 -- 172.21.15.93:0/3989935868 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f46a40a11e0 msgr2=0x7f46a40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.174+0000 7f46c13ff700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f46a40a11e0 0x7f46a40a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f46b400b780 tx=0x7f46b4007e30 comp rx=0 tx=0).stop 2023-11-16T16:48:13.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.174+0000 7f46c13ff700 1 -- 172.21.15.93:0/3989935868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46bc100240 msgr2=0x7f46bc1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.174+0000 7f46c13ff700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46bc100240 0x7f46bc1982d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00e9c0 tx=0x7f46ac00ecd0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.174+0000 7f46c13ff700 1 -- 172.21.15.93:0/3989935868 shutdown_connections 2023-11-16T16:48:13.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.174+0000 7f46c13ff700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f46a40a11e0 0x7f46a40a3690 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.174+0000 7f46c13ff700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46bc100240 0x7f46bc1982d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.174+0000 7f46c13ff700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46bc101dc0 0x7f46bc19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.174+0000 7f46c13ff700 1 --2- 172.21.15.93:0/3989935868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46bc101430 0x7f46bc198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.174+0000 7f46c13ff700 1 -- 172.21.15.93:0/3989935868 >> 172.21.15.93:0/3989935868 conn(0x7f46bc0fb820 msgr2=0x7f46bc106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.174+0000 7f46c13ff700 1 -- 172.21.15.93:0/3989935868 shutdown_connections 2023-11-16T16:48:13.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.174+0000 7f46c13ff700 1 -- 172.21.15.93:0/3989935868 wait complete. 2023-11-16T16:48:13.193 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-11-16T16:48:13.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:12 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/4272705971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:13.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:12 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3720675718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:13.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:12 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/4272705971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:13.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:12 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3720675718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:13.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.389+0000 7fe98b1ff700 1 -- 172.21.15.93:0/3129759509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe984101db0 msgr2=0x7fe984108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.389+0000 7fe98b1ff700 1 --2- 172.21.15.93:0/3129759509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe984101db0 0x7fe984108670 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00b0a0 tx=0x7fe97c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.389+0000 7fe98b1ff700 1 -- 172.21.15.93:0/3129759509 shutdown_connections 2023-11-16T16:48:13.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.389+0000 7fe98b1ff700 1 --2- 172.21.15.93:0/3129759509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe984101db0 0x7fe984108670 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.389+0000 7fe98b1ff700 1 --2- 172.21.15.93:0/3129759509 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe984100230 0x7fe984100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.389+0000 7fe98b1ff700 1 --2- 172.21.15.93:0/3129759509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe984101420 0x7fe984101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.389+0000 7fe98b1ff700 1 -- 172.21.15.93:0/3129759509 >> 172.21.15.93:0/3129759509 conn(0x7fe9840fb810 msgr2=0x7fe9840fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.389+0000 7fe98b1ff700 1 -- 172.21.15.93:0/3129759509 shutdown_connections 2023-11-16T16:48:13.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.389+0000 7fe98b1ff700 1 -- 172.21.15.93:0/3129759509 wait complete. 2023-11-16T16:48:13.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.390+0000 7fe98b1ff700 1 Processor -- start 2023-11-16T16:48:13.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.390+0000 7fe98b1ff700 1 -- start start 2023-11-16T16:48:13.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.390+0000 7fe98b1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe984100230 0x7fe9840748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.390+0000 7fe98b1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe984101420 0x7fe984072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.390+0000 7fe98b1ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe984101db0 0x7fe984073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.390+0000 7fe98b1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe984073bc0 con 0x7fe984100230 2023-11-16T16:48:13.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.390+0000 7fe98b1ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe984073d30 con 0x7fe984101db0 2023-11-16T16:48:13.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.390+0000 7fe98b1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9841a00f0 con 0x7fe984101420 2023-11-16T16:48:13.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe988f9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe984100230 0x7fe9840748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe988f9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe984100230 0x7fe9840748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44736/0 (socket says 172.21.15.93:44736) 2023-11-16T16:48:13.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe988f9b700 1 -- 172.21.15.93:0/1936336069 learned_addr learned my addr 172.21.15.93:0/1936336069 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe983fff700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe984101420 0x7fe984072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe98979c700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe984101db0 0x7fe984073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe988f9b700 1 -- 172.21.15.93:0/1936336069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe984101420 msgr2=0x7fe984072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe988f9b700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe984101420 0x7fe984072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe988f9b700 1 -- 172.21.15.93:0/1936336069 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe984101db0 msgr2=0x7fe984073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe988f9b700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe984101db0 0x7fe984073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe988f9b700 1 -- 172.21.15.93:0/1936336069 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c009d00 con 0x7fe984100230 2023-11-16T16:48:13.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe983fff700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe984101420 0x7fe984072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:13.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe98979c700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe984101db0 0x7fe984073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:13.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe988f9b700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe984100230 0x7fe9840748a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe97800e970 tx=0x7fe97800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.391+0000 7fe981ffb700 1 -- 172.21.15.93:0/1936336069 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978009790 con 0x7fe984100230 2023-11-16T16:48:13.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.392+0000 7fe98b1ff700 1 -- 172.21.15.93:0/1936336069 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98410bc00 con 0x7fe984100230 2023-11-16T16:48:13.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.392+0000 7fe981ffb700 1 -- 172.21.15.93:0/1936336069 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9780044e0 con 0x7fe984100230 2023-11-16T16:48:13.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.392+0000 7fe981ffb700 1 -- 172.21.15.93:0/1936336069 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978003e30 con 0x7fe984100230 2023-11-16T16:48:13.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.392+0000 7fe98b1ff700 1 -- 172.21.15.93:0/1936336069 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98410ba70 con 0x7fe984100230 2023-11-16T16:48:13.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.395+0000 7fe981ffb700 1 -- 172.21.15.93:0/1936336069 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe978004000 con 0x7fe984100230 2023-11-16T16:48:13.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.396+0000 7fe99eb59b80 1 Processor -- start 2023-11-16T16:48:13.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.396+0000 7fe99eb59b80 1 -- start start 2023-11-16T16:48:13.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.396+0000 7fe981ffb700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe9740a1260 0x7fe9740a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.396+0000 7fe981ffb700 1 -- 172.21.15.93:0/1936336069 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe978014070 con 0x7fe984100230 2023-11-16T16:48:13.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.396+0000 7fe99eb59b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 0x5567b1dd6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.396+0000 7fe99eb59b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd7500 0x5567b1dd7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.396+0000 7fe99eb59b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd80c0 0x5567b1dd8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.396+0000 7fe99eb59b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b1cdc010 con 0x5567b1dd6bb0 2023-11-16T16:48:13.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.396+0000 7fe99eb59b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b1d2e850 con 0x5567b1dd80c0 2023-11-16T16:48:13.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.396+0000 7fe99eb59b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b1d27ce0 con 0x5567b1dd7500 2023-11-16T16:48:13.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe98979c700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe9740a1260 0x7fe9740a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe988f9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 0x5567b1dd6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe988f9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 0x5567b1dd6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44738/0 (socket says 172.21.15.93:44738) 2023-11-16T16:48:13.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe988f9b700 1 -- 172.21.15.93:0/266235648 learned_addr learned my addr 172.21.15.93:0/266235648 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe983fff700 1 --2- 172.21.15.93:0/266235648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd7500 0x5567b1dd7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe988f9b700 1 -- 172.21.15.93:0/266235648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd7500 msgr2=0x5567b1dd7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe988f9b700 1 --2- 172.21.15.93:0/266235648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd7500 0x5567b1dd7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe988f9b700 1 -- 172.21.15.93:0/266235648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd80c0 msgr2=0x5567b1dd8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe988f9b700 1 --2- 172.21.15.93:0/266235648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd80c0 0x5567b1dd8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe988f9b700 1 -- 172.21.15.93:0/266235648 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567b1ba95e0 con 0x5567b1dd6bb0 2023-11-16T16:48:13.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe988f9b700 1 --2- 172.21.15.93:0/266235648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 0x5567b1dd6fc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe978096aa0 tx=0x7fe978099bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4aecbcc4d0fa8f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe98979c700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe9740a1260 0x7fe9740a3710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe984074f00 tx=0x7fe97c009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe96effd700 1 -- 172.21.15.93:0/266235648 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780b1070 con 0x5567b1dd6bb0 2023-11-16T16:48:13.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe96effd700 1 -- 172.21.15.93:0/266235648 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe97809a030 con 0x5567b1dd6bb0 2023-11-16T16:48:13.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.397+0000 7fe96effd700 1 -- 172.21.15.93:0/266235648 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780983d0 con 0x5567b1dd6bb0 2023-11-16T16:48:13.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.398+0000 7fe99eb59b80 1 -- 172.21.15.93:0/266235648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 msgr2=0x5567b1dd6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.398+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/266235648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 0x5567b1dd6fc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe978096aa0 tx=0x7fe978099bf0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.398+0000 7fe99eb59b80 1 -- 172.21.15.93:0/266235648 shutdown_connections 2023-11-16T16:48:13.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.398+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/266235648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 0x5567b1dd6fc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.398+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/266235648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd80c0 0x5567b1dd8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.398+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/266235648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd7500 0x5567b1dd7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.398+0000 7fe99eb59b80 1 -- 172.21.15.93:0/266235648 >> 172.21.15.93:0/266235648 conn(0x5567b1dcc310 msgr2=0x5567b1dd1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.398+0000 7fe99eb59b80 1 -- 172.21.15.93:0/266235648 shutdown_connections 2023-11-16T16:48:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.398+0000 7fe99eb59b80 1 -- 172.21.15.93:0/266235648 wait complete. 2023-11-16T16:48:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.399+0000 7fe99eb59b80 1 -- start start 2023-11-16T16:48:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.399+0000 7fe99eb59b80 1 Processor -- start 2023-11-16T16:48:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.399+0000 7fe99eb59b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 0x5567b1de05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.399+0000 7fe99eb59b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1de0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.399+0000 7fe99eb59b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd80c0 0x5567b1de7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.399+0000 7fe99eb59b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b1cc9890 con 0x5567b1dd6bb0 2023-11-16T16:48:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.399+0000 7fe99eb59b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b1cc9530 con 0x5567b1dd7500 2023-11-16T16:48:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.399+0000 7fe99eb59b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b1bab8d0 con 0x5567b1dd80c0 2023-11-16T16:48:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe99eb59b80 10 client.0 _ll_register_callbacks cb 0x5567b1ce7730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:13.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe988f9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 0x5567b1de05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe988f9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 0x5567b1de05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44746/0 (socket says 172.21.15.93:44746) 2023-11-16T16:48:13.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe988f9b700 1 -- 172.21.15.93:0/1785070304 learned_addr learned my addr 172.21.15.93:0/1785070304 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe98979c700 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd80c0 0x5567b1de7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe983fff700 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1de0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe988f9b700 1 -- 172.21.15.93:0/1785070304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd80c0 msgr2=0x5567b1de7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe988f9b700 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd80c0 0x5567b1de7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe988f9b700 1 -- 172.21.15.93:0/1785070304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 msgr2=0x5567b1de0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe988f9b700 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1de0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe988f9b700 1 -- 172.21.15.93:0/1785070304 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98410ba70 con 0x5567b1dd6bb0 2023-11-16T16:48:13.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe988f9b700 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 0x5567b1de05b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe978098380 tx=0x7fe978004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f37f748a9f3f7e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe963fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:48:13.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe963fff700 1 -- 172.21.15.93:0/1785070304 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780b6070 con 0x5567b1dd6bb0 2023-11-16T16:48:13.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe963fff700 1 -- 172.21.15.93:0/1785070304 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe9780a2480 con 0x5567b1dd6bb0 2023-11-16T16:48:13.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.400+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1785070304 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567b1dc9ad0 con 0x5567b1dd6bb0 2023-11-16T16:48:13.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.401+0000 7fe963fff700 1 -- 172.21.15.93:0/1785070304 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9780b1070 con 0x5567b1dd6bb0 2023-11-16T16:48:13.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.401+0000 7fe99eb59b80 10 client.15111 fetch_fsmap learned FSMap version 23 2023-11-16T16:48:13.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.401+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1785070304 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5567b1cee140 con 0x5567b1dd6bb0 2023-11-16T16:48:13.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.401+0000 7fe963fff700 1 -- 172.21.15.93:0/1785070304 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9780ca020 con 0x5567b1dd6bb0 2023-11-16T16:48:13.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.401+0000 7fe963fff700 1 -- 172.21.15.93:0/1785070304 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe978099640 con 0x5567b1dd6bb0 2023-11-16T16:48:13.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.401+0000 7fe99eb59b80 10 client.15111 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:48:13.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.401+0000 7fe99eb59b80 10 client.15111 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-16T16:48:13.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.401+0000 7fe99eb59b80 20 client.15111 populate_metadata read hostname 'smithi093' 2023-11-16T16:48:13.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.401+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x5567b1deb4e0 0x5567b1deaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.401+0000 7fe99eb59b80 4 client.15111 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.401+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1785070304 --> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567b1cea050 con 0x5567b1deb4e0 2023-11-16T16:48:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.402+0000 7fe983fff700 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x5567b1deb4e0 0x5567b1deaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.402+0000 7fe963fff700 10 client.15111 ms_handle_connect on v2:172.21.15.159:6838/149300707 2023-11-16T16:48:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.402+0000 7fe983fff700 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x5567b1deb4e0 0x5567b1deaa60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37266d751b1f3eb0 server_cookie=f761bdb386578046 in_seq=0 out_seq=0 2023-11-16T16:48:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.403+0000 7fe963fff700 1 -- 172.21.15.93:0/1785070304 <== mds.1 v2:172.21.15.159:6838/149300707 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5567b1cea050 con 0x5567b1deb4e0 2023-11-16T16:48:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.403+0000 7fe963fff700 10 client.15111 handle_command_reply: tid=0 2023-11-16T16:48:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.403+0000 7fe99eb59b80 1 client.15111 shutdown 2023-11-16T16:48:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.403+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1785070304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 msgr2=0x5567b1de05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.403+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 0x5567b1de05b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe978098380 tx=0x7fe978004400 comp rx=0 tx=0).stop 2023-11-16T16:48:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.403+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1785070304 shutdown_connections 2023-11-16T16:48:13.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.403+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x5567b1deb4e0 0x5567b1deaa60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.403+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd6bb0 0x5567b1de05b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.403+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd80c0 0x5567b1de7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.403+0000 7fe963fff700 0 client.15111 ms_handle_reset on v2:172.21.15.159:6838/149300707 2023-11-16T16:48:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.403+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/1785070304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1de0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.403+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1785070304 >> 172.21.15.93:0/1785070304 conn(0x5567b1dcc310 msgr2=0x5567b1dd0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.404+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1785070304 shutdown_connections 2023-11-16T16:48:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.404+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1785070304 wait complete. 2023-11-16T16:48:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.404+0000 7fe99eb59b80 20 client.15111 trim_cache size 0 max 16384 2023-11-16T16:48:13.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe99eb59b80 1 Processor -- start 2023-11-16T16:48:13.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe99eb59b80 1 -- start start 2023-11-16T16:48:13.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe99eb59b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 0x5567b1dde4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe99eb59b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1ddea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe99eb59b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 0x5567b1de7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe99eb59b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b1cea050 con 0x5567b1dd80c0 2023-11-16T16:48:13.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe99eb59b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b1cd1730 con 0x5567b1dd7500 2023-11-16T16:48:13.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe99eb59b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b1c33ed0 con 0x5567b1dd6bb0 2023-11-16T16:48:13.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe98979c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 0x5567b1de7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe988f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 0x5567b1dde4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe98979c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 0x5567b1de7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44750/0 (socket says 172.21.15.93:44750) 2023-11-16T16:48:13.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe988f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 0x5567b1dde4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:58782/0 (socket says 172.21.15.93:58782) 2023-11-16T16:48:13.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe98979c700 1 -- 172.21.15.93:0/3518979859 learned_addr learned my addr 172.21.15.93:0/3518979859 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.422+0000 7fe983fff700 1 --2- 172.21.15.93:0/3518979859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1ddea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.423+0000 7fe988f9b700 1 -- 172.21.15.93:0/3518979859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 msgr2=0x5567b1ddea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.423+0000 7fe988f9b700 1 --2- 172.21.15.93:0/3518979859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1ddea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.423+0000 7fe988f9b700 1 -- 172.21.15.93:0/3518979859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 msgr2=0x5567b1de7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.423+0000 7fe988f9b700 1 --2- 172.21.15.93:0/3518979859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 0x5567b1de7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.423+0000 7fe988f9b700 1 -- 172.21.15.93:0/3518979859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567b1cede90 con 0x5567b1dd6bb0 2023-11-16T16:48:13.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.423+0000 7fe98979c700 1 --2- 172.21.15.93:0/3518979859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 0x5567b1de7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:13.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.423+0000 7fe983fff700 1 --2- 172.21.15.93:0/3518979859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1ddea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.423+0000 7fe988f9b700 1 --2- 172.21.15.93:0/3518979859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 0x5567b1dde4e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9780a2090 tx=0x7fe9780985e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87764f3432bcd786 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.423+0000 7fe96f7fe700 1 -- 172.21.15.93:0/3518979859 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780b1020 con 0x5567b1dd6bb0 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.423+0000 7fe96f7fe700 1 -- 172.21.15.93:0/3518979859 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9780ca040 con 0x5567b1dd6bb0 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.423+0000 7fe96f7fe700 1 -- 172.21.15.93:0/3518979859 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780ca2b0 con 0x5567b1dd6bb0 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.424+0000 7fe99eb59b80 1 -- 172.21.15.93:0/3518979859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 msgr2=0x5567b1dde4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.424+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/3518979859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 0x5567b1dde4e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9780a2090 tx=0x7fe9780985e0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.424+0000 7fe99eb59b80 1 -- 172.21.15.93:0/3518979859 shutdown_connections 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.424+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/3518979859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 0x5567b1de7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.424+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/3518979859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 0x5567b1dde4e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.424+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/3518979859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1ddea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.424+0000 7fe99eb59b80 1 -- 172.21.15.93:0/3518979859 >> 172.21.15.93:0/3518979859 conn(0x5567b1dcc310 msgr2=0x5567b1ddf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.424+0000 7fe99eb59b80 1 -- 172.21.15.93:0/3518979859 shutdown_connections 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.424+0000 7fe99eb59b80 1 -- 172.21.15.93:0/3518979859 wait complete. 2023-11-16T16:48:13.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.424+0000 7fe99eb59b80 1 -- start start 2023-11-16T16:48:13.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.424+0000 7fe99eb59b80 1 Processor -- start 2023-11-16T16:48:13.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe99eb59b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 0x5567b1de47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe99eb59b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1de4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe99eb59b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 0x5567b1e3f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe99eb59b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b1de5260 con 0x5567b1dd80c0 2023-11-16T16:48:13.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe99eb59b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b1de53d0 con 0x5567b1dd7500 2023-11-16T16:48:13.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe99eb59b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567b1e3f8a0 con 0x5567b1dd6bb0 2023-11-16T16:48:13.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe99eb59b80 10 client.0 _ll_register_callbacks cb 0x7fe970009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:13.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe983fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1de4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe983fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1de4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:45824/0 (socket says 172.21.15.93:45824) 2023-11-16T16:48:13.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe983fff700 1 -- 172.21.15.93:0/1195728723 learned_addr learned my addr 172.21.15.93:0/1195728723 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe988f9b700 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 0x5567b1de47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe98979c700 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 0x5567b1e3f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe983fff700 1 -- 172.21.15.93:0/1195728723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 msgr2=0x5567b1de47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe983fff700 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 0x5567b1de47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe983fff700 1 -- 172.21.15.93:0/1195728723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 msgr2=0x5567b1e3f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe983fff700 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 0x5567b1e3f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe983fff700 1 -- 172.21.15.93:0/1195728723 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97c025040 con 0x5567b1dd7500 2023-11-16T16:48:13.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe98979c700 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 0x5567b1e3f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:13.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.425+0000 7fe988f9b700 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 0x5567b1de47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:13.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.426+0000 7fe983fff700 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1de4d20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x5567b1de80e0 tx=0x7fe970004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94135c2db5c692bb server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.426+0000 7fe963fff700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:48:13.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.426+0000 7fe963fff700 1 -- 172.21.15.93:0/1195728723 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970017040 con 0x5567b1dd7500 2023-11-16T16:48:13.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.426+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1195728723 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe97809a030 con 0x5567b1dd7500 2023-11-16T16:48:13.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.426+0000 7fe963fff700 1 -- 172.21.15.93:0/1195728723 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe97001ede0 con 0x5567b1dd7500 2023-11-16T16:48:13.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.426+0000 7fe963fff700 1 -- 172.21.15.93:0/1195728723 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe970003680 con 0x5567b1dd7500 2023-11-16T16:48:13.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.426+0000 7fe99eb59b80 10 client.25157 fetch_fsmap learned FSMap version 23 2023-11-16T16:48:13.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.426+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1195728723 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5567b1cee140 con 0x5567b1dd7500 2023-11-16T16:48:13.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.426+0000 7fe963fff700 1 -- 172.21.15.93:0/1195728723 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe97809a030 con 0x5567b1dd7500 2023-11-16T16:48:13.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.427+0000 7fe963fff700 1 -- 172.21.15.93:0/1195728723 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe970012940 con 0x5567b1dd7500 2023-11-16T16:48:13.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.427+0000 7fe99eb59b80 10 client.25157 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:48:13.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.427+0000 7fe99eb59b80 10 client.25157 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-16T16:48:13.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.427+0000 7fe99eb59b80 20 client.25157 populate_metadata read hostname 'smithi093' 2023-11-16T16:48:13.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.427+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x5567b1e44770 0x5567b1e410e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.427+0000 7fe99eb59b80 4 client.25157 mds_command: new command op to 24541 tid=0[{"prefix": "status"}] 2023-11-16T16:48:13.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.427+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1195728723 --> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5567b1cf6610 con 0x5567b1e44770 2023-11-16T16:48:13.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.427+0000 7fe988f9b700 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x5567b1e44770 0x5567b1e410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe988f9b700 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x5567b1e44770 0x5567b1e410e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=678a0d8492d46840 server_cookie=577d67e6766c5c78 in_seq=0 out_seq=0 2023-11-16T16:48:13.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe963fff700 10 client.25157 ms_handle_connect on v2:172.21.15.159:6838/149300707 2023-11-16T16:48:13.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe963fff700 1 -- 172.21.15.93:0/1195728723 <== mds.1 v2:172.21.15.159:6838/149300707 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5567b1cf6610 con 0x5567b1e44770 2023-11-16T16:48:13.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe963fff700 10 client.25157 handle_command_reply: tid=0 2023-11-16T16:48:13.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe99eb59b80 1 client.25157 shutdown 2023-11-16T16:48:13.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1195728723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 msgr2=0x5567b1de4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1de4d20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x5567b1de80e0 tx=0x7fe970004040 comp rx=0 tx=0).stop 2023-11-16T16:48:13.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1195728723 shutdown_connections 2023-11-16T16:48:13.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x5567b1e44770 0x5567b1e410e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5567b1dd80c0 0x5567b1e3f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b1dd6bb0 0x5567b1de47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe963fff700 0 client.25157 ms_handle_reset on v2:172.21.15.159:6838/149300707 2023-11-16T16:48:13.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe99eb59b80 1 --2- 172.21.15.93:0/1195728723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5567b1dd7500 0x5567b1de4d20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1195728723 >> 172.21.15.93:0/1195728723 conn(0x5567b1dcc310 msgr2=0x5567b1ddb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1195728723 shutdown_connections 2023-11-16T16:48:13.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe99eb59b80 1 -- 172.21.15.93:0/1195728723 wait complete. 2023-11-16T16:48:13.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.428+0000 7fe99eb59b80 20 client.25157 trim_cache size 0 max 16384 2023-11-16T16:48:13.436 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:48:13.437 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "6a59c1a8-849e-11ee-8dbe-212e2dc638e7", 2023-11-16T16:48:13.437 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 1, 2023-11-16T16:48:13.437 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24541, 2023-11-16T16:48:13.437 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-11-16T16:48:13.437 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-11-16T16:48:13.437 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-11-16T16:48:13.437 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 64.795330266999997, 2023-11-16T16:48:13.437 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 23, 2023-11-16T16:48:13.437 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 92, 2023-11-16T16:48:13.437 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 90, 2023-11-16T16:48:13.437 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 112.68323930299999 2023-11-16T16:48:13.437 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:48:13.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.429+0000 7fe98b1ff700 1 -- 172.21.15.93:0/1936336069 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe9740a1260 msgr2=0x7fe9740a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.429+0000 7fe98b1ff700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe9740a1260 0x7fe9740a3710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe984074f00 tx=0x7fe97c009590 comp rx=0 tx=0).stop 2023-11-16T16:48:13.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.429+0000 7fe98b1ff700 1 -- 172.21.15.93:0/1936336069 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe984100230 msgr2=0x7fe9840748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.429+0000 7fe98b1ff700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe984100230 0x7fe9840748a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe97800e970 tx=0x7fe97800ed30 comp rx=0 tx=0).stop 2023-11-16T16:48:13.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.429+0000 7fe98b1ff700 1 -- 172.21.15.93:0/1936336069 shutdown_connections 2023-11-16T16:48:13.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.429+0000 7fe98b1ff700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe9740a1260 0x7fe9740a3710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.429+0000 7fe98b1ff700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe984100230 0x7fe9840748a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.429+0000 7fe98b1ff700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe984101db0 0x7fe984073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.429+0000 7fe98b1ff700 1 --2- 172.21.15.93:0/1936336069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe984101420 0x7fe984072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.429+0000 7fe98b1ff700 1 -- 172.21.15.93:0/1936336069 >> 172.21.15.93:0/1936336069 conn(0x7fe9840fb810 msgr2=0x7fe9840fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.429+0000 7fe98b1ff700 1 -- 172.21.15.93:0/1936336069 shutdown_connections 2023-11-16T16:48:13.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.429+0000 7fe98b1ff700 1 -- 172.21.15.93:0/1936336069 wait complete. 2023-11-16T16:48:13.449 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-16T16:48:13.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.647+0000 7f14b8dff700 1 -- 172.21.15.93:0/3666211973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14b4101dc0 msgr2=0x7f14b4108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.647+0000 7f14b8dff700 1 --2- 172.21.15.93:0/3666211973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14b4101dc0 0x7f14b4108680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00b0a0 tx=0x7f14ac00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.647+0000 7f14b8dff700 1 -- 172.21.15.93:0/3666211973 shutdown_connections 2023-11-16T16:48:13.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.647+0000 7f14b8dff700 1 --2- 172.21.15.93:0/3666211973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14b4101dc0 0x7f14b4108680 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.647+0000 7f14b8dff700 1 --2- 172.21.15.93:0/3666211973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14b4100240 0x7f14b4100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.647+0000 7f14b8dff700 1 --2- 172.21.15.93:0/3666211973 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b4101430 0x7f14b4101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.647+0000 7f14b8dff700 1 -- 172.21.15.93:0/3666211973 >> 172.21.15.93:0/3666211973 conn(0x7f14b40fb820 msgr2=0x7f14b40fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.647+0000 7f14b8dff700 1 -- 172.21.15.93:0/3666211973 shutdown_connections 2023-11-16T16:48:13.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.647+0000 7f14b8dff700 1 -- 172.21.15.93:0/3666211973 wait complete. 2023-11-16T16:48:13.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.648+0000 7f14b8dff700 1 Processor -- start 2023-11-16T16:48:13.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.648+0000 7f14b8dff700 1 -- start start 2023-11-16T16:48:13.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.648+0000 7f14b8dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14b4100240 0x7f14b41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b8dff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b4101430 0x7f14b4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b8dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14b4101dc0 0x7f14b419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b8dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b419fe70 con 0x7f14b4100240 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b8dff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b419ffb0 con 0x7f14b4101430 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b8dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b41a0140 con 0x7f14b4101dc0 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14b4100240 0x7f14b41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14b4100240 0x7f14b41982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44782/0 (socket says 172.21.15.93:44782) 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b259c700 1 -- 172.21.15.93:0/1776264245 learned_addr learned my addr 172.21.15.93:0/1776264245 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b1d9b700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b4101430 0x7f14b4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b2d9d700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14b4101dc0 0x7f14b419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b259c700 1 -- 172.21.15.93:0/1776264245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14b4101dc0 msgr2=0x7f14b419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b259c700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14b4101dc0 0x7f14b419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b259c700 1 -- 172.21.15.93:0/1776264245 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b4101430 msgr2=0x7f14b4198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b259c700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b4101430 0x7f14b4198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b259c700 1 -- 172.21.15.93:0/1776264245 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac009d00 con 0x7f14b4100240 2023-11-16T16:48:13.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.649+0000 7f14b259c700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14b4100240 0x7f14b41982f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14a400e970 tx=0x7f14a400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.650+0000 7f14a37fe700 1 -- 172.21.15.93:0/1776264245 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4009790 con 0x7f14b4100240 2023-11-16T16:48:13.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.650+0000 7f14a37fe700 1 -- 172.21.15.93:0/1776264245 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14a400c7e0 con 0x7f14b4100240 2023-11-16T16:48:13.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.650+0000 7f14a37fe700 1 -- 172.21.15.93:0/1776264245 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4003c70 con 0x7f14b4100240 2023-11-16T16:48:13.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.650+0000 7f14b8dff700 1 -- 172.21.15.93:0/1776264245 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b410bc20 con 0x7f14b4100240 2023-11-16T16:48:13.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.650+0000 7f14b8dff700 1 -- 172.21.15.93:0/1776264245 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b410ba90 con 0x7f14b4100240 2023-11-16T16:48:13.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.654+0000 7f14a37fe700 1 -- 172.21.15.93:0/1776264245 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f14a400cc50 con 0x7f14b4100240 2023-11-16T16:48:13.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.655+0000 7f14cc70cb80 1 Processor -- start 2023-11-16T16:48:13.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.655+0000 7f14cc70cb80 1 -- start start 2023-11-16T16:48:13.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14cc70cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da189fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14cc70cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18a500 0x5571da18aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14cc70cb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18b0c0 0x5571da18b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14cc70cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571da08f010 con 0x5571da189bb0 2023-11-16T16:48:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14cc70cb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571da0e1850 con 0x5571da18b0c0 2023-11-16T16:48:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14cc70cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571da0dace0 con 0x5571da18a500 2023-11-16T16:48:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14b2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da189fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14b1d9b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18b0c0 0x5571da18b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14b2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da189fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44790/0 (socket says 172.21.15.93:44790) 2023-11-16T16:48:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14b2d9d700 1 -- 172.21.15.93:0/134970947 learned_addr learned my addr 172.21.15.93:0/134970947 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14a37fe700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f149c0a1260 0x7f149c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14b259c700 1 --2- 172.21.15.93:0/134970947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18a500 0x5571da18aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14b2d9d700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f149c0a1260 0x7f149c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.656+0000 7f14b1d9b700 1 -- 172.21.15.93:0/134970947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18a500 msgr2=0x5571da18aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.657+0000 7f14b1d9b700 1 --2- 172.21.15.93:0/134970947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18a500 0x5571da18aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.657+0000 7f14b1d9b700 1 -- 172.21.15.93:0/134970947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 msgr2=0x5571da189fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.657+0000 7f14b1d9b700 1 --2- 172.21.15.93:0/134970947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da189fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.657+0000 7f14a37fe700 1 -- 172.21.15.93:0/1776264245 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f14a4014070 con 0x7f14b4100240 2023-11-16T16:48:13.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.657+0000 7f14b1d9b700 1 -- 172.21.15.93:0/134970947 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571d9f5c5e0 con 0x5571da18b0c0 2023-11-16T16:48:13.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.657+0000 7f14b1d9b700 1 --2- 172.21.15.93:0/134970947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18b0c0 0x5571da18b530 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009cb0 tx=0x7f14a8011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=856b996da09fb68d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.657+0000 7f14b2d9d700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f149c0a1260 0x7f149c0a3710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009cd0 tx=0x7f14ac00b3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.657+0000 7f14a0ff9700 1 -- 172.21.15.93:0/134970947 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8011d20 con 0x5571da18b0c0 2023-11-16T16:48:13.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.657+0000 7f14a0ff9700 1 -- 172.21.15.93:0/134970947 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14a8011e80 con 0x5571da18b0c0 2023-11-16T16:48:13.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.657+0000 7f14a0ff9700 1 -- 172.21.15.93:0/134970947 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8005280 con 0x5571da18b0c0 2023-11-16T16:48:13.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.658+0000 7f14cc70cb80 1 -- 172.21.15.93:0/134970947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18b0c0 msgr2=0x5571da18b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.658+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/134970947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18b0c0 0x5571da18b530 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009cb0 tx=0x7f14a8011440 comp rx=0 tx=0).stop 2023-11-16T16:48:13.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.658+0000 7f14cc70cb80 1 -- 172.21.15.93:0/134970947 shutdown_connections 2023-11-16T16:48:13.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.658+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/134970947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da189fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.658+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/134970947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18b0c0 0x5571da18b530 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.658+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/134970947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18a500 0x5571da18aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.658+0000 7f14cc70cb80 1 -- 172.21.15.93:0/134970947 >> 172.21.15.93:0/134970947 conn(0x5571da17f310 msgr2=0x5571da184620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.658+0000 7f14cc70cb80 1 -- 172.21.15.93:0/134970947 shutdown_connections 2023-11-16T16:48:13.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.658+0000 7f14cc70cb80 1 -- 172.21.15.93:0/134970947 wait complete. 2023-11-16T16:48:13.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.658+0000 7f14cc70cb80 1 -- start start 2023-11-16T16:48:13.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.659+0000 7f14cc70cb80 1 Processor -- start 2023-11-16T16:48:13.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.659+0000 7f14cc70cb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da189bb0 0x5571da193550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.659+0000 7f14cc70cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da18a500 0x5571da193a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.659+0000 7f14cc70cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 0x5571da19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.659+0000 7f14cc70cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571da07c890 con 0x5571da18a500 2023-11-16T16:48:13.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.659+0000 7f14cc70cb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571da07c530 con 0x5571da189bb0 2023-11-16T16:48:13.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.659+0000 7f14cc70cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571d9f5e8d0 con 0x5571da18b0c0 2023-11-16T16:48:13.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.659+0000 7f14cc70cb80 10 client.0 _ll_register_callbacks cb 0x5571da09a730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:13.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.659+0000 7f14b1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da18a500 0x5571da193a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.659+0000 7f14b1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da18a500 0x5571da193a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44794/0 (socket says 172.21.15.93:44794) 2023-11-16T16:48:13.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.659+0000 7f14b1d9b700 1 -- 172.21.15.93:0/1252509554 learned_addr learned my addr 172.21.15.93:0/1252509554 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.659+0000 7f14b259c700 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da189bb0 0x5571da193550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.660+0000 7f14b2d9d700 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 0x5571da19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.660+0000 7f14b1d9b700 1 -- 172.21.15.93:0/1252509554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 msgr2=0x5571da19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.660+0000 7f14b1d9b700 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 0x5571da19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.660+0000 7f14b1d9b700 1 -- 172.21.15.93:0/1252509554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da189bb0 msgr2=0x5571da193550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.660+0000 7f14b1d9b700 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da189bb0 0x5571da193550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.660+0000 7f14b1d9b700 1 -- 172.21.15.93:0/1252509554 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a80098e0 con 0x5571da18a500 2023-11-16T16:48:13.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.660+0000 7f14b1d9b700 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da18a500 0x5571da193a90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f14a8011790 tx=0x7f14a8011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e96c8eca068d38b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.660+0000 7f1499ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:48:13.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.660+0000 7f1499ffb700 1 -- 172.21.15.93:0/1252509554 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8004260 con 0x5571da18a500 2023-11-16T16:48:13.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.660+0000 7f14cc70cb80 1 -- 172.21.15.93:0/1252509554 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571da09d050 con 0x5571da18a500 2023-11-16T16:48:13.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.660+0000 7f1499ffb700 1 -- 172.21.15.93:0/1252509554 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f14a80043c0 con 0x5571da18a500 2023-11-16T16:48:13.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.661+0000 7f1499ffb700 1 -- 172.21.15.93:0/1252509554 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14a8004580 con 0x5571da18a500 2023-11-16T16:48:13.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.661+0000 7f14cc70cb80 10 client.15129 fetch_fsmap learned FSMap version 23 2023-11-16T16:48:13.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.661+0000 7f14cc70cb80 1 -- 172.21.15.93:0/1252509554 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5571da0a1140 con 0x5571da18a500 2023-11-16T16:48:13.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.661+0000 7f1499ffb700 1 -- 172.21.15.93:0/1252509554 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571da09d050 con 0x5571da18a500 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.661+0000 7f1499ffb700 1 -- 172.21.15.93:0/1252509554 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f14a8003bb0 con 0x5571da18a500 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.661+0000 7f14cc70cb80 10 client.15129 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.661+0000 7f14cc70cb80 10 client.15129 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.661+0000 7f14cc70cb80 20 client.15129 populate_metadata read hostname 'smithi093' 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.661+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5571da19e690 0x5571da19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.661+0000 7f14cc70cb80 4 client.15129 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.661+0000 7f14cc70cb80 1 -- 172.21.15.93:0/1252509554 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571da0a9610 con 0x5571da19e690 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.662+0000 7f14b259c700 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5571da19e690 0x5571da19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.662+0000 7f1499ffb700 10 client.15129 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.662+0000 7f14b259c700 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5571da19e690 0x5571da19dac0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7884ccf04b4412e3 server_cookie=2a6e9ba95da16a52 in_seq=0 out_seq=0 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f1499ffb700 1 -- 172.21.15.93:0/1252509554 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5571da0a9610 con 0x5571da19e690 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f1499ffb700 10 client.15129 handle_command_reply: tid=0 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f14cc70cb80 1 client.15129 shutdown 2023-11-16T16:48:13.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f14cc70cb80 1 -- 172.21.15.93:0/1252509554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da18a500 msgr2=0x5571da193a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da18a500 0x5571da193a90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f14a8011790 tx=0x7f14a8011bb0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f14cc70cb80 1 -- 172.21.15.93:0/1252509554 shutdown_connections 2023-11-16T16:48:13.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5571da19e690 0x5571da19dac0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 0x5571da19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f1499ffb700 0 client.15129 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:48:13.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da189bb0 0x5571da193550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/1252509554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da18a500 0x5571da193a90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f14cc70cb80 1 -- 172.21.15.93:0/1252509554 >> 172.21.15.93:0/1252509554 conn(0x5571da17f310 msgr2=0x5571da183d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f14cc70cb80 1 -- 172.21.15.93:0/1252509554 shutdown_connections 2023-11-16T16:48:13.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f14cc70cb80 1 -- 172.21.15.93:0/1252509554 wait complete. 2023-11-16T16:48:13.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.663+0000 7f14cc70cb80 20 client.15129 trim_cache size 0 max 16384 2023-11-16T16:48:13.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.681+0000 7f14cc70cb80 1 Processor -- start 2023-11-16T16:48:13.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.681+0000 7f14cc70cb80 1 -- start start 2023-11-16T16:48:13.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14cc70cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da1a7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14cc70cb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18a500 0x5571da19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14cc70cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 0x5571da19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14cc70cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571da09d050 con 0x5571da189bb0 2023-11-16T16:48:13.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14cc70cb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571d9fe6ed0 con 0x5571da18a500 2023-11-16T16:48:13.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14cc70cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571da076e10 con 0x5571da18b0c0 2023-11-16T16:48:13.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da1a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da1a7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44806/0 (socket says 172.21.15.93:44806) 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14b259c700 1 -- 172.21.15.93:0/2844343724 learned_addr learned my addr 172.21.15.93:0/2844343724 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14b1d9b700 1 --2- 172.21.15.93:0/2844343724 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18a500 0x5571da19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14b2d9d700 1 --2- 172.21.15.93:0/2844343724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 0x5571da19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14b259c700 1 -- 172.21.15.93:0/2844343724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 msgr2=0x5571da19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14b259c700 1 --2- 172.21.15.93:0/2844343724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 0x5571da19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14b259c700 1 -- 172.21.15.93:0/2844343724 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18a500 msgr2=0x5571da19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14b259c700 1 --2- 172.21.15.93:0/2844343724 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18a500 0x5571da19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.682+0000 7f14b259c700 1 -- 172.21.15.93:0/2844343724 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571da0a0e90 con 0x5571da189bb0 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14b259c700 1 --2- 172.21.15.93:0/2844343724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da1a7420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14a400efd0 tx=0x7f14a4003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5c8cddbada251e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14a17fa700 1 -- 172.21.15.93:0/2844343724 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4097fa0 con 0x5571da189bb0 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14a17fa700 1 -- 172.21.15.93:0/2844343724 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14a4098200 con 0x5571da189bb0 2023-11-16T16:48:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14a17fa700 1 -- 172.21.15.93:0/2844343724 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a409a320 con 0x5571da189bb0 2023-11-16T16:48:13.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14cc70cb80 1 -- 172.21.15.93:0/2844343724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 msgr2=0x5571da1a7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/2844343724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da1a7420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14a400efd0 tx=0x7f14a4003710 comp rx=0 tx=0).stop 2023-11-16T16:48:13.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14cc70cb80 1 -- 172.21.15.93:0/2844343724 shutdown_connections 2023-11-16T16:48:13.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/2844343724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da1a7420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/2844343724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 0x5571da19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/2844343724 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18a500 0x5571da19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14cc70cb80 1 -- 172.21.15.93:0/2844343724 >> 172.21.15.93:0/2844343724 conn(0x5571da17f310 msgr2=0x5571da1845f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14cc70cb80 1 -- 172.21.15.93:0/2844343724 shutdown_connections 2023-11-16T16:48:13.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.683+0000 7f14cc70cb80 1 -- 172.21.15.93:0/2844343724 wait complete. 2023-11-16T16:48:13.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.684+0000 7f14cc70cb80 1 -- start start 2023-11-16T16:48:13.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.684+0000 7f14cc70cb80 1 Processor -- start 2023-11-16T16:48:13.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.684+0000 7f14cc70cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da195450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.684+0000 7f14cc70cb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18a500 0x5571da19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.684+0000 7f14cc70cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 0x5571da195990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.684+0000 7f14cc70cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571da196080 con 0x5571da189bb0 2023-11-16T16:48:13.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.684+0000 7f14cc70cb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571da1f2550 con 0x5571da18a500 2023-11-16T16:48:13.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.684+0000 7f14cc70cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571da1f2710 con 0x5571da18b0c0 2023-11-16T16:48:13.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.684+0000 7f14cc70cb80 10 client.0 _ll_register_callbacks cb 0x7f14a8005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:13.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f14b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f14b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da195450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44812/0 (socket says 172.21.15.93:44812) 2023-11-16T16:48:13.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f14b259c700 1 -- 172.21.15.93:0/3687666249 learned_addr learned my addr 172.21.15.93:0/3687666249 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:13.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f14b1d9b700 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18a500 0x5571da19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f14b2d9d700 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 0x5571da195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f14b259c700 1 -- 172.21.15.93:0/3687666249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 msgr2=0x5571da195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f14b259c700 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 0x5571da195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f14b259c700 1 -- 172.21.15.93:0/3687666249 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18a500 msgr2=0x5571da19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:13.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f14b259c700 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18a500 0x5571da19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f14b259c700 1 -- 172.21.15.93:0/3687666249 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b410ba90 con 0x5571da189bb0 2023-11-16T16:48:13.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f14b259c700 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da195450 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14a4004800 tx=0x7f14a4098370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4ac865922b952a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:13.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f1499ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:48:13.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f1499ffb700 1 -- 172.21.15.93:0/3687666249 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4099350 con 0x5571da189bb0 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f1499ffb700 1 -- 172.21.15.93:0/3687666249 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f14a40994b0 con 0x5571da189bb0 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f14cc70cb80 1 -- 172.21.15.93:0/3687666249 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571da17cad0 con 0x5571da189bb0 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.685+0000 7f1499ffb700 1 -- 172.21.15.93:0/3687666249 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14a40ca2a0 con 0x5571da189bb0 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.686+0000 7f14cc70cb80 10 client.15135 fetch_fsmap learned FSMap version 23 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.686+0000 7f14cc70cb80 1 -- 172.21.15.93:0/3687666249 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5571d9ef1c10 con 0x5571da189bb0 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.686+0000 7f1499ffb700 1 -- 172.21.15.93:0/3687666249 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571da17cad0 con 0x5571da189bb0 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.686+0000 7f1499ffb700 1 -- 172.21.15.93:0/3687666249 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f14a40cab70 con 0x5571da189bb0 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.686+0000 7f14cc70cb80 10 client.15135 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.686+0000 7f14cc70cb80 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.686+0000 7f14cc70cb80 20 client.15135 populate_metadata read hostname 'smithi093' 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.686+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5571da1f6fe0 0x5571da1f3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.686+0000 7f14cc70cb80 4 client.15135 mds_command: new command op to 24566 tid=0[{"prefix": "status"}] 2023-11-16T16:48:13.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.686+0000 7f14cc70cb80 1 -- 172.21.15.93:0/3687666249 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5571d9f29f20 con 0x5571da1f6fe0 2023-11-16T16:48:13.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.686+0000 7f14b1d9b700 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5571da1f6fe0 0x5571da1f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:13.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f14b1d9b700 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5571da1f6fe0 0x5571da1f3990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9276f58db60cc52e server_cookie=5f49d9b2d5c0ef15 in_seq=0 out_seq=0 2023-11-16T16:48:13.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f1499ffb700 10 client.15135 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f1499ffb700 1 -- 172.21.15.93:0/3687666249 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5571d9f29f20 con 0x5571da1f6fe0 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f1499ffb700 10 client.15135 handle_command_reply: tid=0 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f14cc70cb80 1 client.15135 shutdown 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f14cc70cb80 1 -- 172.21.15.93:0/3687666249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 msgr2=0x5571da195450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da195450 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14a4004800 tx=0x7f14a4098370 comp rx=0 tx=0).stop 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f14cc70cb80 1 -- 172.21.15.93:0/3687666249 shutdown_connections 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5571da1f6fe0 0x5571da1f3990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571da189bb0 0x5571da195450 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571da18b0c0 0x5571da195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f1499ffb700 0 client.15135 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f14cc70cb80 1 --2- 172.21.15.93:0/3687666249 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5571da18a500 0x5571da19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f14cc70cb80 1 -- 172.21.15.93:0/3687666249 >> 172.21.15.93:0/3687666249 conn(0x5571da17f310 msgr2=0x5571da19dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.687+0000 7f14cc70cb80 1 -- 172.21.15.93:0/3687666249 shutdown_connections 2023-11-16T16:48:13.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14cc70cb80 1 -- 172.21.15.93:0/3687666249 wait complete. 2023-11-16T16:48:13.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14cc70cb80 20 client.15135 trim_cache size 0 max 16384 2023-11-16T16:48:13.694 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:48:13.694 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "6a59c1a8-849e-11ee-8dbe-212e2dc638e7", 2023-11-16T16:48:13.694 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 0, 2023-11-16T16:48:13.694 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24566, 2023-11-16T16:48:13.694 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-11-16T16:48:13.694 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-11-16T16:48:13.694 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-11-16T16:48:13.695 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 96.560578894000002, 2023-11-16T16:48:13.695 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 23, 2023-11-16T16:48:13.695 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 92, 2023-11-16T16:48:13.695 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 90, 2023-11-16T16:48:13.695 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 104.323674436 2023-11-16T16:48:13.695 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:48:13.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14b8dff700 1 -- 172.21.15.93:0/1776264245 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f149c0a1260 msgr2=0x7f149c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14b8dff700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f149c0a1260 0x7f149c0a3710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009cd0 tx=0x7f14ac00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14b8dff700 1 -- 172.21.15.93:0/1776264245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14b4100240 msgr2=0x7f14b41982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:13.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14b8dff700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14b4100240 0x7f14b41982f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14a400e970 tx=0x7f14a400ec80 comp rx=0 tx=0).stop 2023-11-16T16:48:13.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14b8dff700 1 -- 172.21.15.93:0/1776264245 shutdown_connections 2023-11-16T16:48:13.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14b8dff700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f149c0a1260 0x7f149c0a3710 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14b8dff700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14b4100240 0x7f14b41982f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14b8dff700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14b4101dc0 0x7f14b419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14b8dff700 1 --2- 172.21.15.93:0/1776264245 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b4101430 0x7f14b4198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:13.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14b8dff700 1 -- 172.21.15.93:0/1776264245 >> 172.21.15.93:0/1776264245 conn(0x7f14b40fb820 msgr2=0x7f14b4106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:13.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14b8dff700 1 -- 172.21.15.93:0/1776264245 shutdown_connections 2023-11-16T16:48:13.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:13.688+0000 7f14b8dff700 1 -- 172.21.15.93:0/1776264245 wait complete. 2023-11-16T16:48:13.708 INFO:tasks.cephadm:Setup complete, yielding 2023-11-16T16:48:13.708 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-16T16:48:13.717 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-11-16T16:48:13.717 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch status' 2023-11-16T16:48:14.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:13 smithi093 ceph-mon[97863]: pgmap v333: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:14.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:13 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1865228119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:14.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:13 smithi159 ceph-mon[131528]: pgmap v333: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:14.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:13 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1865228119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:14.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:13 smithi125 ceph-mon[131076]: pgmap v333: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:14.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:13 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1865228119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:14.562 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:48:16.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.090+0000 7f888da7c700 1 -- 172.21.15.93:0/3910385077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88881002f0 msgr2=0x7f8888100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:16.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.090+0000 7f888da7c700 1 --2- 172.21.15.93:0/3910385077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88881002f0 0x7f8888100700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8878009990 tx=0x7f8878009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:16.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.091+0000 7f888da7c700 1 -- 172.21.15.93:0/3910385077 shutdown_connections 2023-11-16T16:48:16.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.091+0000 7f888da7c700 1 --2- 172.21.15.93:0/3910385077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88881002f0 0x7f8888100700 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:16.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.091+0000 7f888da7c700 1 --2- 172.21.15.93:0/3910385077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8888101e80 0x7f8888108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:16.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.091+0000 7f888da7c700 1 --2- 172.21.15.93:0/3910385077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88881014f0 0x7f8888101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:16.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.091+0000 7f888da7c700 1 -- 172.21.15.93:0/3910385077 >> 172.21.15.93:0/3910385077 conn(0x7f88880fb900 msgr2=0x7f88880fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:16.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.091+0000 7f888da7c700 1 -- 172.21.15.93:0/3910385077 shutdown_connections 2023-11-16T16:48:16.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.091+0000 7f888da7c700 1 -- 172.21.15.93:0/3910385077 wait complete. 2023-11-16T16:48:16.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f888da7c700 1 Processor -- start 2023-11-16T16:48:16.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f888da7c700 1 -- start start 2023-11-16T16:48:16.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f888da7c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88881002f0 0x7f88881960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:16.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f888da7c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88881014f0 0x7f88881965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:16.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f888da7c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8888101e80 0x7f888819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:16.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f888da7c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888819dc30 con 0x7f88881014f0 2023-11-16T16:48:16.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f888da7c700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888819dd70 con 0x7f88881002f0 2023-11-16T16:48:16.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f888da7c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888819df00 con 0x7f8888101e80 2023-11-16T16:48:16.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f8886ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88881014f0 0x7f88881965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:16.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f8886ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88881014f0 0x7f88881965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44834/0 (socket says 172.21.15.93:44834) 2023-11-16T16:48:16.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f8886ffd700 1 -- 172.21.15.93:0/1506623584 learned_addr learned my addr 172.21.15.93:0/1506623584 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:16.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f8887fff700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8888101e80 0x7f888819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:16.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.092+0000 7f88877fe700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88881002f0 0x7f88881960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:16.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.093+0000 7f8886ffd700 1 -- 172.21.15.93:0/1506623584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8888101e80 msgr2=0x7f888819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:16.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.093+0000 7f8886ffd700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8888101e80 0x7f888819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:16.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.093+0000 7f8886ffd700 1 -- 172.21.15.93:0/1506623584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88881002f0 msgr2=0x7f88881960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:16.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.093+0000 7f8886ffd700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88881002f0 0x7f88881960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:16.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.093+0000 7f8886ffd700 1 -- 172.21.15.93:0/1506623584 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88780095f0 con 0x7f88881014f0 2023-11-16T16:48:16.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.093+0000 7f8886ffd700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88881014f0 0x7f88881965e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f887c00e9d0 tx=0x7f887c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:16.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.093+0000 7f8884ff9700 1 -- 172.21.15.93:0/1506623584 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c009790 con 0x7f88881014f0 2023-11-16T16:48:16.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.093+0000 7f8884ff9700 1 -- 172.21.15.93:0/1506623584 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f887c00ee70 con 0x7f88881014f0 2023-11-16T16:48:16.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.093+0000 7f8884ff9700 1 -- 172.21.15.93:0/1506623584 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c010ea0 con 0x7f88881014f0 2023-11-16T16:48:16.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.093+0000 7f888da7c700 1 -- 172.21.15.93:0/1506623584 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888810bcb0 con 0x7f88881014f0 2023-11-16T16:48:16.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.093+0000 7f888da7c700 1 -- 172.21.15.93:0/1506623584 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888810bb20 con 0x7f88881014f0 2023-11-16T16:48:16.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.095+0000 7f888da7c700 1 -- 172.21.15.93:0/1506623584 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8888107930 con 0x7f88881014f0 2023-11-16T16:48:16.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.096+0000 7f8884ff9700 1 -- 172.21.15.93:0/1506623584 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f887c00cbd0 con 0x7f88881014f0 2023-11-16T16:48:16.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.098+0000 7f8884ff9700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f88700a11e0 0x7f88700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:16.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.098+0000 7f8884ff9700 1 -- 172.21.15.93:0/1506623584 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f887c014070 con 0x7f88881014f0 2023-11-16T16:48:16.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.098+0000 7f88877fe700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f88700a11e0 0x7f88700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:16.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.099+0000 7f88877fe700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f88700a11e0 0x7f88700a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f887800b2e0 tx=0x7f8878005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:16.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.100+0000 7f8884ff9700 1 -- 172.21.15.93:0/1506623584 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f887c0966f0 con 0x7f88881014f0 2023-11-16T16:48:16.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:15 smithi093 ceph-mon[97863]: pgmap v334: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:16.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:15 smithi159 ceph-mon[131528]: pgmap v334: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:16.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.315+0000 7f888da7c700 1 -- 172.21.15.93:0/1506623584 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8888061c70 con 0x7f88700a11e0 2023-11-16T16:48:16.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.316+0000 7f8884ff9700 1 -- 172.21.15.93:0/1506623584 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f8888061c70 con 0x7f88700a11e0 2023-11-16T16:48:16.318 INFO:teuthology.orchestra.run.smithi093.stdout:Backend: cephadm 2023-11-16T16:48:16.318 INFO:teuthology.orchestra.run.smithi093.stdout:Available: Yes 2023-11-16T16:48:16.318 INFO:teuthology.orchestra.run.smithi093.stdout:Paused: No 2023-11-16T16:48:16.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.318+0000 7f888da7c700 1 -- 172.21.15.93:0/1506623584 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f88700a11e0 msgr2=0x7f88700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:16.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.318+0000 7f888da7c700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f88700a11e0 0x7f88700a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f887800b2e0 tx=0x7f8878005fb0 comp rx=0 tx=0).stop 2023-11-16T16:48:16.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.318+0000 7f888da7c700 1 -- 172.21.15.93:0/1506623584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88881014f0 msgr2=0x7f88881965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:16.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.318+0000 7f888da7c700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88881014f0 0x7f88881965e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f887c00e9d0 tx=0x7f887c00ece0 comp rx=0 tx=0).stop 2023-11-16T16:48:16.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.318+0000 7f888da7c700 1 -- 172.21.15.93:0/1506623584 shutdown_connections 2023-11-16T16:48:16.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.318+0000 7f888da7c700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f88700a11e0 0x7f88700a3690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:16.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.318+0000 7f888da7c700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8888101e80 0x7f888819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:16.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.318+0000 7f888da7c700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88881002f0 0x7f88881960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:16.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.318+0000 7f888da7c700 1 --2- 172.21.15.93:0/1506623584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88881014f0 0x7f88881965e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:16.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.318+0000 7f888da7c700 1 -- 172.21.15.93:0/1506623584 >> 172.21.15.93:0/1506623584 conn(0x7f88880fb900 msgr2=0x7f8888104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:16.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.319+0000 7f888da7c700 1 -- 172.21.15.93:0/1506623584 shutdown_connections 2023-11-16T16:48:16.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:16.319+0000 7f888da7c700 1 -- 172.21.15.93:0/1506623584 wait complete. 2023-11-16T16:48:16.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:15 smithi125 ceph-mon[131076]: pgmap v334: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:16.852 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-11-16T16:48:17.704 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:48:18.146 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:17 smithi093 ceph-mon[97863]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:18.147 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:17 smithi093 ceph-mon[97863]: pgmap v335: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:18.234 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:17 smithi159 ceph-mon[131528]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:18.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:17 smithi159 ceph-mon[131528]: pgmap v335: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:18.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:17 smithi125 ceph-mon[131076]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:18.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:17 smithi125 ceph-mon[131076]: pgmap v335: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:19.266 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.264+0000 7f836e6ac700 1 -- 172.21.15.93:0/3502572877 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8368101e80 msgr2=0x7f8368108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:19.266 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.264+0000 7f836e6ac700 1 --2- 172.21.15.93:0/3502572877 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8368101e80 0x7f8368108740 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f836000b0a0 tx=0x7f836000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:19.266 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.265+0000 7f836e6ac700 1 -- 172.21.15.93:0/3502572877 shutdown_connections 2023-11-16T16:48:19.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.265+0000 7f836e6ac700 1 --2- 172.21.15.93:0/3502572877 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8368101e80 0x7f8368108740 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:19.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.265+0000 7f836e6ac700 1 --2- 172.21.15.93:0/3502572877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83681002f0 0x7f8368100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:19.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.265+0000 7f836e6ac700 1 --2- 172.21.15.93:0/3502572877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83681014f0 0x7f8368101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:19.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.265+0000 7f836e6ac700 1 -- 172.21.15.93:0/3502572877 >> 172.21.15.93:0/3502572877 conn(0x7f83680fb900 msgr2=0x7f83680fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:19.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.265+0000 7f836e6ac700 1 -- 172.21.15.93:0/3502572877 shutdown_connections 2023-11-16T16:48:19.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.265+0000 7f836e6ac700 1 -- 172.21.15.93:0/3502572877 wait complete. 2023-11-16T16:48:19.268 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.266+0000 7f836e6ac700 1 Processor -- start 2023-11-16T16:48:19.268 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.266+0000 7f836e6ac700 1 -- start start 2023-11-16T16:48:19.269 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f836e6ac700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83681002f0 0x7f8368196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:19.269 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f836e6ac700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83681014f0 0x7f8368196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:19.269 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f836e6ac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8368101e80 0x7f836819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:19.269 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f836e6ac700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836819dca0 con 0x7f8368101e80 2023-11-16T16:48:19.269 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f836e6ac700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836819dde0 con 0x7f83681014f0 2023-11-16T16:48:19.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f836e6ac700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836819df70 con 0x7f83681002f0 2023-11-16T16:48:19.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f836cc49700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8368101e80 0x7f836819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:19.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f836cc49700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8368101e80 0x7f836819d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39552/0 (socket says 172.21.15.93:39552) 2023-11-16T16:48:19.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f836cc49700 1 -- 172.21.15.93:0/461168251 learned_addr learned my addr 172.21.15.93:0/461168251 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:19.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f8367fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83681002f0 0x7f8368196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:19.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f83677fe700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83681014f0 0x7f8368196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:19.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f836cc49700 1 -- 172.21.15.93:0/461168251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83681002f0 msgr2=0x7f8368196110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:19.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f836cc49700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83681002f0 0x7f8368196110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:19.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.267+0000 7f836cc49700 1 -- 172.21.15.93:0/461168251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83681014f0 msgr2=0x7f8368196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:19.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.268+0000 7f836cc49700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83681014f0 0x7f8368196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:19.271 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.268+0000 7f836cc49700 1 -- 172.21.15.93:0/461168251 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8360009d00 con 0x7f8368101e80 2023-11-16T16:48:19.271 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.268+0000 7f836cc49700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8368101e80 0x7f836819d6d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8360007a60 tx=0x7f836000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:19.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.268+0000 7f83657fa700 1 -- 172.21.15.93:0/461168251 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836000e040 con 0x7f8368101e80 2023-11-16T16:48:19.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.268+0000 7f83657fa700 1 -- 172.21.15.93:0/461168251 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8360004c50 con 0x7f8368101e80 2023-11-16T16:48:19.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.268+0000 7f83657fa700 1 -- 172.21.15.93:0/461168251 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83600052a0 con 0x7f8368101e80 2023-11-16T16:48:19.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.268+0000 7f836e6ac700 1 -- 172.21.15.93:0/461168251 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836810bc50 con 0x7f8368101e80 2023-11-16T16:48:19.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.268+0000 7f836e6ac700 1 -- 172.21.15.93:0/461168251 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836810bac0 con 0x7f8368101e80 2023-11-16T16:48:19.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.269+0000 7f836e6ac700 1 -- 172.21.15.93:0/461168251 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8368107870 con 0x7f8368101e80 2023-11-16T16:48:19.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.271+0000 7f83657fa700 1 -- 172.21.15.93:0/461168251 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f836001d470 con 0x7f8368101e80 2023-11-16T16:48:19.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.272+0000 7f83657fa700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f83500a1260 0x7f83500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:19.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.273+0000 7f83657fa700 1 -- 172.21.15.93:0/461168251 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f83600ce330 con 0x7f8368101e80 2023-11-16T16:48:19.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.273+0000 7f8367fff700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f83500a1260 0x7f83500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:19.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.273+0000 7f8367fff700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f83500a1260 0x7f83500a3710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8368101350 tx=0x7f8358009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:19.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.276+0000 7f83657fa700 1 -- 172.21.15.93:0/461168251 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8360097580 con 0x7f8368101e80 2023-11-16T16:48:19.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.481+0000 7f836e6ac700 1 -- 172.21.15.93:0/461168251 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8368061ae0 con 0x7f83500a1260 2023-11-16T16:48:19.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.494+0000 7f83657fa700 1 -- 172.21.15.93:0/461168251 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f8368061ae0 con 0x7f83500a1260 2023-11-16T16:48:19.496 INFO:teuthology.orchestra.run.smithi093.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-11-16T16:48:19.496 INFO:teuthology.orchestra.run.smithi093.stdout:mds.a smithi093 running (112s) 25s ago 112s 13.7M - 18.0.0-3905-g5e4a743a d5b1f852d682 9e7518683264 2023-11-16T16:48:19.496 INFO:teuthology.orchestra.run.smithi093.stdout:mds.b smithi125 running (110s) 38s ago 110s 29.8M - 18.0.0-3905-g5e4a743a d5b1f852d682 f719c66ca243 2023-11-16T16:48:19.496 INFO:teuthology.orchestra.run.smithi093.stdout:mds.c smithi159 running (2m) 38s ago 2m 18.3M - 18.0.0-3905-g5e4a743a d5b1f852d682 c38354ce2264 2023-11-16T16:48:19.496 INFO:teuthology.orchestra.run.smithi093.stdout:mds.d smithi093 running (2m) 25s ago 2m 13.9M - 18.0.0-3905-g5e4a743a d5b1f852d682 dc40bbcff651 2023-11-16T16:48:19.496 INFO:teuthology.orchestra.run.smithi093.stdout:mds.e smithi125 running (2m) 38s ago 2m 17.1M - 18.0.0-3905-g5e4a743a d5b1f852d682 410e537278fa 2023-11-16T16:48:19.496 INFO:teuthology.orchestra.run.smithi093.stdout:mds.f smithi159 running (2m) 38s ago 2m 17.2M - 18.0.0-3905-g5e4a743a d5b1f852d682 5ec490df026f 2023-11-16T16:48:19.496 INFO:teuthology.orchestra.run.smithi093.stdout:mds.g smithi093 running (115s) 25s ago 115s 13.8M - 18.0.0-3905-g5e4a743a d5b1f852d682 cf1b72894d54 2023-11-16T16:48:19.496 INFO:teuthology.orchestra.run.smithi093.stdout:mds.h smithi125 running (2m) 38s ago 2m 18.5M - 18.0.0-3905-g5e4a743a d5b1f852d682 8cf7b457af4a 2023-11-16T16:48:19.496 INFO:teuthology.orchestra.run.smithi093.stdout:mds.i smithi159 running (119s) 38s ago 118s 18.2M - 18.0.0-3905-g5e4a743a d5b1f852d682 416d8f2fa150 2023-11-16T16:48:19.496 INFO:teuthology.orchestra.run.smithi093.stdout:mds.j smithi093 running (2m) 25s ago 2m 13.8M - 18.0.0-3905-g5e4a743a d5b1f852d682 1ebf0b3cc285 2023-11-16T16:48:19.496 INFO:teuthology.orchestra.run.smithi093.stdout:mds.k smithi125 running (2m) 38s ago 2m 14.1M - 18.0.0-3905-g5e4a743a d5b1f852d682 0a638153ce8e 2023-11-16T16:48:19.497 INFO:teuthology.orchestra.run.smithi093.stdout:mds.l smithi159 running (2m) 38s ago 2m 16.5M - 18.0.0-3905-g5e4a743a d5b1f852d682 4f8b16423417 2023-11-16T16:48:19.497 INFO:teuthology.orchestra.run.smithi093.stdout:mgr.x smithi093 *:9283,8765 running (9m) 25s ago 9m 524M - 18.0.0-3905-g5e4a743a d5b1f852d682 9644b5ae03ce 2023-11-16T16:48:19.497 INFO:teuthology.orchestra.run.smithi093.stdout:mgr.y smithi125 *:8443,8765 running (7m) 38s ago 7m 446M - 18.0.0-3905-g5e4a743a d5b1f852d682 1feb1e3a87b3 2023-11-16T16:48:19.497 INFO:teuthology.orchestra.run.smithi093.stdout:mgr.z smithi159 *:8443,8765 running (7m) 38s ago 7m 448M - 18.0.0-3905-g5e4a743a d5b1f852d682 85d0ba0609d7 2023-11-16T16:48:19.497 INFO:teuthology.orchestra.run.smithi093.stdout:mon.a smithi093 running (9m) 25s ago 9m 94.9M 2048M 18.0.0-3905-g5e4a743a d5b1f852d682 c53a933466d7 2023-11-16T16:48:19.497 INFO:teuthology.orchestra.run.smithi093.stdout:mon.b smithi125 running (7m) 38s ago 7m 62.6M 2048M 18.0.0-3905-g5e4a743a d5b1f852d682 f3eb6aa64d25 2023-11-16T16:48:19.497 INFO:teuthology.orchestra.run.smithi093.stdout:mon.c smithi159 running (7m) 38s ago 7m 64.3M 2048M 18.0.0-3905-g5e4a743a d5b1f852d682 9d282f28b78b 2023-11-16T16:48:19.497 INFO:teuthology.orchestra.run.smithi093.stdout:osd.0 smithi093 running (6m) 25s ago 6m 74.2M 4317M 18.0.0-3905-g5e4a743a d5b1f852d682 a94ca879104c 2023-11-16T16:48:19.497 INFO:teuthology.orchestra.run.smithi093.stdout:osd.1 smithi125 running (6m) 38s ago 6m 78.5M 4292M 18.0.0-3905-g5e4a743a d5b1f852d682 2b16c7e8902a 2023-11-16T16:48:19.497 INFO:teuthology.orchestra.run.smithi093.stdout:osd.2 smithi159 running (6m) 38s ago 6m 77.2M 4292M 18.0.0-3905-g5e4a743a d5b1f852d682 9315fe0886af 2023-11-16T16:48:19.497 INFO:teuthology.orchestra.run.smithi093.stdout:osd.3 smithi093 running (5m) 25s ago 5m 66.5M 4317M 18.0.0-3905-g5e4a743a d5b1f852d682 25b7e12acc34 2023-11-16T16:48:19.498 INFO:teuthology.orchestra.run.smithi093.stdout:osd.4 smithi125 running (5m) 38s ago 5m 65.0M 4292M 18.0.0-3905-g5e4a743a d5b1f852d682 54a51cd050ff 2023-11-16T16:48:19.498 INFO:teuthology.orchestra.run.smithi093.stdout:osd.5 smithi159 running (5m) 38s ago 5m 68.3M 4292M 18.0.0-3905-g5e4a743a d5b1f852d682 0b219846fbac 2023-11-16T16:48:19.498 INFO:teuthology.orchestra.run.smithi093.stdout:osd.6 smithi093 running (4m) 25s ago 4m 61.5M 4317M 18.0.0-3905-g5e4a743a d5b1f852d682 1e1dc7a51051 2023-11-16T16:48:19.498 INFO:teuthology.orchestra.run.smithi093.stdout:osd.7 smithi125 running (4m) 38s ago 4m 66.2M 4292M 18.0.0-3905-g5e4a743a d5b1f852d682 60084f208acc 2023-11-16T16:48:19.498 INFO:teuthology.orchestra.run.smithi093.stdout:osd.8 smithi159 running (3m) 38s ago 3m 64.0M 4292M 18.0.0-3905-g5e4a743a d5b1f852d682 c215453879fd 2023-11-16T16:48:19.498 INFO:teuthology.orchestra.run.smithi093.stdout:osd.9 smithi093 running (3m) 25s ago 3m 59.9M 4317M 18.0.0-3905-g5e4a743a d5b1f852d682 bbcd25449e97 2023-11-16T16:48:19.498 INFO:teuthology.orchestra.run.smithi093.stdout:osd.10 smithi125 running (3m) 38s ago 3m 63.3M 4292M 18.0.0-3905-g5e4a743a d5b1f852d682 7e8d054c52ee 2023-11-16T16:48:19.498 INFO:teuthology.orchestra.run.smithi093.stdout:osd.11 smithi159 running (2m) 38s ago 2m 58.2M 4292M 18.0.0-3905-g5e4a743a d5b1f852d682 fa66c51eb51b 2023-11-16T16:48:19.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.496+0000 7f836e6ac700 1 -- 172.21.15.93:0/461168251 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f83500a1260 msgr2=0x7f83500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:19.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.496+0000 7f836e6ac700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f83500a1260 0x7f83500a3710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8368101350 tx=0x7f8358009260 comp rx=0 tx=0).stop 2023-11-16T16:48:19.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.496+0000 7f836e6ac700 1 -- 172.21.15.93:0/461168251 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8368101e80 msgr2=0x7f836819d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:19.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.496+0000 7f836e6ac700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8368101e80 0x7f836819d6d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8360007a60 tx=0x7f836000bab0 comp rx=0 tx=0).stop 2023-11-16T16:48:19.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.497+0000 7f836e6ac700 1 -- 172.21.15.93:0/461168251 shutdown_connections 2023-11-16T16:48:19.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.497+0000 7f836e6ac700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f83500a1260 0x7f83500a3710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:19.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.497+0000 7f836e6ac700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8368101e80 0x7f836819d6d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:19.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.497+0000 7f836e6ac700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83681002f0 0x7f8368196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:19.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.497+0000 7f836e6ac700 1 --2- 172.21.15.93:0/461168251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83681014f0 0x7f8368196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:19.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.497+0000 7f836e6ac700 1 -- 172.21.15.93:0/461168251 >> 172.21.15.93:0/461168251 conn(0x7f83680fb900 msgr2=0x7f83681049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:19.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.497+0000 7f836e6ac700 1 -- 172.21.15.93:0/461168251 shutdown_connections 2023-11-16T16:48:19.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:19.497+0000 7f836e6ac700 1 -- 172.21.15.93:0/461168251 wait complete. 2023-11-16T16:48:19.998 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-11-16T16:48:20.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:19 smithi093 ceph-mon[97863]: pgmap v336: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-16T16:48:20.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:19 smithi159 ceph-mon[131528]: pgmap v336: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-16T16:48:20.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:19 smithi125 ceph-mon[131076]: pgmap v336: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-16T16:48:20.878 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:48:21.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:20 smithi093 ceph-mon[97863]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:21.234 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:20 smithi159 ceph-mon[131528]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:21.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:20 smithi125 ceph-mon[131076]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:22.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:21 smithi093 ceph-mon[97863]: pgmap v337: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:22.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:21 smithi093 ceph-mon[97863]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-11-16T16:48:22.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:21 smithi159 ceph-mon[131528]: pgmap v337: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:22.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:21 smithi159 ceph-mon[131528]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-11-16T16:48:22.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:21 smithi125 ceph-mon[131076]: pgmap v337: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:22.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:21 smithi125 ceph-mon[131076]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-11-16T16:48:22.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.539+0000 7f537b891700 1 -- 172.21.15.93:0/2221488357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5374100d70 msgr2=0x7f5374103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:22.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.539+0000 7f537b891700 1 --2- 172.21.15.93:0/2221488357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5374100d70 0x7f5374103150 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f53700098f0 tx=0x7f5370009c00 comp rx=0 tx=0).stop 2023-11-16T16:48:22.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.539+0000 7f537b891700 1 -- 172.21.15.93:0/2221488357 shutdown_connections 2023-11-16T16:48:22.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.539+0000 7f537b891700 1 --2- 172.21.15.93:0/2221488357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5374100d70 0x7f5374103150 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:22.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.539+0000 7f537b891700 1 --2- 172.21.15.93:0/2221488357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5374105fb0 0x7f5374108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:22.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.539+0000 7f537b891700 1 --2- 172.21.15.93:0/2221488357 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5374103690 0x7f5374105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:22.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.539+0000 7f537b891700 1 -- 172.21.15.93:0/2221488357 >> 172.21.15.93:0/2221488357 conn(0x7f53740fa7f0 msgr2=0x7f53740fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:22.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.539+0000 7f537b891700 1 -- 172.21.15.93:0/2221488357 shutdown_connections 2023-11-16T16:48:22.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.539+0000 7f537b891700 1 -- 172.21.15.93:0/2221488357 wait complete. 2023-11-16T16:48:22.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.540+0000 7f537b891700 1 Processor -- start 2023-11-16T16:48:22.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.540+0000 7f537b891700 1 -- start start 2023-11-16T16:48:22.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f537b891700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5374100d70 0x7f53741960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:22.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f537b891700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5374103690 0x7f5374196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:22.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f537b891700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5374105fb0 0x7f537419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:22.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f537b891700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537419dc70 con 0x7f5374103690 2023-11-16T16:48:22.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f537b891700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537419ddb0 con 0x7f5374100d70 2023-11-16T16:48:22.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f537b891700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537419df40 con 0x7f5374105fb0 2023-11-16T16:48:22.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f5378e2c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5374103690 0x7f5374196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:22.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f5378e2c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5374103690 0x7f5374196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39582/0 (socket says 172.21.15.93:39582) 2023-11-16T16:48:22.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f5378e2c700 1 -- 172.21.15.93:0/402538887 learned_addr learned my addr 172.21.15.93:0/402538887 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:22.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f537962d700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5374100d70 0x7f53741960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:22.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f5379e2e700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5374105fb0 0x7f537419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:22.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f5378e2c700 1 -- 172.21.15.93:0/402538887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5374105fb0 msgr2=0x7f537419d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:22.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f5378e2c700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5374105fb0 0x7f537419d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:22.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f5378e2c700 1 -- 172.21.15.93:0/402538887 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5374100d70 msgr2=0x7f53741960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:22.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f5378e2c700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5374100d70 0x7f53741960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:22.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f5378e2c700 1 -- 172.21.15.93:0/402538887 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5370009520 con 0x7f5374103690 2023-11-16T16:48:22.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.541+0000 7f537962d700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5374100d70 0x7f53741960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:22.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.542+0000 7f5378e2c700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5374103690 0x7f5374196620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f536400e9d0 tx=0x7f536400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:22.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.542+0000 7f536a7fc700 1 -- 172.21.15.93:0/402538887 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364009790 con 0x7f5374103690 2023-11-16T16:48:22.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.542+0000 7f536a7fc700 1 -- 172.21.15.93:0/402538887 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f536400ee70 con 0x7f5374103690 2023-11-16T16:48:22.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.542+0000 7f537b891700 1 -- 172.21.15.93:0/402538887 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537410bcd0 con 0x7f5374103690 2023-11-16T16:48:22.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.542+0000 7f536a7fc700 1 -- 172.21.15.93:0/402538887 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364010ea0 con 0x7f5374103690 2023-11-16T16:48:22.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.542+0000 7f537b891700 1 -- 172.21.15.93:0/402538887 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537410bb40 con 0x7f5374103690 2023-11-16T16:48:22.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.543+0000 7f537b891700 1 -- 172.21.15.93:0/402538887 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53740fc360 con 0x7f5374103690 2023-11-16T16:48:22.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.545+0000 7f536a7fc700 1 -- 172.21.15.93:0/402538887 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f536400cbd0 con 0x7f5374103690 2023-11-16T16:48:22.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.546+0000 7f536a7fc700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f53600a1150 0x7f53600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:22.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.546+0000 7f536a7fc700 1 -- 172.21.15.93:0/402538887 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f5364014070 con 0x7f5374103690 2023-11-16T16:48:22.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.547+0000 7f537962d700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f53600a1150 0x7f53600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:22.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.547+0000 7f537962d700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f53600a1150 0x7f53600a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5370005e90 tx=0x7f5370019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:22.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.550+0000 7f536a7fc700 1 -- 172.21.15.93:0/402538887 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5364051730 con 0x7f5374103690 2023-11-16T16:48:22.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.756+0000 7f537b891700 1 -- 172.21.15.93:0/402538887 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5374061a30 con 0x7f53600a1150 2023-11-16T16:48:22.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.762+0000 7f536a7fc700 1 -- 172.21.15.93:0/402538887 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f5374061a30 con 0x7f53600a1150 2023-11-16T16:48:22.763 INFO:teuthology.orchestra.run.smithi093.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-11-16T16:48:22.763 INFO:teuthology.orchestra.run.smithi093.stdout:mds.all 12/12 42s ago 2m smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2023-11-16T16:48:22.764 INFO:teuthology.orchestra.run.smithi093.stdout:mgr 3/3 42s ago 7m smithi093=x;smithi125=y;smithi159=z;count:3 2023-11-16T16:48:22.764 INFO:teuthology.orchestra.run.smithi093.stdout:mon 3/3 42s ago 7m smithi093:172.21.15.93=a;smithi125:172.21.15.125=b;smithi159:172.21.15.159=c;count:3 2023-11-16T16:48:22.764 INFO:teuthology.orchestra.run.smithi093.stdout:osd 12 42s ago - 2023-11-16T16:48:22.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.764+0000 7f537b891700 1 -- 172.21.15.93:0/402538887 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f53600a1150 msgr2=0x7f53600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:22.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.764+0000 7f537b891700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f53600a1150 0x7f53600a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5370005e90 tx=0x7f5370019040 comp rx=0 tx=0).stop 2023-11-16T16:48:22.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.764+0000 7f537b891700 1 -- 172.21.15.93:0/402538887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5374103690 msgr2=0x7f5374196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:22.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.764+0000 7f537b891700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5374103690 0x7f5374196620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f536400e9d0 tx=0x7f536400ece0 comp rx=0 tx=0).stop 2023-11-16T16:48:22.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.764+0000 7f537b891700 1 -- 172.21.15.93:0/402538887 shutdown_connections 2023-11-16T16:48:22.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.764+0000 7f537b891700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f53600a1150 0x7f53600a3600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:22.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.764+0000 7f537b891700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5374105fb0 0x7f537419d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:22.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.764+0000 7f537b891700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5374100d70 0x7f53741960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:22.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.764+0000 7f537b891700 1 --2- 172.21.15.93:0/402538887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5374103690 0x7f5374196620 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:22.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.764+0000 7f537b891700 1 -- 172.21.15.93:0/402538887 >> 172.21.15.93:0/402538887 conn(0x7f53740fa7f0 msgr2=0x7f53741019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:22.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.764+0000 7f537b891700 1 -- 172.21.15.93:0/402538887 shutdown_connections 2023-11-16T16:48:22.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:22.764+0000 7f537b891700 1 -- 172.21.15.93:0/402538887 wait complete. 2023-11-16T16:48:23.270 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-11-16T16:48:24.113 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:48:24.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:23 smithi159 ceph-mon[131528]: pgmap v338: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-16T16:48:24.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:23 smithi159 ceph-mon[131528]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:24.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:23 smithi125 ceph-mon[131076]: pgmap v338: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-16T16:48:24.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:23 smithi125 ceph-mon[131076]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:24.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:23 smithi093 ceph-mon[97863]: pgmap v338: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-16T16:48:24.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:23 smithi093 ceph-mon[97863]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:25.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.715+0000 7f6efae44700 1 -- 172.21.15.93:0/393508486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ef41014f0 msgr2=0x7f6ef4101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.715+0000 7f6efae44700 1 --2- 172.21.15.93:0/393508486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ef41014f0 0x7f6ef4101940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009990 tx=0x7f6ee0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.716+0000 7f6efae44700 1 -- 172.21.15.93:0/393508486 shutdown_connections 2023-11-16T16:48:25.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.716+0000 7f6efae44700 1 --2- 172.21.15.93:0/393508486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ef4101e80 0x7f6ef4108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.716+0000 7f6efae44700 1 --2- 172.21.15.93:0/393508486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ef41002f0 0x7f6ef4100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.716+0000 7f6efae44700 1 --2- 172.21.15.93:0/393508486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ef41014f0 0x7f6ef4101940 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.716+0000 7f6efae44700 1 -- 172.21.15.93:0/393508486 >> 172.21.15.93:0/393508486 conn(0x7f6ef40fb900 msgr2=0x7f6ef40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:25.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.716+0000 7f6efae44700 1 -- 172.21.15.93:0/393508486 shutdown_connections 2023-11-16T16:48:25.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.716+0000 7f6efae44700 1 -- 172.21.15.93:0/393508486 wait complete. 2023-11-16T16:48:25.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.717+0000 7f6efae44700 1 Processor -- start 2023-11-16T16:48:25.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.717+0000 7f6efae44700 1 -- start start 2023-11-16T16:48:25.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.717+0000 7f6efae44700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ef41002f0 0x7f6ef4196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6efae44700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ef41014f0 0x7f6ef4196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6efae44700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ef4101e80 0x7f6ef419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6efae44700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef419db90 con 0x7f6ef41002f0 2023-11-16T16:48:25.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6efae44700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef419dcd0 con 0x7f6ef4101e80 2023-11-16T16:48:25.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6efae44700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef419de60 con 0x7f6ef41014f0 2023-11-16T16:48:25.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6ef8be0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ef41002f0 0x7f6ef4196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6ef8be0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ef41002f0 0x7f6ef4196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39606/0 (socket says 172.21.15.93:39606) 2023-11-16T16:48:25.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6ef8be0700 1 -- 172.21.15.93:0/3901715343 learned_addr learned my addr 172.21.15.93:0/3901715343 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:25.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6ef3fff700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ef41014f0 0x7f6ef4196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6ef93e1700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ef4101e80 0x7f6ef419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6ef8be0700 1 -- 172.21.15.93:0/3901715343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ef41014f0 msgr2=0x7f6ef4196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6ef8be0700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ef41014f0 0x7f6ef4196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6ef8be0700 1 -- 172.21.15.93:0/3901715343 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ef4101e80 msgr2=0x7f6ef419d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6ef8be0700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ef4101e80 0x7f6ef419d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.718+0000 7f6ef8be0700 1 -- 172.21.15.93:0/3901715343 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee00095f0 con 0x7f6ef41002f0 2023-11-16T16:48:25.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.719+0000 7f6ef8be0700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ef41002f0 0x7f6ef4196000 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800e9a0 tx=0x7f6ee800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:25.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.719+0000 7f6ef1ffb700 1 -- 172.21.15.93:0/3901715343 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8009790 con 0x7f6ef41002f0 2023-11-16T16:48:25.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.719+0000 7f6ef1ffb700 1 -- 172.21.15.93:0/3901715343 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee800c810 con 0x7f6ef41002f0 2023-11-16T16:48:25.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.719+0000 7f6ef1ffb700 1 -- 172.21.15.93:0/3901715343 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8003c70 con 0x7f6ef41002f0 2023-11-16T16:48:25.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.719+0000 7f6efae44700 1 -- 172.21.15.93:0/3901715343 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef410bc50 con 0x7f6ef41002f0 2023-11-16T16:48:25.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.719+0000 7f6efae44700 1 -- 172.21.15.93:0/3901715343 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef410bac0 con 0x7f6ef41002f0 2023-11-16T16:48:25.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.720+0000 7f6efae44700 1 -- 172.21.15.93:0/3901715343 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef419e550 con 0x7f6ef41002f0 2023-11-16T16:48:25.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.722+0000 7f6ef1ffb700 1 -- 172.21.15.93:0/3901715343 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6ee800cc80 con 0x7f6ef41002f0 2023-11-16T16:48:25.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.723+0000 7f6ef1ffb700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6ee40a11d0 0x7f6ee40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.724+0000 7f6ef1ffb700 1 -- 172.21.15.93:0/3901715343 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f6ee8014070 con 0x7f6ef41002f0 2023-11-16T16:48:25.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.724+0000 7f6ef3fff700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6ee40a11d0 0x7f6ee40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.724+0000 7f6ef3fff700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6ee40a11d0 0x7f6ee40a3680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000b680 tx=0x7f6ee0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:25.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.728+0000 7f6ef1ffb700 1 -- 172.21.15.93:0/3901715343 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6ee8096180 con 0x7f6ef41002f0 2023-11-16T16:48:25.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.932+0000 7f6efae44700 1 -- 172.21.15.93:0/3901715343 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ef419eb00 con 0x7f6ee40a11d0 2023-11-16T16:48:25.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.934+0000 7f6ef1ffb700 1 -- 172.21.15.93:0/3901715343 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f6ef419eb00 con 0x7f6ee40a11d0 2023-11-16T16:48:25.935 INFO:teuthology.orchestra.run.smithi093.stdout:HOST ADDR LABELS STATUS 2023-11-16T16:48:25.936 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093 172.21.15.93 2023-11-16T16:48:25.936 INFO:teuthology.orchestra.run.smithi093.stdout:smithi125 172.21.15.125 2023-11-16T16:48:25.936 INFO:teuthology.orchestra.run.smithi093.stdout:smithi159 172.21.15.159 2023-11-16T16:48:25.936 INFO:teuthology.orchestra.run.smithi093.stdout:3 hosts in cluster 2023-11-16T16:48:25.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.936+0000 7f6efae44700 1 -- 172.21.15.93:0/3901715343 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6ee40a11d0 msgr2=0x7f6ee40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.936+0000 7f6efae44700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6ee40a11d0 0x7f6ee40a3680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000b680 tx=0x7f6ee0005fd0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.936+0000 7f6efae44700 1 -- 172.21.15.93:0/3901715343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ef41002f0 msgr2=0x7f6ef4196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.936+0000 7f6efae44700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ef41002f0 0x7f6ef4196000 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800e9a0 tx=0x7f6ee800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.936+0000 7f6efae44700 1 -- 172.21.15.93:0/3901715343 shutdown_connections 2023-11-16T16:48:25.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.936+0000 7f6efae44700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6ee40a11d0 0x7f6ee40a3680 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.936+0000 7f6efae44700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ef41002f0 0x7f6ef4196000 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.936+0000 7f6efae44700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ef4101e80 0x7f6ef419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.936+0000 7f6efae44700 1 --2- 172.21.15.93:0/3901715343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ef41014f0 0x7f6ef4196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.936+0000 7f6efae44700 1 -- 172.21.15.93:0/3901715343 >> 172.21.15.93:0/3901715343 conn(0x7f6ef40fb900 msgr2=0x7f6ef40fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:25.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.936+0000 7f6efae44700 1 -- 172.21.15.93:0/3901715343 shutdown_connections 2023-11-16T16:48:25.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:25.936+0000 7f6efae44700 1 -- 172.21.15.93:0/3901715343 wait complete. 2023-11-16T16:48:26.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:25 smithi159 ceph-mon[131528]: pgmap v339: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:48:26.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:25 smithi125 ceph-mon[131076]: pgmap v339: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:48:26.360 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:25 smithi093 ceph-mon[97863]: pgmap v339: 129 pgs: 129 active+clean; 640 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:48:26.408 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-11-16T16:48:27.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:26 smithi159 ceph-mon[131528]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:27.259 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:48:27.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:26 smithi125 ceph-mon[131076]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:27.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:26 smithi093 ceph-mon[97863]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:28.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:27 smithi093 ceph-mon[97863]: pgmap v340: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:48:28.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:27 smithi159 ceph-mon[131528]: pgmap v340: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:48:28.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:27 smithi125 ceph-mon[131076]: pgmap v340: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:48:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.814+0000 7fd5823c6700 1 -- 172.21.15.93:0/750135077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd57c1014f0 msgr2=0x7fd57c101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.814+0000 7fd5823c6700 1 --2- 172.21.15.93:0/750135077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd57c1014f0 0x7fd57c101940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd570009990 tx=0x7fd570009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.814+0000 7fd5823c6700 1 -- 172.21.15.93:0/750135077 shutdown_connections 2023-11-16T16:48:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.814+0000 7fd5823c6700 1 --2- 172.21.15.93:0/750135077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57c101e80 0x7fd57c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.814+0000 7fd5823c6700 1 --2- 172.21.15.93:0/750135077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd57c1002f0 0x7fd57c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.814+0000 7fd5823c6700 1 --2- 172.21.15.93:0/750135077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd57c1014f0 0x7fd57c101940 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.814+0000 7fd5823c6700 1 -- 172.21.15.93:0/750135077 >> 172.21.15.93:0/750135077 conn(0x7fd57c0fb900 msgr2=0x7fd57c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.814+0000 7fd5823c6700 1 -- 172.21.15.93:0/750135077 shutdown_connections 2023-11-16T16:48:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.814+0000 7fd5823c6700 1 -- 172.21.15.93:0/750135077 wait complete. 2023-11-16T16:48:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.815+0000 7fd5823c6700 1 Processor -- start 2023-11-16T16:48:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.815+0000 7fd5823c6700 1 -- start start 2023-11-16T16:48:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd5823c6700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57c1002f0 0x7fd57c196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd5823c6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd57c1014f0 0x7fd57c1965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd5823c6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd57c101e80 0x7fd57c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd5823c6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c19dc00 con 0x7fd57c101e80 2023-11-16T16:48:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd5823c6700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c19dd40 con 0x7fd57c1002f0 2023-11-16T16:48:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd5823c6700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c19ded0 con 0x7fd57c1014f0 2023-11-16T16:48:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd57bfff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57c1002f0 0x7fd57c196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd580963700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd57c101e80 0x7fd57c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd57bfff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57c1002f0 0x7fd57c196070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:37374/0 (socket says 172.21.15.93:37374) 2023-11-16T16:48:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd57bfff700 1 -- 172.21.15.93:0/3701213560 learned_addr learned my addr 172.21.15.93:0/3701213560 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:28.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd580963700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd57c101e80 0x7fd57c19d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42166/0 (socket says 172.21.15.93:42166) 2023-11-16T16:48:28.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd57b7fe700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd57c1014f0 0x7fd57c1965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:28.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd57bfff700 1 -- 172.21.15.93:0/3701213560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd57c1014f0 msgr2=0x7fd57c1965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:28.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd57bfff700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd57c1014f0 0x7fd57c1965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:28.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd57bfff700 1 -- 172.21.15.93:0/3701213560 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd57c101e80 msgr2=0x7fd57c19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:28.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd57bfff700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd57c101e80 0x7fd57c19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:28.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.816+0000 7fd57bfff700 1 -- 172.21.15.93:0/3701213560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5700095f0 con 0x7fd57c1002f0 2023-11-16T16:48:28.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.817+0000 7fd57bfff700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57c1002f0 0x7fd57c196070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00e970 tx=0x7fd56c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:28.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.817+0000 7fd5797fa700 1 -- 172.21.15.93:0/3701213560 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c009790 con 0x7fd57c1002f0 2023-11-16T16:48:28.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.817+0000 7fd5823c6700 1 -- 172.21.15.93:0/3701213560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c10bc90 con 0x7fd57c1002f0 2023-11-16T16:48:28.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.817+0000 7fd5797fa700 1 -- 172.21.15.93:0/3701213560 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd56c00c7e0 con 0x7fd57c1002f0 2023-11-16T16:48:28.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.817+0000 7fd5797fa700 1 -- 172.21.15.93:0/3701213560 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c003c70 con 0x7fd57c1002f0 2023-11-16T16:48:28.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.817+0000 7fd5823c6700 1 -- 172.21.15.93:0/3701213560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c10bb00 con 0x7fd57c1002f0 2023-11-16T16:48:28.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.818+0000 7fd5823c6700 1 -- 172.21.15.93:0/3701213560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57c066ea0 con 0x7fd57c1002f0 2023-11-16T16:48:28.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.820+0000 7fd5797fa700 1 -- 172.21.15.93:0/3701213560 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd56c00c950 con 0x7fd57c1002f0 2023-11-16T16:48:28.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.821+0000 7fd5797fa700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd5640a1150 0x7fd5640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:28.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.822+0000 7fd5797fa700 1 -- 172.21.15.93:0/3701213560 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fd56c014070 con 0x7fd57c1002f0 2023-11-16T16:48:28.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.822+0000 7fd57b7fe700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd5640a1150 0x7fd5640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:28.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.822+0000 7fd57b7fe700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd5640a1150 0x7fd5640a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd57000b2e0 tx=0x7fd570005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:28.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:28.826+0000 7fd5797fa700 1 -- 172.21.15.93:0/3701213560 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd56c099ce0 con 0x7fd57c1002f0 2023-11-16T16:48:29.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.032+0000 7fd5823c6700 1 -- 172.21.15.93:0/3701213560 --> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd57c19ea80 con 0x7fd5640a1150 2023-11-16T16:48:29.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.034+0000 7fd5797fa700 1 -- 172.21.15.93:0/3701213560 <== mgr.14152 v2:172.21.15.93:6800/3283196047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7fd57c19ea80 con 0x7fd5640a1150 2023-11-16T16:48:29.037 INFO:teuthology.orchestra.run.smithi093.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-11-16T16:48:29.037 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040150400BGN 372G No 3m ago LVM detected, locked 2023-11-16T16:48:29.037 INFO:teuthology.orchestra.run.smithi093.stdout:smithi125 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040123400BGN 372G No 3m ago LVM detected, locked 2023-11-16T16:48:29.037 INFO:teuthology.orchestra.run.smithi093.stdout:smithi159 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GH400BGN 372G No 2m ago LVM detected, locked 2023-11-16T16:48:29.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.037+0000 7fd5823c6700 1 -- 172.21.15.93:0/3701213560 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd5640a1150 msgr2=0x7fd5640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:29.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.037+0000 7fd5823c6700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd5640a1150 0x7fd5640a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd57000b2e0 tx=0x7fd570005fb0 comp rx=0 tx=0).stop 2023-11-16T16:48:29.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.037+0000 7fd5823c6700 1 -- 172.21.15.93:0/3701213560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57c1002f0 msgr2=0x7fd57c196070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.037+0000 7fd5823c6700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57c1002f0 0x7fd57c196070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00e970 tx=0x7fd56c00ec80 comp rx=0 tx=0).stop 2023-11-16T16:48:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.037+0000 7fd5823c6700 1 -- 172.21.15.93:0/3701213560 shutdown_connections 2023-11-16T16:48:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.037+0000 7fd5823c6700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd5640a1150 0x7fd5640a3600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.037+0000 7fd5823c6700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd57c101e80 0x7fd57c19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.037+0000 7fd5823c6700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57c1002f0 0x7fd57c196070 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.037+0000 7fd5823c6700 1 --2- 172.21.15.93:0/3701213560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd57c1014f0 0x7fd57c1965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.037+0000 7fd5823c6700 1 -- 172.21.15.93:0/3701213560 >> 172.21.15.93:0/3701213560 conn(0x7fd57c0fb900 msgr2=0x7fd57c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.037+0000 7fd5823c6700 1 -- 172.21.15.93:0/3701213560 shutdown_connections 2023-11-16T16:48:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:29.037+0000 7fd5823c6700 1 -- 172.21.15.93:0/3701213560 wait complete. 2023-11-16T16:48:29.551 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-16T16:48:29.563 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-11-16T16:48:29.563 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-11-16T16:48:30.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:29 smithi159 ceph-mon[131528]: pgmap v341: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:48:30.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:29 smithi159 ceph-mon[131528]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:30.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:29 smithi125 ceph-mon[131076]: pgmap v341: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:48:30.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:29 smithi125 ceph-mon[131076]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:30.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:29 smithi093 ceph-mon[97863]: pgmap v341: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:48:30.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:29 smithi093 ceph-mon[97863]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:48:30.407 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:48:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:31 smithi093 ceph-mon[97863]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-16T16:48:31.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:31 smithi159 ceph-mon[131528]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-16T16:48:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:31 smithi125 ceph-mon[131076]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-16T16:48:32.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.064+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1071591684 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeac1002f0 msgr2=0x7efeac100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:32.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.064+0000 7efeb2bf3700 1 --2- 172.21.15.93:0/1071591684 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeac1002f0 0x7efeac100700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efea00098f0 tx=0x7efea0009c00 comp rx=0 tx=0).stop 2023-11-16T16:48:32.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.064+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1071591684 shutdown_connections 2023-11-16T16:48:32.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.064+0000 7efeb2bf3700 1 --2- 172.21.15.93:0/1071591684 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeac1002f0 0x7efeac100700 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:32.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.064+0000 7efeb2bf3700 1 --2- 172.21.15.93:0/1071591684 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efeac101e80 0x7efeac108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:32.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.064+0000 7efeb2bf3700 1 --2- 172.21.15.93:0/1071591684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efeac1014f0 0x7efeac101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:32.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.064+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1071591684 >> 172.21.15.93:0/1071591684 conn(0x7efeac0fb900 msgr2=0x7efeac0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:32.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.064+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1071591684 shutdown_connections 2023-11-16T16:48:32.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.064+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1071591684 wait complete. 2023-11-16T16:48:32.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.065+0000 7efeb2bf3700 1 Processor -- start 2023-11-16T16:48:32.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.065+0000 7efeb2bf3700 1 -- start start 2023-11-16T16:48:32.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.065+0000 7efeb2bf3700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efeac1002f0 0x7efeac1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:32.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.065+0000 7efeb2bf3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeac1014f0 0x7efeac198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:32.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeb2bf3700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efeac101e80 0x7efeac19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:32.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeb2bf3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac19fe70 con 0x7efeac1014f0 2023-11-16T16:48:32.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeb2bf3700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac19ffb0 con 0x7efeac1002f0 2023-11-16T16:48:32.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeb2bf3700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac1a0140 con 0x7efeac101e80 2023-11-16T16:48:32.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeabfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeac1014f0 0x7efeac198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:32.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeabfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeac1014f0 0x7efeac198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42184/0 (socket says 172.21.15.93:42184) 2023-11-16T16:48:32.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeabfff700 1 -- 172.21.15.93:0/1698362886 learned_addr learned my addr 172.21.15.93:0/1698362886 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:32.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeb1190700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efeac101e80 0x7efeac19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:32.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeb098f700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efeac1002f0 0x7efeac1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:32.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeabfff700 1 -- 172.21.15.93:0/1698362886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efeac101e80 msgr2=0x7efeac19f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:32.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeabfff700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efeac101e80 0x7efeac19f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:32.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeabfff700 1 -- 172.21.15.93:0/1698362886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efeac1002f0 msgr2=0x7efeac1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:32.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeabfff700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efeac1002f0 0x7efeac1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:32.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeabfff700 1 -- 172.21.15.93:0/1698362886 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea0009520 con 0x7efeac1014f0 2023-11-16T16:48:32.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.066+0000 7efeabfff700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeac1014f0 0x7efeac198820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efe9800e9d0 tx=0x7efe9800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:32.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.067+0000 7efea9ffb700 1 -- 172.21.15.93:0/1698362886 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98009790 con 0x7efeac1014f0 2023-11-16T16:48:32.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.067+0000 7efea9ffb700 1 -- 172.21.15.93:0/1698362886 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe9800ee70 con 0x7efeac1014f0 2023-11-16T16:48:32.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.067+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1698362886 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac10bcb0 con 0x7efeac1014f0 2023-11-16T16:48:32.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.067+0000 7efea9ffb700 1 -- 172.21.15.93:0/1698362886 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98010ea0 con 0x7efeac1014f0 2023-11-16T16:48:32.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.067+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1698362886 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac10bb20 con 0x7efeac1014f0 2023-11-16T16:48:32.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.068+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1698362886 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeac192a10 con 0x7efeac1014f0 2023-11-16T16:48:32.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.070+0000 7efea9ffb700 1 -- 172.21.15.93:0/1698362886 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efe9800cbd0 con 0x7efeac1014f0 2023-11-16T16:48:32.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.071+0000 7efea9ffb700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efe9c0a1150 0x7efe9c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:32.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.071+0000 7efea9ffb700 1 -- 172.21.15.93:0/1698362886 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7efe98014070 con 0x7efeac1014f0 2023-11-16T16:48:32.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.071+0000 7efeb098f700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efe9c0a1150 0x7efe9c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:32.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.072+0000 7efeb098f700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efe9c0a1150 0x7efe9c0a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efea0009fd0 tx=0x7efea0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:32.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.075+0000 7efea9ffb700 1 -- 172.21.15.93:0/1698362886 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efe98051730 con 0x7efeac1014f0 2023-11-16T16:48:32.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.316+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1698362886 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efeac02cfe0 con 0x7efeac1014f0 2023-11-16T16:48:32.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.317+0000 7efea9ffb700 1 -- 172.21.15.93:0/1698362886 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2903 (secure 0 0 0) 0x7efe98096210 con 0x7efeac1014f0 2023-11-16T16:48:32.319 INFO:teuthology.orchestra.run.smithi093.stdout:e23 2023-11-16T16:48:32.319 INFO:teuthology.orchestra.run.smithi093.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-11-16T16:48:32.319 INFO:teuthology.orchestra.run.smithi093.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T16:48:32.319 INFO:teuthology.orchestra.run.smithi093.stdout:legacy client fscid: 1 2023-11-16T16:48:32.319 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:32.320 INFO:teuthology.orchestra.run.smithi093.stdout:Filesystem 'cephfs' (1) 2023-11-16T16:48:32.320 INFO:teuthology.orchestra.run.smithi093.stdout:fs_name cephfs 2023-11-16T16:48:32.320 INFO:teuthology.orchestra.run.smithi093.stdout:epoch 23 2023-11-16T16:48:32.320 INFO:teuthology.orchestra.run.smithi093.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-11-16T16:48:32.320 INFO:teuthology.orchestra.run.smithi093.stdout:created 2023-11-16T16:46:37.124338+0000 2023-11-16T16:48:32.320 INFO:teuthology.orchestra.run.smithi093.stdout:modified 2023-11-16T16:47:10.311966+0000 2023-11-16T16:48:32.320 INFO:teuthology.orchestra.run.smithi093.stdout:tableserver 0 2023-11-16T16:48:32.320 INFO:teuthology.orchestra.run.smithi093.stdout:root 0 2023-11-16T16:48:32.321 INFO:teuthology.orchestra.run.smithi093.stdout:session_timeout 300 2023-11-16T16:48:32.321 INFO:teuthology.orchestra.run.smithi093.stdout:session_autoclose 300 2023-11-16T16:48:32.321 INFO:teuthology.orchestra.run.smithi093.stdout:max_file_size 1099511627776 2023-11-16T16:48:32.321 INFO:teuthology.orchestra.run.smithi093.stdout:max_xattr_size 65536 2023-11-16T16:48:32.321 INFO:teuthology.orchestra.run.smithi093.stdout:required_client_features {} 2023-11-16T16:48:32.321 INFO:teuthology.orchestra.run.smithi093.stdout:last_failure 0 2023-11-16T16:48:32.321 INFO:teuthology.orchestra.run.smithi093.stdout:last_failure_osd_epoch 0 2023-11-16T16:48:32.321 INFO:teuthology.orchestra.run.smithi093.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T16:48:32.321 INFO:teuthology.orchestra.run.smithi093.stdout:max_mds 3 2023-11-16T16:48:32.322 INFO:teuthology.orchestra.run.smithi093.stdout:in 0,1,2 2023-11-16T16:48:32.322 INFO:teuthology.orchestra.run.smithi093.stdout:up {0=24566,1=24541,2=24533} 2023-11-16T16:48:32.322 INFO:teuthology.orchestra.run.smithi093.stdout:failed 2023-11-16T16:48:32.322 INFO:teuthology.orchestra.run.smithi093.stdout:damaged 2023-11-16T16:48:32.322 INFO:teuthology.orchestra.run.smithi093.stdout:stopped 2023-11-16T16:48:32.322 INFO:teuthology.orchestra.run.smithi093.stdout:data_pools [3,4] 2023-11-16T16:48:32.322 INFO:teuthology.orchestra.run.smithi093.stdout:metadata_pool 2 2023-11-16T16:48:32.322 INFO:teuthology.orchestra.run.smithi093.stdout:inline_data disabled 2023-11-16T16:48:32.323 INFO:teuthology.orchestra.run.smithi093.stdout:balancer 2023-11-16T16:48:32.323 INFO:teuthology.orchestra.run.smithi093.stdout:bal_rank_mask -1 2023-11-16T16:48:32.323 INFO:teuthology.orchestra.run.smithi093.stdout:standby_count_wanted 0 2023-11-16T16:48:32.323 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b{0:24566} state up:active seq 3 addr [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:48:32.323 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c{0:24523} state up:standby-replay seq 1 addr [v2:172.21.15.159:6836/778982589,v1:172.21.15.159:6837/778982589] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:48:32.323 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.i{1:24541} state up:active seq 13 addr [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:48:32.323 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.f{1:24511} state up:standby-replay seq 1 addr [v2:172.21.15.159:6834/3072074959,v1:172.21.15.159:6835/3072074959] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:48:32.323 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.h{2:24533} state up:active seq 14 addr [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:48:32.323 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.e{2:24506} state up:standby-replay seq 1 addr [v2:172.21.15.125:6834/2117404468,v1:172.21.15.125:6835/2117404468] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:48:32.324 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:32.324 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:32.324 INFO:teuthology.orchestra.run.smithi093.stdout:Standby daemons: 2023-11-16T16:48:32.324 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:32.324 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.j{-1:14463} state up:standby seq 1 addr [v2:172.21.15.93:6834/278429922,v1:172.21.15.93:6835/278429922] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:48:32.324 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d{-1:14475} state up:standby seq 1 addr [v2:172.21.15.93:6836/3647861869,v1:172.21.15.93:6837/3647861869] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:48:32.324 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.g{-1:14520} state up:standby seq 1 addr [v2:172.21.15.93:6838/3614592201,v1:172.21.15.93:6839/3614592201] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:48:32.324 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a{-1:14526} state up:standby seq 1 addr [v2:172.21.15.93:6840/4068789362,v1:172.21.15.93:6841/4068789362] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:48:32.325 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.l{-1:24460} state up:standby seq 1 addr [v2:172.21.15.159:6832/2711358292,v1:172.21.15.159:6833/2711358292] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:48:32.325 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.k{-1:24470} state up:standby seq 1 addr [v2:172.21.15.125:6832/1299400539,v1:172.21.15.125:6833/1299400539] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:48:32.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.319+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1698362886 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efe9c0a1150 msgr2=0x7efe9c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:32.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.319+0000 7efeb2bf3700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efe9c0a1150 0x7efe9c0a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efea0009fd0 tx=0x7efea0019040 comp rx=0 tx=0).stop 2023-11-16T16:48:32.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.319+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1698362886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeac1014f0 msgr2=0x7efeac198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:32.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.319+0000 7efeb2bf3700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeac1014f0 0x7efeac198820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efe9800e9d0 tx=0x7efe9800ece0 comp rx=0 tx=0).stop 2023-11-16T16:48:32.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.320+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1698362886 shutdown_connections 2023-11-16T16:48:32.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.320+0000 7efeb2bf3700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efe9c0a1150 0x7efe9c0a3600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:32.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.320+0000 7efeb2bf3700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efeac101e80 0x7efeac19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:32.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.320+0000 7efeb2bf3700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efeac1002f0 0x7efeac1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:32.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.320+0000 7efeb2bf3700 1 --2- 172.21.15.93:0/1698362886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeac1014f0 0x7efeac198820 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:32.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.320+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1698362886 >> 172.21.15.93:0/1698362886 conn(0x7efeac0fb900 msgr2=0x7efeac106ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:32.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.320+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1698362886 shutdown_connections 2023-11-16T16:48:32.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:32.320+0000 7efeb2bf3700 1 -- 172.21.15.93:0/1698362886 wait complete. 2023-11-16T16:48:32.327 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 23 2023-11-16T16:48:32.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:32 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1698362886' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-16T16:48:32.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:32 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1698362886' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-16T16:48:32.801 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-11-16T16:48:32.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:32 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1698362886' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-16T16:48:33.648 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:48:33.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:33 smithi093 ceph-mon[97863]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:48:33.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:33 smithi159 ceph-mon[131528]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:48:33.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:33 smithi125 ceph-mon[131076]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:48:35.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.186+0000 7fba99740700 1 -- 172.21.15.93:0/2189635750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba94101e80 msgr2=0x7fba94108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:35.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.186+0000 7fba99740700 1 --2- 172.21.15.93:0/2189635750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba94101e80 0x7fba94108740 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00b0a0 tx=0x7fba8c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:35.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.187+0000 7fba99740700 1 -- 172.21.15.93:0/2189635750 shutdown_connections 2023-11-16T16:48:35.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.187+0000 7fba99740700 1 --2- 172.21.15.93:0/2189635750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba94101e80 0x7fba94108740 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:35.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.187+0000 7fba99740700 1 --2- 172.21.15.93:0/2189635750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba941002f0 0x7fba94100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:35.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.187+0000 7fba99740700 1 --2- 172.21.15.93:0/2189635750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba941014f0 0x7fba94101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:35.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.187+0000 7fba99740700 1 -- 172.21.15.93:0/2189635750 >> 172.21.15.93:0/2189635750 conn(0x7fba940fb900 msgr2=0x7fba940fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:35.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.187+0000 7fba99740700 1 -- 172.21.15.93:0/2189635750 shutdown_connections 2023-11-16T16:48:35.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.187+0000 7fba99740700 1 -- 172.21.15.93:0/2189635750 wait complete. 2023-11-16T16:48:35.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.188+0000 7fba99740700 1 Processor -- start 2023-11-16T16:48:35.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.188+0000 7fba99740700 1 -- start start 2023-11-16T16:48:35.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.188+0000 7fba99740700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba941002f0 0x7fba94198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:35.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.188+0000 7fba99740700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba941014f0 0x7fba94198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:35.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.188+0000 7fba99740700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba94101e80 0x7fba9419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:35.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.188+0000 7fba99740700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9419fed0 con 0x7fba941014f0 2023-11-16T16:48:35.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.188+0000 7fba99740700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba941a0010 con 0x7fba941002f0 2023-11-16T16:48:35.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.188+0000 7fba99740700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba941a01a0 con 0x7fba94101e80 2023-11-16T16:48:35.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.188+0000 7fba927fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba941014f0 0x7fba94198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:35.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.189+0000 7fba92ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba941002f0 0x7fba94198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:35.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.189+0000 7fba92ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba941002f0 0x7fba94198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:37412/0 (socket says 172.21.15.93:37412) 2023-11-16T16:48:35.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.189+0000 7fba92ffd700 1 -- 172.21.15.93:0/73945155 learned_addr learned my addr 172.21.15.93:0/73945155 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:35.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.189+0000 7fba937fe700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba94101e80 0x7fba9419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:35.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.189+0000 7fba92ffd700 1 -- 172.21.15.93:0/73945155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba94101e80 msgr2=0x7fba9419f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:35.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.189+0000 7fba92ffd700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba94101e80 0x7fba9419f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:35.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.189+0000 7fba92ffd700 1 -- 172.21.15.93:0/73945155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba941014f0 msgr2=0x7fba94198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:35.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.189+0000 7fba92ffd700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba941014f0 0x7fba94198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:35.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.189+0000 7fba92ffd700 1 -- 172.21.15.93:0/73945155 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c009d00 con 0x7fba941002f0 2023-11-16T16:48:35.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.189+0000 7fba927fc700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba941014f0 0x7fba94198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:35.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.189+0000 7fba92ffd700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba941002f0 0x7fba94198340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fba8400e970 tx=0x7fba8400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:35.195 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.189+0000 7fba7bfff700 1 -- 172.21.15.93:0/73945155 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84009790 con 0x7fba941002f0 2023-11-16T16:48:35.195 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.190+0000 7fba7bfff700 1 -- 172.21.15.93:0/73945155 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba8400c7e0 con 0x7fba941002f0 2023-11-16T16:48:35.195 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.190+0000 7fba99740700 1 -- 172.21.15.93:0/73945155 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9410bc90 con 0x7fba941002f0 2023-11-16T16:48:35.195 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.190+0000 7fba7bfff700 1 -- 172.21.15.93:0/73945155 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84003c70 con 0x7fba941002f0 2023-11-16T16:48:35.195 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.190+0000 7fba99740700 1 -- 172.21.15.93:0/73945155 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9410bb00 con 0x7fba941002f0 2023-11-16T16:48:35.195 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.191+0000 7fba99740700 1 -- 172.21.15.93:0/73945155 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba94066ea0 con 0x7fba941002f0 2023-11-16T16:48:35.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.193+0000 7fba7bfff700 1 -- 172.21.15.93:0/73945155 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fba8400c950 con 0x7fba941002f0 2023-11-16T16:48:35.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.194+0000 7fba7bfff700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fba7c0a1150 0x7fba7c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:35.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.194+0000 7fba7bfff700 1 -- 172.21.15.93:0/73945155 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fba84014070 con 0x7fba941002f0 2023-11-16T16:48:35.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.194+0000 7fba927fc700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fba7c0a1150 0x7fba7c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:35.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.195+0000 7fba927fc700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fba7c0a1150 0x7fba7c0a3600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba88011490 tx=0x7fba88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:35.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.198+0000 7fba7bfff700 1 -- 172.21.15.93:0/73945155 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fba84099ce0 con 0x7fba941002f0 2023-11-16T16:48:35.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.396+0000 7fba99740700 1 -- 172.21.15.93:0/73945155 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fba941a13d0 con 0x7fba941002f0 2023-11-16T16:48:35.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.398+0000 7fba7bfff700 1 -- 172.21.15.93:0/73945155 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) v1 ==== 56+0+5761 (secure 0 0 0) 0x7fba84099870 con 0x7fba941002f0 2023-11-16T16:48:35.400 INFO:teuthology.orchestra.run.smithi093.stdout:epoch 92 2023-11-16T16:48:35.400 INFO:teuthology.orchestra.run.smithi093.stdout:fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 2023-11-16T16:48:35.400 INFO:teuthology.orchestra.run.smithi093.stdout:created 2023-11-16T16:38:54.903278+0000 2023-11-16T16:48:35.400 INFO:teuthology.orchestra.run.smithi093.stdout:modified 2023-11-16T16:47:53.331398+0000 2023-11-16T16:48:35.400 INFO:teuthology.orchestra.run.smithi093.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-11-16T16:48:35.400 INFO:teuthology.orchestra.run.smithi093.stdout:crush_version 27 2023-11-16T16:48:35.400 INFO:teuthology.orchestra.run.smithi093.stdout:full_ratio 0.95 2023-11-16T16:48:35.400 INFO:teuthology.orchestra.run.smithi093.stdout:backfillfull_ratio 0.9 2023-11-16T16:48:35.400 INFO:teuthology.orchestra.run.smithi093.stdout:nearfull_ratio 0.85 2023-11-16T16:48:35.401 INFO:teuthology.orchestra.run.smithi093.stdout:require_min_compat_client luminous 2023-11-16T16:48:35.401 INFO:teuthology.orchestra.run.smithi093.stdout:min_compat_client luminous 2023-11-16T16:48:35.401 INFO:teuthology.orchestra.run.smithi093.stdout:require_osd_release reef 2023-11-16T16:48:35.401 INFO:teuthology.orchestra.run.smithi093.stdout:stretch_mode_enabled false 2023-11-16T16:48:35.401 INFO:teuthology.orchestra.run.smithi093.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-11-16T16:48:35.401 INFO:teuthology.orchestra.run.smithi093.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-11-16T16:48:35.401 INFO:teuthology.orchestra.run.smithi093.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-11-16T16:48:35.401 INFO:teuthology.orchestra.run.smithi093.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-11-16T16:48:35.401 INFO:teuthology.orchestra.run.smithi093.stdout:max_osd 12 2023-11-16T16:48:35.401 INFO:teuthology.orchestra.run.smithi093.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6802/1213182748,v1:172.21.15.93:6803/1213182748] [v2:172.21.15.93:6804/1213182748,v1:172.21.15.93:6805/1213182748] exists,up 329eef05-603a-4f45-be62-d6317cc25e60 2023-11-16T16:48:35.402 INFO:teuthology.orchestra.run.smithi093.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.125:6800/2084419115,v1:172.21.15.125:6801/2084419115] [v2:172.21.15.125:6802/2084419115,v1:172.21.15.125:6803/2084419115] exists,up b72319a2-0c12-4b08-b78f-85d3af107bca 2023-11-16T16:48:35.402 INFO:teuthology.orchestra.run.smithi093.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6800/189817992,v1:172.21.15.159:6801/189817992] [v2:172.21.15.159:6802/189817992,v1:172.21.15.159:6803/189817992] exists,up 0d17be2a-b82b-4a45-9015-c451f7884bc5 2023-11-16T16:48:35.402 INFO:teuthology.orchestra.run.smithi093.stdout:osd.3 up in weight 1 up_from 25 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6810/2350765600,v1:172.21.15.93:6811/2350765600] [v2:172.21.15.93:6812/2350765600,v1:172.21.15.93:6813/2350765600] exists,up 71e6b247-710a-4bbb-9f6e-6333b3ea25bd 2023-11-16T16:48:35.402 INFO:teuthology.orchestra.run.smithi093.stdout:osd.4 up in weight 1 up_from 33 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.125:6808/3405048088,v1:172.21.15.125:6809/3405048088] [v2:172.21.15.125:6810/3405048088,v1:172.21.15.125:6811/3405048088] exists,up b965a7b2-6838-4b3a-b469-e2165cc42d16 2023-11-16T16:48:35.402 INFO:teuthology.orchestra.run.smithi093.stdout:osd.5 up in weight 1 up_from 38 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6808/1380209909,v1:172.21.15.159:6809/1380209909] [v2:172.21.15.159:6810/1380209909,v1:172.21.15.159:6811/1380209909] exists,up c4277ae8-0ebf-4650-a0aa-3bd3f7f96968 2023-11-16T16:48:35.402 INFO:teuthology.orchestra.run.smithi093.stdout:osd.6 up in weight 1 up_from 46 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6818/1437083195,v1:172.21.15.93:6819/1437083195] [v2:172.21.15.93:6820/1437083195,v1:172.21.15.93:6821/1437083195] exists,up 5ee6be21-c42d-441f-ac51-099370cf27bb 2023-11-16T16:48:35.402 INFO:teuthology.orchestra.run.smithi093.stdout:osd.7 up in weight 1 up_from 51 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.125:6816/3103492922,v1:172.21.15.125:6817/3103492922] [v2:172.21.15.125:6818/3103492922,v1:172.21.15.125:6819/3103492922] exists,up ca36594b-cd4d-4d71-a448-b677bdc862aa 2023-11-16T16:48:35.402 INFO:teuthology.orchestra.run.smithi093.stdout:osd.8 up in weight 1 up_from 59 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6816/1960504083,v1:172.21.15.159:6817/1960504083] [v2:172.21.15.159:6818/1960504083,v1:172.21.15.159:6819/1960504083] exists,up e4ec0a32-5a4c-441b-97cb-548c992ec12d 2023-11-16T16:48:35.402 INFO:teuthology.orchestra.run.smithi093.stdout:osd.9 up in weight 1 up_from 67 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6826/594522260,v1:172.21.15.93:6827/594522260] [v2:172.21.15.93:6828/594522260,v1:172.21.15.93:6829/594522260] exists,up 0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed 2023-11-16T16:48:35.402 INFO:teuthology.orchestra.run.smithi093.stdout:osd.10 up in weight 1 up_from 72 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.125:6824/1757421100,v1:172.21.15.125:6825/1757421100] [v2:172.21.15.125:6826/1757421100,v1:172.21.15.125:6827/1757421100] exists,up 2d7c0597-8595-4c28-8b1b-12e74fc554eb 2023-11-16T16:48:35.403 INFO:teuthology.orchestra.run.smithi093.stdout:osd.11 up in weight 1 up_from 79 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6824/2996534422,v1:172.21.15.159:6825/2996534422] [v2:172.21.15.159:6826/2996534422,v1:172.21.15.159:6827/2996534422] exists,up 77e5e471-2cae-44cc-82e3-13888533f31d 2023-11-16T16:48:35.403 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 2.8 [2,11] 2023-11-16T16:48:35.403 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 2.b [2,4] 2023-11-16T16:48:35.403 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 2.1d [8,11] 2023-11-16T16:48:35.403 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.2 [1,2] 2023-11-16T16:48:35.403 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.6 [3,8] 2023-11-16T16:48:35.403 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.14 [4,6] 2023-11-16T16:48:35.403 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.17 [4,7] 2023-11-16T16:48:35.403 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.1a [5,7] 2023-11-16T16:48:35.403 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.1b [3,6] 2023-11-16T16:48:35.404 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.1d [4,8,5,9] 2023-11-16T16:48:35.404 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.24 [4,9] 2023-11-16T16:48:35.404 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.37 [1,9] 2023-11-16T16:48:35.404 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 4.2 [9,1] 2023-11-16T16:48:35.404 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 4.5 [5,10] 2023-11-16T16:48:35.404 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 4.10 [9,8] 2023-11-16T16:48:35.404 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 4.13 [9,8] 2023-11-16T16:48:35.404 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/1465890024 expires 2023-11-17T16:39:50.432397+0000 2023-11-16T16:48:35.404 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/3409666551 expires 2023-11-17T16:39:50.432397+0000 2023-11-16T16:48:35.404 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:6800/2198359920 expires 2023-11-17T16:39:50.432397+0000 2023-11-16T16:48:35.405 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:6801/772471105 expires 2023-11-17T16:39:24.569132+0000 2023-11-16T16:48:35.405 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:6800/772471105 expires 2023-11-17T16:39:24.569132+0000 2023-11-16T16:48:35.405 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:6801/2198359920 expires 2023-11-17T16:39:50.432397+0000 2023-11-16T16:48:35.405 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/438537209 expires 2023-11-17T16:39:24.569132+0000 2023-11-16T16:48:35.405 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/2518663109 expires 2023-11-17T16:39:24.569132+0000 2023-11-16T16:48:35.405 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/696516505 expires 2023-11-17T16:39:24.569132+0000 2023-11-16T16:48:35.405 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/3700969994 expires 2023-11-17T16:39:50.432397+0000 2023-11-16T16:48:35.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.400+0000 7fba99740700 1 -- 172.21.15.93:0/73945155 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fba7c0a1150 msgr2=0x7fba7c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:35.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.400+0000 7fba99740700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fba7c0a1150 0x7fba7c0a3600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba88011490 tx=0x7fba88009260 comp rx=0 tx=0).stop 2023-11-16T16:48:35.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.400+0000 7fba99740700 1 -- 172.21.15.93:0/73945155 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba941002f0 msgr2=0x7fba94198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:35.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.400+0000 7fba99740700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba941002f0 0x7fba94198340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fba8400e970 tx=0x7fba8400ec80 comp rx=0 tx=0).stop 2023-11-16T16:48:35.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.400+0000 7fba99740700 1 -- 172.21.15.93:0/73945155 shutdown_connections 2023-11-16T16:48:35.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.400+0000 7fba99740700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fba7c0a1150 0x7fba7c0a3600 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:35.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.400+0000 7fba99740700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba94101e80 0x7fba9419f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:35.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.400+0000 7fba99740700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba941002f0 0x7fba94198340 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:35.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.400+0000 7fba99740700 1 --2- 172.21.15.93:0/73945155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba941014f0 0x7fba94198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:35.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.400+0000 7fba99740700 1 -- 172.21.15.93:0/73945155 >> 172.21.15.93:0/73945155 conn(0x7fba940fb900 msgr2=0x7fba940fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:35.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.400+0000 7fba99740700 1 -- 172.21.15.93:0/73945155 shutdown_connections 2023-11-16T16:48:35.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:35.400+0000 7fba99740700 1 -- 172.21.15.93:0/73945155 wait complete. 2023-11-16T16:48:35.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:35 smithi093 ceph-mon[97863]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:48:35.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:35 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/73945155' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-16T16:48:35.940 INFO:teuthology.run_tasks:Running task fs.ready... 2023-11-16T16:48:35.952 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:48:35.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:35 smithi159 ceph-mon[131528]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:48:35.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:35 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/73945155' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-16T16:48:36.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:35 smithi125 ceph-mon[131076]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:48:36.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:35 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/73945155' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-16T16:48:36.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.153+0000 7fc06179d700 1 -- 172.21.15.93:0/3554486709 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc05c1000b0 msgr2=0x7fc05c1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:36.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.153+0000 7fc06179d700 1 --2- 172.21.15.93:0/3554486709 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc05c1000b0 0x7fc05c1004c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009990 tx=0x7fc04c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.154+0000 7fc06179d700 1 -- 172.21.15.93:0/3554486709 shutdown_connections 2023-11-16T16:48:36.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.154+0000 7fc06179d700 1 --2- 172.21.15.93:0/3554486709 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc05c1000b0 0x7fc05c1004c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.154+0000 7fc06179d700 1 --2- 172.21.15.93:0/3554486709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc05c101cf0 0x7fc05c108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.154+0000 7fc06179d700 1 --2- 172.21.15.93:0/3554486709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc05c101340 0x7fc05c1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.154+0000 7fc06179d700 1 -- 172.21.15.93:0/3554486709 >> 172.21.15.93:0/3554486709 conn(0x7fc05c0fb6d0 msgr2=0x7fc05c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:36.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.154+0000 7fc06179d700 1 -- 172.21.15.93:0/3554486709 shutdown_connections 2023-11-16T16:48:36.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.154+0000 7fc06179d700 1 -- 172.21.15.93:0/3554486709 wait complete. 2023-11-16T16:48:36.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.155+0000 7fc06179d700 1 Processor -- start 2023-11-16T16:48:36.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.155+0000 7fc06179d700 1 -- start start 2023-11-16T16:48:36.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.155+0000 7fc06179d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc05c1000b0 0x7fc05c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:36.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.155+0000 7fc06179d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc05c101340 0x7fc05c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:36.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.155+0000 7fc06179d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc05c101cf0 0x7fc05c19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:36.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.155+0000 7fc06179d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c19fb00 con 0x7fc05c101340 2023-11-16T16:48:36.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.155+0000 7fc06179d700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c19fc40 con 0x7fc05c1000b0 2023-11-16T16:48:36.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.155+0000 7fc06179d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c19fdd0 con 0x7fc05c101cf0 2023-11-16T16:48:36.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.155+0000 7fc05b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc05c101340 0x7fc05c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:36.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.155+0000 7fc05b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc05c101340 0x7fc05c1984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42228/0 (socket says 172.21.15.93:42228) 2023-11-16T16:48:36.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.155+0000 7fc05b7fe700 1 -- 172.21.15.93:0/1412675907 learned_addr learned my addr 172.21.15.93:0/1412675907 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:36.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.155+0000 7fc05bfff700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc05c1000b0 0x7fc05c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:36.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.156+0000 7fc060f9c700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc05c101cf0 0x7fc05c19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:36.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.156+0000 7fc05b7fe700 1 -- 172.21.15.93:0/1412675907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc05c101cf0 msgr2=0x7fc05c19f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:36.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.156+0000 7fc05b7fe700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc05c101cf0 0x7fc05c19f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.156+0000 7fc05b7fe700 1 -- 172.21.15.93:0/1412675907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc05c1000b0 msgr2=0x7fc05c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:36.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.156+0000 7fc05b7fe700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc05c1000b0 0x7fc05c197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.156+0000 7fc05b7fe700 1 -- 172.21.15.93:0/1412675907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c0095f0 con 0x7fc05c101340 2023-11-16T16:48:36.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.156+0000 7fc060f9c700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc05c101cf0 0x7fc05c19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:36.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.156+0000 7fc05b7fe700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc05c101340 0x7fc05c1984c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc05000e9a0 tx=0x7fc05000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:36.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.156+0000 7fc0597fa700 1 -- 172.21.15.93:0/1412675907 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050009790 con 0x7fc05c101340 2023-11-16T16:48:36.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.156+0000 7fc06179d700 1 -- 172.21.15.93:0/1412675907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05c10baa0 con 0x7fc05c101340 2023-11-16T16:48:36.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.156+0000 7fc0597fa700 1 -- 172.21.15.93:0/1412675907 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc05000cbd0 con 0x7fc05c101340 2023-11-16T16:48:36.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.156+0000 7fc06179d700 1 -- 172.21.15.93:0/1412675907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05c10b910 con 0x7fc05c101340 2023-11-16T16:48:36.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.157+0000 7fc0597fa700 1 -- 172.21.15.93:0/1412675907 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050018620 con 0x7fc05c101340 2023-11-16T16:48:36.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.157+0000 7fc06179d700 1 -- 172.21.15.93:0/1412675907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05c1926f0 con 0x7fc05c101340 2023-11-16T16:48:36.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.159+0000 7fc0597fa700 1 -- 172.21.15.93:0/1412675907 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc05000cd40 con 0x7fc05c101340 2023-11-16T16:48:36.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.161+0000 7fc0597fa700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc0440a1150 0x7fc0440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:36.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.162+0000 7fc0597fa700 1 -- 172.21.15.93:0/1412675907 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fc050014070 con 0x7fc05c101340 2023-11-16T16:48:36.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.162+0000 7fc05bfff700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc0440a1150 0x7fc0440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:36.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.162+0000 7fc05bfff700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc0440a1150 0x7fc0440a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00b2e0 tx=0x7fc04c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:36.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.166+0000 7fc0597fa700 1 -- 172.21.15.93:0/1412675907 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc050051730 con 0x7fc05c101340 2023-11-16T16:48:36.368 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.367+0000 7fc06179d700 1 -- 172.21.15.93:0/1412675907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc05c067980 con 0x7fc05c101340 2023-11-16T16:48:36.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.369+0000 7fc0597fa700 1 -- 172.21.15.93:0/1412675907 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24795 (secure 0 0 0) 0x7fc0500962b0 con 0x7fc05c101340 2023-11-16T16:48:36.371 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:36.371 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":92,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:47:53.331398+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":1,"to":2}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:48:36.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.371+0000 7fc06179d700 1 -- 172.21.15.93:0/1412675907 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc0440a1150 msgr2=0x7fc0440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:36.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.371+0000 7fc06179d700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc0440a1150 0x7fc0440a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00b2e0 tx=0x7fc04c005fb0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.371+0000 7fc06179d700 1 -- 172.21.15.93:0/1412675907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc05c101340 msgr2=0x7fc05c1984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:36.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.371+0000 7fc06179d700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc05c101340 0x7fc05c1984c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc05000e9a0 tx=0x7fc05000ecb0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.371+0000 7fc06179d700 1 -- 172.21.15.93:0/1412675907 shutdown_connections 2023-11-16T16:48:36.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.371+0000 7fc06179d700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc0440a1150 0x7fc0440a3600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.371+0000 7fc06179d700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc05c101cf0 0x7fc05c19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.371+0000 7fc06179d700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc05c1000b0 0x7fc05c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.371+0000 7fc06179d700 1 --2- 172.21.15.93:0/1412675907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc05c101340 0x7fc05c1984c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.371+0000 7fc06179d700 1 -- 172.21.15.93:0/1412675907 >> 172.21.15.93:0/1412675907 conn(0x7fc05c0fb6d0 msgr2=0x7fc05c106c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:36.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.372+0000 7fc06179d700 1 -- 172.21.15.93:0/1412675907 shutdown_connections 2023-11-16T16:48:36.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.372+0000 7fc06179d700 1 -- 172.21.15.93:0/1412675907 wait complete. 2023-11-16T16:48:36.393 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:42:13.029135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:46:35.301321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:46:36.552545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-16T16:46:39.628380+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-16T16:48:36.393 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:48:36.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.592+0000 7fbef7fff700 1 -- 172.21.15.93:0/735082933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbef0103660 msgr2=0x7fbef0105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:36.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.592+0000 7fbef7fff700 1 --2- 172.21.15.93:0/735082933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbef0103660 0x7fbef0105a40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009990 tx=0x7fbee0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.592+0000 7fbef7fff700 1 -- 172.21.15.93:0/735082933 shutdown_connections 2023-11-16T16:48:36.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.592+0000 7fbef7fff700 1 --2- 172.21.15.93:0/735082933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbef0105f80 0x7fbef01083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.592+0000 7fbef7fff700 1 --2- 172.21.15.93:0/735082933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef0100d40 0x7fbef0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.592+0000 7fbef7fff700 1 --2- 172.21.15.93:0/735082933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbef0103660 0x7fbef0105a40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.592+0000 7fbef7fff700 1 -- 172.21.15.93:0/735082933 >> 172.21.15.93:0/735082933 conn(0x7fbef00fa7c0 msgr2=0x7fbef00fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:36.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.592+0000 7fbef7fff700 1 -- 172.21.15.93:0/735082933 shutdown_connections 2023-11-16T16:48:36.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.592+0000 7fbef7fff700 1 -- 172.21.15.93:0/735082933 wait complete. 2023-11-16T16:48:36.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.593+0000 7fbef7fff700 1 Processor -- start 2023-11-16T16:48:36.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.593+0000 7fbef7fff700 1 -- start start 2023-11-16T16:48:36.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.593+0000 7fbef7fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef0100d40 0x7fbef01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:36.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.593+0000 7fbef7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbef0103660 0x7fbef0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.593+0000 7fbef7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbef0105f80 0x7fbef019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.593+0000 7fbef7fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef019fe40 con 0x7fbef0105f80 2023-11-16T16:48:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.593+0000 7fbef7fff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef019ff80 con 0x7fbef0100d40 2023-11-16T16:48:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.593+0000 7fbef7fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01a0110 con 0x7fbef0103660 2023-11-16T16:48:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef5d9b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef0100d40 0x7fbef01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef559a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbef0103660 0x7fbef0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbef0105f80 0x7fbef019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef5d9b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef0100d40 0x7fbef01982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:37454/0 (socket says 172.21.15.93:37454) 2023-11-16T16:48:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef559a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbef0103660 0x7fbef0198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:36302/0 (socket says 172.21.15.93:36302) 2023-11-16T16:48:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef5d9b700 1 -- 172.21.15.93:0/4119570244 learned_addr learned my addr 172.21.15.93:0/4119570244 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef5d9b700 1 -- 172.21.15.93:0/4119570244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbef0103660 msgr2=0x7fbef0198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:36.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef5d9b700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbef0103660 0x7fbef0198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef5d9b700 1 -- 172.21.15.93:0/4119570244 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbef0105f80 msgr2=0x7fbef019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:36.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef5d9b700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbef0105f80 0x7fbef019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef5d9b700 1 -- 172.21.15.93:0/4119570244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee00095f0 con 0x7fbef0100d40 2023-11-16T16:48:36.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef659c700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbef0105f80 0x7fbef019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:36.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.594+0000 7fbef5d9b700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef0100d40 0x7fbef01982c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00e9a0 tx=0x7fbeec00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:36.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.595+0000 7fbee6ffd700 1 -- 172.21.15.93:0/4119570244 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec009790 con 0x7fbef0100d40 2023-11-16T16:48:36.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.595+0000 7fbef7fff700 1 -- 172.21.15.93:0/4119570244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef010bca0 con 0x7fbef0100d40 2023-11-16T16:48:36.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.595+0000 7fbee6ffd700 1 -- 172.21.15.93:0/4119570244 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeec010430 con 0x7fbef0100d40 2023-11-16T16:48:36.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.595+0000 7fbee6ffd700 1 -- 172.21.15.93:0/4119570244 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec018680 con 0x7fbef0100d40 2023-11-16T16:48:36.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.595+0000 7fbef7fff700 1 -- 172.21.15.93:0/4119570244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef010bb10 con 0x7fbef0100d40 2023-11-16T16:48:36.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.596+0000 7fbef7fff700 1 -- 172.21.15.93:0/4119570244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef0107af0 con 0x7fbef0100d40 2023-11-16T16:48:36.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.598+0000 7fbee6ffd700 1 -- 172.21.15.93:0/4119570244 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbeec0105a0 con 0x7fbef0100d40 2023-11-16T16:48:36.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.600+0000 7fbee6ffd700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbedc0a1150 0x7fbedc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:36.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.600+0000 7fbee6ffd700 1 -- 172.21.15.93:0/4119570244 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fbeec014070 con 0x7fbef0100d40 2023-11-16T16:48:36.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.600+0000 7fbef559a700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbedc0a1150 0x7fbedc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:36.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.601+0000 7fbef559a700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbedc0a1150 0x7fbedc0a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbee0000c00 tx=0x7fbee0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:36.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.602+0000 7fbee6ffd700 1 -- 172.21.15.93:0/4119570244 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbeec096410 con 0x7fbef0100d40 2023-11-16T16:48:36.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:36 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1412675907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:36.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.805+0000 7fbef7fff700 1 -- 172.21.15.93:0/4119570244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbef0067b00 con 0x7fbef0100d40 2023-11-16T16:48:36.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.805+0000 7fbee6ffd700 1 -- 172.21.15.93:0/4119570244 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbeec0260a0 con 0x7fbef0100d40 2023-11-16T16:48:36.807 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-11-16T16:48:36.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.808+0000 7fbef7fff700 1 -- 172.21.15.93:0/4119570244 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbedc0a1150 msgr2=0x7fbedc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:36.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.808+0000 7fbef7fff700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbedc0a1150 0x7fbedc0a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbee0000c00 tx=0x7fbee0005fb0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.808+0000 7fbef7fff700 1 -- 172.21.15.93:0/4119570244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef0100d40 msgr2=0x7fbef01982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:36.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.808+0000 7fbef7fff700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef0100d40 0x7fbef01982c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00e9a0 tx=0x7fbeec00ecb0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.808+0000 7fbef7fff700 1 -- 172.21.15.93:0/4119570244 shutdown_connections 2023-11-16T16:48:36.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.808+0000 7fbef7fff700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbedc0a1150 0x7fbedc0a3600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.808+0000 7fbef7fff700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbef0105f80 0x7fbef019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.808+0000 7fbef7fff700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef0100d40 0x7fbef01982c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.808+0000 7fbef7fff700 1 --2- 172.21.15.93:0/4119570244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbef0103660 0x7fbef0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:36.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.808+0000 7fbef7fff700 1 -- 172.21.15.93:0/4119570244 >> 172.21.15.93:0/4119570244 conn(0x7fbef00fa7c0 msgr2=0x7fbef01019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:36.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.808+0000 7fbef7fff700 1 -- 172.21.15.93:0/4119570244 shutdown_connections 2023-11-16T16:48:36.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:36.808+0000 7fbef7fff700 1 -- 172.21.15.93:0/4119570244 wait complete. 2023-11-16T16:48:36.828 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:48:36.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:36 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1412675907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:37.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.027+0000 7f18921ff700 1 -- 172.21.15.93:0/3558738290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f188c073550 msgr2=0x7f188c0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.027+0000 7f18921ff700 1 --2- 172.21.15.93:0/3558738290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f188c073550 0x7f188c0739c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1874009990 tx=0x7f1874009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.027+0000 7f18921ff700 1 -- 172.21.15.93:0/3558738290 shutdown_connections 2023-11-16T16:48:37.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.027+0000 7f18921ff700 1 --2- 172.21.15.93:0/3558738290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f188c10c890 0x7f188c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.027+0000 7f18921ff700 1 --2- 172.21.15.93:0/3558738290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f188c074b30 0x7f188c072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.027+0000 7f18921ff700 1 --2- 172.21.15.93:0/3558738290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f188c073550 0x7f188c0739c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.028+0000 7f18921ff700 1 -- 172.21.15.93:0/3558738290 >> 172.21.15.93:0/3558738290 conn(0x7f188c0fb8a0 msgr2=0x7f188c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:37.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.028+0000 7f18921ff700 1 -- 172.21.15.93:0/3558738290 shutdown_connections 2023-11-16T16:48:37.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.028+0000 7f18921ff700 1 -- 172.21.15.93:0/3558738290 wait complete. 2023-11-16T16:48:37.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f18921ff700 1 Processor -- start 2023-11-16T16:48:37.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f18921ff700 1 -- start start 2023-11-16T16:48:37.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f18921ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f188c073550 0x7f188c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:37.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f18921ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f188c074b30 0x7f188c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:37.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f18921ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f188c10c890 0x7f188c1a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:37.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f18921ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1a4210 con 0x7f188c10c890 2023-11-16T16:48:37.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f18921ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1a4350 con 0x7f188c074b30 2023-11-16T16:48:37.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f18921ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1a44e0 con 0x7f188c073550 2023-11-16T16:48:37.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f188bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f188c10c890 0x7f188c1a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:37.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f188bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f188c10c890 0x7f188c1a3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38948/0 (socket says 172.21.15.93:38948) 2023-11-16T16:48:37.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f188bfff700 1 -- 172.21.15.93:0/1958357595 learned_addr learned my addr 172.21.15.93:0/1958357595 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:37.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f1883fff700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f188c074b30 0x7f188c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:37.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.029+0000 7f188b7fe700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f188c073550 0x7f188c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:37.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.030+0000 7f188bfff700 1 -- 172.21.15.93:0/1958357595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f188c073550 msgr2=0x7f188c19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.030+0000 7f188bfff700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f188c073550 0x7f188c19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.030+0000 7f188bfff700 1 -- 172.21.15.93:0/1958357595 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f188c074b30 msgr2=0x7f188c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.030+0000 7f188bfff700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f188c074b30 0x7f188c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.030+0000 7f188bfff700 1 -- 172.21.15.93:0/1958357595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18740095f0 con 0x7f188c10c890 2023-11-16T16:48:37.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.030+0000 7f188bfff700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f188c10c890 0x7f188c1a3c40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1884007ee0 tx=0x7f188400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:37.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.030+0000 7f18897fa700 1 -- 172.21.15.93:0/1958357595 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188400dd20 con 0x7f188c10c890 2023-11-16T16:48:37.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.030+0000 7f18897fa700 1 -- 172.21.15.93:0/1958357595 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f188400f040 con 0x7f188c10c890 2023-11-16T16:48:37.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.030+0000 7f18921ff700 1 -- 172.21.15.93:0/1958357595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188c1122a0 con 0x7f188c10c890 2023-11-16T16:48:37.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.030+0000 7f18897fa700 1 -- 172.21.15.93:0/1958357595 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18840048a0 con 0x7f188c10c890 2023-11-16T16:48:37.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.030+0000 7f18921ff700 1 -- 172.21.15.93:0/1958357595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c112110 con 0x7f188c10c890 2023-11-16T16:48:37.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.031+0000 7f18921ff700 1 -- 172.21.15.93:0/1958357595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c1023e0 con 0x7f188c10c890 2023-11-16T16:48:37.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.033+0000 7f18897fa700 1 -- 172.21.15.93:0/1958357595 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1884003a00 con 0x7f188c10c890 2023-11-16T16:48:37.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.034+0000 7f18897fa700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f18780a1150 0x7f18780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:37.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.035+0000 7f18897fa700 1 -- 172.21.15.93:0/1958357595 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f18840cc550 con 0x7f188c10c890 2023-11-16T16:48:37.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.035+0000 7f188b7fe700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f18780a1150 0x7f18780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:37.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.035+0000 7f188b7fe700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f18780a1150 0x7f18780a3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f187c005950 tx=0x7f187c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:37.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.038+0000 7f18897fa700 1 -- 172.21.15.93:0/1958357595 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f188404e730 con 0x7f188c10c890 2023-11-16T16:48:37.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:36 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1412675907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:37.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.241+0000 7f18921ff700 1 -- 172.21.15.93:0/1958357595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f188c061700 con 0x7f188c10c890 2023-11-16T16:48:37.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.241+0000 7f18897fa700 1 -- 172.21.15.93:0/1958357595 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1884095870 con 0x7f188c10c890 2023-11-16T16:48:37.243 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:48:37.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.244+0000 7f18921ff700 1 -- 172.21.15.93:0/1958357595 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f18780a1150 msgr2=0x7f18780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.244+0000 7f18921ff700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f18780a1150 0x7f18780a3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f187c005950 tx=0x7f187c00b410 comp rx=0 tx=0).stop 2023-11-16T16:48:37.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.244+0000 7f18921ff700 1 -- 172.21.15.93:0/1958357595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f188c10c890 msgr2=0x7f188c1a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.244+0000 7f18921ff700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f188c10c890 0x7f188c1a3c40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1884007ee0 tx=0x7f188400d3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.244+0000 7f18921ff700 1 -- 172.21.15.93:0/1958357595 shutdown_connections 2023-11-16T16:48:37.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.244+0000 7f18921ff700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f18780a1150 0x7f18780a3600 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.244+0000 7f18921ff700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f188c10c890 0x7f188c1a3c40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.244+0000 7f18921ff700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f188c073550 0x7f188c19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.244+0000 7f18921ff700 1 --2- 172.21.15.93:0/1958357595 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f188c074b30 0x7f188c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.244+0000 7f18921ff700 1 -- 172.21.15.93:0/1958357595 >> 172.21.15.93:0/1958357595 conn(0x7f188c0fb8a0 msgr2=0x7f188c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:37.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.244+0000 7f18921ff700 1 -- 172.21.15.93:0/1958357595 shutdown_connections 2023-11-16T16:48:37.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.244+0000 7f18921ff700 1 -- 172.21.15.93:0/1958357595 wait complete. 2023-11-16T16:48:37.264 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:48:37.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.463+0000 7f0f6adff700 1 -- 172.21.15.93:0/1816384902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f64074b30 msgr2=0x7f0f64072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.463+0000 7f0f6adff700 1 --2- 172.21.15.93:0/1816384902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f64074b30 0x7f0f64072f80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0f58009990 tx=0x7f0f58009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.463+0000 7f0f6adff700 1 -- 172.21.15.93:0/1816384902 shutdown_connections 2023-11-16T16:48:37.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.463+0000 7f0f6adff700 1 --2- 172.21.15.93:0/1816384902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f64074b30 0x7f0f64072f80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.463+0000 7f0f6adff700 1 --2- 172.21.15.93:0/1816384902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f6410c890 0x7f0f6410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.463+0000 7f0f6adff700 1 --2- 172.21.15.93:0/1816384902 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f64073550 0x7f0f640739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.463+0000 7f0f6adff700 1 -- 172.21.15.93:0/1816384902 >> 172.21.15.93:0/1816384902 conn(0x7f0f640fb8a0 msgr2=0x7f0f640fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:37.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.463+0000 7f0f6adff700 1 -- 172.21.15.93:0/1816384902 shutdown_connections 2023-11-16T16:48:37.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.463+0000 7f0f6adff700 1 -- 172.21.15.93:0/1816384902 wait complete. 2023-11-16T16:48:37.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f6adff700 1 Processor -- start 2023-11-16T16:48:37.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f6adff700 1 -- start start 2023-11-16T16:48:37.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f6adff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f64073550 0x7f0f6419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:37.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f6adff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f64074b30 0x7f0f6419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:37.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f6adff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f6410c890 0x7f0f641a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:37.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f6adff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f641a41f0 con 0x7f0f64074b30 2023-11-16T16:48:37.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f6adff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f641a4330 con 0x7f0f64073550 2023-11-16T16:48:37.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f6adff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f641a44c0 con 0x7f0f6410c890 2023-11-16T16:48:37.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f63fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f64074b30 0x7f0f6419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:37.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f63fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f64074b30 0x7f0f6419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38966/0 (socket says 172.21.15.93:38966) 2023-11-16T16:48:37.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f68b9b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f64073550 0x7f0f6419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:37.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f63fff700 1 -- 172.21.15.93:0/2121802655 learned_addr learned my addr 172.21.15.93:0/2121802655 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:37.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.464+0000 7f0f6939c700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f6410c890 0x7f0f641a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:37.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.465+0000 7f0f68b9b700 1 -- 172.21.15.93:0/2121802655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f6410c890 msgr2=0x7f0f641a3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.465+0000 7f0f68b9b700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f6410c890 0x7f0f641a3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.465+0000 7f0f68b9b700 1 -- 172.21.15.93:0/2121802655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f64074b30 msgr2=0x7f0f6419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.465+0000 7f0f68b9b700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f64074b30 0x7f0f6419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.465+0000 7f0f68b9b700 1 -- 172.21.15.93:0/2121802655 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580095f0 con 0x7f0f64073550 2023-11-16T16:48:37.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.465+0000 7f0f63fff700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f64074b30 0x7f0f6419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:37.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.465+0000 7f0f68b9b700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f64073550 0x7f0f6419c670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f580098e0 tx=0x7f0f58009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:37.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.465+0000 7f0f61ffb700 1 -- 172.21.15.93:0/2121802655 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5801d070 con 0x7f0f64073550 2023-11-16T16:48:37.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.465+0000 7f0f6adff700 1 -- 172.21.15.93:0/2121802655 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f64112280 con 0x7f0f64073550 2023-11-16T16:48:37.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.465+0000 7f0f61ffb700 1 -- 172.21.15.93:0/2121802655 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f58021410 con 0x7f0f64073550 2023-11-16T16:48:37.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.465+0000 7f0f61ffb700 1 -- 172.21.15.93:0/2121802655 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58005720 con 0x7f0f64073550 2023-11-16T16:48:37.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.465+0000 7f0f6adff700 1 -- 172.21.15.93:0/2121802655 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f641120f0 con 0x7f0f64073550 2023-11-16T16:48:37.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.466+0000 7f0f6adff700 1 -- 172.21.15.93:0/2121802655 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f641023e0 con 0x7f0f64073550 2023-11-16T16:48:37.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.468+0000 7f0f61ffb700 1 -- 172.21.15.93:0/2121802655 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0f58021580 con 0x7f0f64073550 2023-11-16T16:48:37.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.470+0000 7f0f61ffb700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f0f540a1150 0x7f0f540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:37.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.470+0000 7f0f61ffb700 1 -- 172.21.15.93:0/2121802655 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f0f580ce620 con 0x7f0f64073550 2023-11-16T16:48:37.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.470+0000 7f0f63fff700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f0f540a1150 0x7f0f540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:37.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.471+0000 7f0f63fff700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f0f540a1150 0x7f0f540a3600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f50005950 tx=0x7f0f50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:37.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.472+0000 7f0f61ffb700 1 -- 172.21.15.93:0/2121802655 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0f580979d0 con 0x7f0f64073550 2023-11-16T16:48:37.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.679+0000 7f0f6adff700 1 -- 172.21.15.93:0/2121802655 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f64061700 con 0x7f0f64073550 2023-11-16T16:48:37.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.680+0000 7f0f61ffb700 1 -- 172.21.15.93:0/2121802655 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f5809aff0 con 0x7f0f64073550 2023-11-16T16:48:37.681 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-11-16T16:48:37.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.682+0000 7f0f6adff700 1 -- 172.21.15.93:0/2121802655 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f0f540a1150 msgr2=0x7f0f540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.682+0000 7f0f6adff700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f0f540a1150 0x7f0f540a3600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f50005950 tx=0x7f0f50009260 comp rx=0 tx=0).stop 2023-11-16T16:48:37.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.682+0000 7f0f6adff700 1 -- 172.21.15.93:0/2121802655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f64073550 msgr2=0x7f0f6419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.682+0000 7f0f6adff700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f64073550 0x7f0f6419c670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f580098e0 tx=0x7f0f58009580 comp rx=0 tx=0).stop 2023-11-16T16:48:37.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.682+0000 7f0f6adff700 1 -- 172.21.15.93:0/2121802655 shutdown_connections 2023-11-16T16:48:37.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.682+0000 7f0f6adff700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f0f540a1150 0x7f0f540a3600 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.682+0000 7f0f6adff700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f6410c890 0x7f0f641a3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.682+0000 7f0f6adff700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f64073550 0x7f0f6419c670 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.682+0000 7f0f6adff700 1 --2- 172.21.15.93:0/2121802655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f64074b30 0x7f0f6419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.682+0000 7f0f6adff700 1 -- 172.21.15.93:0/2121802655 >> 172.21.15.93:0/2121802655 conn(0x7f0f640fb8a0 msgr2=0x7f0f640fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:37.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.682+0000 7f0f6adff700 1 -- 172.21.15.93:0/2121802655 shutdown_connections 2023-11-16T16:48:37.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.683+0000 7f0f6adff700 1 -- 172.21.15.93:0/2121802655 wait complete. 2023-11-16T16:48:37.703 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-16T16:48:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:37 smithi093 ceph-mon[97863]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:48:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:37 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/4119570244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:48:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:37 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1958357595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:48:37.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.903+0000 7f5112bff700 1 -- 172.21.15.93:0/3945001736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c1014c0 msgr2=0x7f510c101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.903+0000 7f5112bff700 1 --2- 172.21.15.93:0/3945001736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c1014c0 0x7f510c101910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009990 tx=0x7f50f8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.903+0000 7f5112bff700 1 -- 172.21.15.93:0/3945001736 shutdown_connections 2023-11-16T16:48:37.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.903+0000 7f5112bff700 1 --2- 172.21.15.93:0/3945001736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f510c101e50 0x7f510c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.903+0000 7f5112bff700 1 --2- 172.21.15.93:0/3945001736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f510c1002d0 0x7f510c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.903+0000 7f5112bff700 1 --2- 172.21.15.93:0/3945001736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c1014c0 0x7f510c101910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.903+0000 7f5112bff700 1 -- 172.21.15.93:0/3945001736 >> 172.21.15.93:0/3945001736 conn(0x7f510c0fb8b0 msgr2=0x7f510c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:37.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.903+0000 7f5112bff700 1 -- 172.21.15.93:0/3945001736 shutdown_connections 2023-11-16T16:48:37.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.903+0000 7f5112bff700 1 -- 172.21.15.93:0/3945001736 wait complete. 2023-11-16T16:48:37.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.904+0000 7f5112bff700 1 Processor -- start 2023-11-16T16:48:37.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.904+0000 7f5112bff700 1 -- start start 2023-11-16T16:48:37.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f5112bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c1002d0 0x7f510c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f5112bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f510c1014c0 0x7f510c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f5112bff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f510c101e50 0x7f510c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f5112bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c073bc0 con 0x7f510c1002d0 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f5112bff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c073d30 con 0x7f510c101e50 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f5112bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c1a00f0 con 0x7f510c1014c0 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f511099b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c1002d0 0x7f510c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f511099b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c1002d0 0x7f510c0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38982/0 (socket says 172.21.15.93:38982) 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f511099b700 1 -- 172.21.15.93:0/1124056184 learned_addr learned my addr 172.21.15.93:0/1124056184 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f510bfff700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f510c1014c0 0x7f510c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f511119c700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f510c101e50 0x7f510c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f511099b700 1 -- 172.21.15.93:0/1124056184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f510c1014c0 msgr2=0x7f510c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f511099b700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f510c1014c0 0x7f510c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f511099b700 1 -- 172.21.15.93:0/1124056184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f510c101e50 msgr2=0x7f510c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:37.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f511099b700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f510c101e50 0x7f510c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:37.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f511099b700 1 -- 172.21.15.93:0/1124056184 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80095f0 con 0x7f510c1002d0 2023-11-16T16:48:37.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.905+0000 7f511119c700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f510c101e50 0x7f510c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:37.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.906+0000 7f511099b700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c1002d0 0x7f510c0748a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f510000d980 tx=0x7f510000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:37.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.906+0000 7f5109ffb700 1 -- 172.21.15.93:0/1124056184 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000b950 con 0x7f510c1002d0 2023-11-16T16:48:37.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.906+0000 7f5109ffb700 1 -- 172.21.15.93:0/1124056184 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5100007620 con 0x7f510c1002d0 2023-11-16T16:48:37.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.906+0000 7f5112bff700 1 -- 172.21.15.93:0/1124056184 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510c10bc80 con 0x7f510c1002d0 2023-11-16T16:48:37.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.906+0000 7f5109ffb700 1 -- 172.21.15.93:0/1124056184 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000f3c0 con 0x7f510c1002d0 2023-11-16T16:48:37.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.906+0000 7f5112bff700 1 -- 172.21.15.93:0/1124056184 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510c10baf0 con 0x7f510c1002d0 2023-11-16T16:48:37.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.907+0000 7f5112bff700 1 -- 172.21.15.93:0/1124056184 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510c107250 con 0x7f510c1002d0 2023-11-16T16:48:37.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.909+0000 7f5109ffb700 1 -- 172.21.15.93:0/1124056184 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5100010640 con 0x7f510c1002d0 2023-11-16T16:48:37.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.910+0000 7f5109ffb700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f50fc0a11d0 0x7f50fc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:37.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.910+0000 7f5109ffb700 1 -- 172.21.15.93:0/1124056184 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f51000cd410 con 0x7f510c1002d0 2023-11-16T16:48:37.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.911+0000 7f510bfff700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f50fc0a11d0 0x7f50fc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:37.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.911+0000 7f510bfff700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f50fc0a11d0 0x7f50fc0a3680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f50f800b2e0 tx=0x7f50f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:37.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:37.912+0000 7f5109ffb700 1 -- 172.21.15.93:0/1124056184 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5100096730 con 0x7f510c1002d0 2023-11-16T16:48:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:37 smithi159 ceph-mon[131528]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:48:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:37 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/4119570244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:48:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:37 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1958357595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:48:38.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:37 smithi125 ceph-mon[131076]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:48:38.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:37 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/4119570244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:48:38.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:37 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1958357595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:48:38.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.112+0000 7f5112bff700 1 -- 172.21.15.93:0/1124056184 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f510c061700 con 0x7f510c1002d0 2023-11-16T16:48:38.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.112+0000 7f5109ffb700 1 -- 172.21.15.93:0/1124056184 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f51000160a0 con 0x7f510c1002d0 2023-11-16T16:48:38.114 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:48:38.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.115+0000 7f5112bff700 1 -- 172.21.15.93:0/1124056184 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f50fc0a11d0 msgr2=0x7f50fc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:38.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.115+0000 7f5112bff700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f50fc0a11d0 0x7f50fc0a3680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f50f800b2e0 tx=0x7f50f8005fb0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.115+0000 7f5112bff700 1 -- 172.21.15.93:0/1124056184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c1002d0 msgr2=0x7f510c0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:38.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.115+0000 7f5112bff700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c1002d0 0x7f510c0748a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f510000d980 tx=0x7f510000dc90 comp rx=0 tx=0).stop 2023-11-16T16:48:38.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.115+0000 7f5112bff700 1 -- 172.21.15.93:0/1124056184 shutdown_connections 2023-11-16T16:48:38.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.115+0000 7f5112bff700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f50fc0a11d0 0x7f50fc0a3680 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.115+0000 7f5112bff700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c1002d0 0x7f510c0748a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.115+0000 7f5112bff700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f510c101e50 0x7f510c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.115+0000 7f5112bff700 1 --2- 172.21.15.93:0/1124056184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f510c1014c0 0x7f510c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.115+0000 7f5112bff700 1 -- 172.21.15.93:0/1124056184 >> 172.21.15.93:0/1124056184 conn(0x7f510c0fb8b0 msgr2=0x7f510c0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:38.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.115+0000 7f5112bff700 1 -- 172.21.15.93:0/1124056184 shutdown_connections 2023-11-16T16:48:38.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.115+0000 7f5112bff700 1 -- 172.21.15.93:0/1124056184 wait complete. 2023-11-16T16:48:38.136 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:48:38.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.330+0000 7fddd7fff700 1 -- 172.21.15.93:0/247472055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fddd0101e10 msgr2=0x7fddd01086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:38.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.330+0000 7fddd7fff700 1 --2- 172.21.15.93:0/247472055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fddd0101e10 0x7fddd01086d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fddc800b0a0 tx=0x7fddc800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.330+0000 7fddd7fff700 1 -- 172.21.15.93:0/247472055 shutdown_connections 2023-11-16T16:48:38.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.330+0000 7fddd7fff700 1 --2- 172.21.15.93:0/247472055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fddd0101e10 0x7fddd01086d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.330+0000 7fddd7fff700 1 --2- 172.21.15.93:0/247472055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddd0100290 0x7fddd01006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.330+0000 7fddd7fff700 1 --2- 172.21.15.93:0/247472055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddd0101480 0x7fddd01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.330+0000 7fddd7fff700 1 -- 172.21.15.93:0/247472055 >> 172.21.15.93:0/247472055 conn(0x7fddd00fb870 msgr2=0x7fddd00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:38.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.330+0000 7fddd7fff700 1 -- 172.21.15.93:0/247472055 shutdown_connections 2023-11-16T16:48:38.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.330+0000 7fddd7fff700 1 -- 172.21.15.93:0/247472055 wait complete. 2023-11-16T16:48:38.334 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.331+0000 7fddd7fff700 1 Processor -- start 2023-11-16T16:48:38.334 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.331+0000 7fddd7fff700 1 -- start start 2023-11-16T16:48:38.334 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddd0100290 0x7fddd0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:38.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fddd0101480 0x7fddd0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:38.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd7fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddd0101e10 0x7fddd019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:38.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd7fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd019fdb0 con 0x7fddd0101480 2023-11-16T16:48:38.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd7fff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd019fef0 con 0x7fddd0101e10 2023-11-16T16:48:38.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd7fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01a0080 con 0x7fddd0100290 2023-11-16T16:48:38.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd559a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fddd0101480 0x7fddd0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:38.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd559a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fddd0101480 0x7fddd0198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39000/0 (socket says 172.21.15.93:39000) 2023-11-16T16:48:38.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd559a700 1 -- 172.21.15.93:0/29051006 learned_addr learned my addr 172.21.15.93:0/29051006 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:38.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd5d9b700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddd0100290 0x7fddd0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:38.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd659c700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddd0101e10 0x7fddd019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:38.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd559a700 1 -- 172.21.15.93:0/29051006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddd0100290 msgr2=0x7fddd0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:38.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd559a700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddd0100290 0x7fddd0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd559a700 1 -- 172.21.15.93:0/29051006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddd0101e10 msgr2=0x7fddd019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:38.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd559a700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddd0101e10 0x7fddd019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd559a700 1 -- 172.21.15.93:0/29051006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc009520 con 0x7fddd0101480 2023-11-16T16:48:38.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.332+0000 7fddd659c700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddd0101e10 0x7fddd019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:38.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.333+0000 7fddd559a700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fddd0101480 0x7fddd0198770 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fddc000e9a0 tx=0x7fddc000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:38.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.333+0000 7fddc6ffd700 1 -- 172.21.15.93:0/29051006 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0009790 con 0x7fddd0101480 2023-11-16T16:48:38.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.333+0000 7fddc6ffd700 1 -- 172.21.15.93:0/29051006 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddc0010430 con 0x7fddd0101480 2023-11-16T16:48:38.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.333+0000 7fddd7fff700 1 -- 172.21.15.93:0/29051006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc8009d00 con 0x7fddd0101480 2023-11-16T16:48:38.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.333+0000 7fddc6ffd700 1 -- 172.21.15.93:0/29051006 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0018680 con 0x7fddd0101480 2023-11-16T16:48:38.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.333+0000 7fddd7fff700 1 -- 172.21.15.93:0/29051006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd010bc20 con 0x7fddd0101480 2023-11-16T16:48:38.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.334+0000 7fddd7fff700 1 -- 172.21.15.93:0/29051006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd0067a60 con 0x7fddd0101480 2023-11-16T16:48:38.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.336+0000 7fddc6ffd700 1 -- 172.21.15.93:0/29051006 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fddc00105a0 con 0x7fddd0101480 2023-11-16T16:48:38.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.338+0000 7fddc6ffd700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fddbc0a1150 0x7fddbc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:38.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.338+0000 7fddc6ffd700 1 -- 172.21.15.93:0/29051006 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fddc0014070 con 0x7fddd0101480 2023-11-16T16:48:38.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.338+0000 7fddd5d9b700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fddbc0a1150 0x7fddbc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:38.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.339+0000 7fddd5d9b700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fddbc0a1150 0x7fddbc0a3600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fddd01012e0 tx=0x7fddcc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:38.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.342+0000 7fddc6ffd700 1 -- 172.21.15.93:0/29051006 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fddc0051730 con 0x7fddd0101480 2023-11-16T16:48:38.583 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.582+0000 7fddd7fff700 1 -- 172.21.15.93:0/29051006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddd01a0c80 con 0x7fddd0101480 2023-11-16T16:48:38.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.583+0000 7fddc6ffd700 1 -- 172.21.15.93:0/29051006 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10811 (secure 0 0 0) 0x7fddc0096220 con 0x7fddd0101480 2023-11-16T16:48:38.584 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:38.584 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3614592201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3614592201},{"type":"v1","addr":"172.21.15.93:6839","nonce":3614592201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4068789362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4068789362},{"type":"v1","addr":"172.21.15.93:6841","nonce":4068789362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:46:37.124338+0000","modified":"2023-11-16T16:47:10.311966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24541,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/778982589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":778982589},{"type":"v1","addr":"172.21.15.159:6837","nonce":778982589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6839/149300707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":149300707},{"type":"v1","addr":"172.21.15.159:6839","nonce":149300707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/3590027328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":3590027328},{"type":"v1","addr":"172.21.15.125:6839","nonce":3590027328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:48:38.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.585+0000 7fddd7fff700 1 -- 172.21.15.93:0/29051006 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fddbc0a1150 msgr2=0x7fddbc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:38.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.585+0000 7fddd7fff700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fddbc0a1150 0x7fddbc0a3600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fddd01012e0 tx=0x7fddcc009260 comp rx=0 tx=0).stop 2023-11-16T16:48:38.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.585+0000 7fddd7fff700 1 -- 172.21.15.93:0/29051006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fddd0101480 msgr2=0x7fddd0198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:38.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.585+0000 7fddd7fff700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fddd0101480 0x7fddd0198770 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fddc000e9a0 tx=0x7fddc000ecb0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.585+0000 7fddd7fff700 1 -- 172.21.15.93:0/29051006 shutdown_connections 2023-11-16T16:48:38.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.585+0000 7fddd7fff700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fddbc0a1150 0x7fddbc0a3600 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.585+0000 7fddd7fff700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddd0101e10 0x7fddd019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.585+0000 7fddd7fff700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddd0100290 0x7fddd0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.585+0000 7fddd7fff700 1 --2- 172.21.15.93:0/29051006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fddd0101480 0x7fddd0198770 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.585+0000 7fddd7fff700 1 -- 172.21.15.93:0/29051006 >> 172.21.15.93:0/29051006 conn(0x7fddd00fb870 msgr2=0x7fddd0106d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:38.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.585+0000 7fddd7fff700 1 -- 172.21.15.93:0/29051006 shutdown_connections 2023-11-16T16:48:38.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.585+0000 7fddd7fff700 1 -- 172.21.15.93:0/29051006 wait complete. 2023-11-16T16:48:38.588 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 23 2023-11-16T16:48:38.607 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:48:38.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.804+0000 7fe8c7bff700 1 -- 172.21.15.93:0/2419628644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8c01001c0 msgr2=0x7fe8c01005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:38.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.804+0000 7fe8c7bff700 1 --2- 172.21.15.93:0/2419628644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8c01001c0 0x7fe8c01005d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009990 tx=0x7fe8bc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.804+0000 7fe8c7bff700 1 -- 172.21.15.93:0/2419628644 shutdown_connections 2023-11-16T16:48:38.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.804+0000 7fe8c7bff700 1 --2- 172.21.15.93:0/2419628644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8c01001c0 0x7fe8c01005d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.804+0000 7fe8c7bff700 1 --2- 172.21.15.93:0/2419628644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8c0101d40 0x7fe8c0108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.804+0000 7fe8c7bff700 1 --2- 172.21.15.93:0/2419628644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c01013b0 0x7fe8c0101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.804+0000 7fe8c7bff700 1 -- 172.21.15.93:0/2419628644 >> 172.21.15.93:0/2419628644 conn(0x7fe8c00fb7a0 msgr2=0x7fe8c00fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:38.807 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.804+0000 7fe8c7bff700 1 -- 172.21.15.93:0/2419628644 shutdown_connections 2023-11-16T16:48:38.807 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.804+0000 7fe8c7bff700 1 -- 172.21.15.93:0/2419628644 wait complete. 2023-11-16T16:48:38.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.805+0000 7fe8c7bff700 1 Processor -- start 2023-11-16T16:48:38.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.805+0000 7fe8c7bff700 1 -- start start 2023-11-16T16:48:38.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.805+0000 7fe8c7bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8c01001c0 0x7fe8c00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:38.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.805+0000 7fe8c7bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8c01013b0 0x7fe8c0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:38.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.805+0000 7fe8c7bff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c0101d40 0x7fe8c0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:38.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.805+0000 7fe8c7bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0073bc0 con 0x7fe8c01001c0 2023-11-16T16:48:38.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.805+0000 7fe8c7bff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0073d30 con 0x7fe8c0101d40 2023-11-16T16:48:38.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.805+0000 7fe8c7bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c01a00a0 con 0x7fe8c01013b0 2023-11-16T16:48:38.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8c599b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8c01001c0 0x7fe8c00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:38.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8c599b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8c01001c0 0x7fe8c00748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39014/0 (socket says 172.21.15.93:39014) 2023-11-16T16:48:38.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8c599b700 1 -- 172.21.15.93:0/1931791908 learned_addr learned my addr 172.21.15.93:0/1931791908 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:38.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8c519a700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8c01013b0 0x7fe8c0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:38.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8c619c700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c0101d40 0x7fe8c0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:38.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8c599b700 1 -- 172.21.15.93:0/1931791908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8c01013b0 msgr2=0x7fe8c0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:38.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8c599b700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8c01013b0 0x7fe8c0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8c599b700 1 -- 172.21.15.93:0/1931791908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c0101d40 msgr2=0x7fe8c0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:38.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8c599b700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c0101d40 0x7fe8c0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:38.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8c599b700 1 -- 172.21.15.93:0/1931791908 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0095f0 con 0x7fe8c01001c0 2023-11-16T16:48:38.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8c599b700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8c01001c0 0x7fe8c00748a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc000c00 tx=0x7fe8bc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:38.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8b6ffd700 1 -- 172.21.15.93:0/1931791908 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc01d070 con 0x7fe8c01001c0 2023-11-16T16:48:38.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8b6ffd700 1 -- 172.21.15.93:0/1931791908 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8bc003f60 con 0x7fe8c01001c0 2023-11-16T16:48:38.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.806+0000 7fe8c7bff700 1 -- 172.21.15.93:0/1931791908 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c010bb80 con 0x7fe8c01001c0 2023-11-16T16:48:38.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.807+0000 7fe8b6ffd700 1 -- 172.21.15.93:0/1931791908 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc022b50 con 0x7fe8c01001c0 2023-11-16T16:48:38.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.807+0000 7fe8c7bff700 1 -- 172.21.15.93:0/1931791908 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c010b9f0 con 0x7fe8c01001c0 2023-11-16T16:48:38.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.808+0000 7fe8c7bff700 1 -- 172.21.15.93:0/1931791908 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c01077a0 con 0x7fe8c01001c0 2023-11-16T16:48:38.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.809+0000 7fe8b6ffd700 1 -- 172.21.15.93:0/1931791908 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe8bc0040d0 con 0x7fe8c01001c0 2023-11-16T16:48:38.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.811+0000 7fe8b6ffd700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe8ac0a1150 0x7fe8ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:38.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.811+0000 7fe8b6ffd700 1 -- 172.21.15.93:0/1931791908 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe8bc0ce3e0 con 0x7fe8c01001c0 2023-11-16T16:48:38.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.811+0000 7fe8c519a700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe8ac0a1150 0x7fe8ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:38.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.812+0000 7fe8c519a700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe8ac0a1150 0x7fe8ac0a3600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0005950 tx=0x7fe8b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:38.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:38.815+0000 7fe8b6ffd700 1 -- 172.21.15.93:0/1931791908 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe8bc097790 con 0x7fe8c01001c0 2023-11-16T16:48:38.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:38 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2121802655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:48:38.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:38 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1124056184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:48:38.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:38 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/29051006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:38 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2121802655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:48:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:38 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1124056184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:48:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:38 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/29051006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:39.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.016+0000 7fe8c7bff700 1 -- 172.21.15.93:0/1931791908 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8c002cfe0 con 0x7fe8c01001c0 2023-11-16T16:48:39.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.017+0000 7fe8b6ffd700 1 -- 172.21.15.93:0/1931791908 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24795 (secure 0 0 0) 0x7fe8bc09adb0 con 0x7fe8c01001c0 2023-11-16T16:48:39.019 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:39.020 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":92,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:47:53.331398+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":1,"to":2}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:48:39.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.019+0000 7fe8c7bff700 1 -- 172.21.15.93:0/1931791908 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe8ac0a1150 msgr2=0x7fe8ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:39.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.019+0000 7fe8c7bff700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe8ac0a1150 0x7fe8ac0a3600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0005950 tx=0x7fe8b000b410 comp rx=0 tx=0).stop 2023-11-16T16:48:39.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.020+0000 7fe8c7bff700 1 -- 172.21.15.93:0/1931791908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8c01001c0 msgr2=0x7fe8c00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:39.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.020+0000 7fe8c7bff700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8c01001c0 0x7fe8c00748a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc000c00 tx=0x7fe8bc005740 comp rx=0 tx=0).stop 2023-11-16T16:48:39.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.020+0000 7fe8c7bff700 1 -- 172.21.15.93:0/1931791908 shutdown_connections 2023-11-16T16:48:39.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.020+0000 7fe8c7bff700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe8ac0a1150 0x7fe8ac0a3600 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.020+0000 7fe8c7bff700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8c01001c0 0x7fe8c00748a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.020+0000 7fe8c7bff700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c0101d40 0x7fe8c0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.020+0000 7fe8c7bff700 1 --2- 172.21.15.93:0/1931791908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8c01013b0 0x7fe8c0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.020+0000 7fe8c7bff700 1 -- 172.21.15.93:0/1931791908 >> 172.21.15.93:0/1931791908 conn(0x7fe8c00fb7a0 msgr2=0x7fe8c01048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:39.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.020+0000 7fe8c7bff700 1 -- 172.21.15.93:0/1931791908 shutdown_connections 2023-11-16T16:48:39.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.020+0000 7fe8c7bff700 1 -- 172.21.15.93:0/1931791908 wait complete. 2023-11-16T16:48:39.041 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:42:13.029135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:46:35.301321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:46:36.552545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-16T16:46:39.628380+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-16T16:48:39.041 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:48:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:38 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2121802655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:48:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:38 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1124056184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:48:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:38 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/29051006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:39.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.241+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1226540127 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b38100d40 msgr2=0x7f5b38103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:39.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.241+0000 7f5b3f7ff700 1 --2- 172.21.15.93:0/1226540127 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b38100d40 0x7f5b38103120 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009990 tx=0x7f5b34009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.241+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1226540127 shutdown_connections 2023-11-16T16:48:39.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.241+0000 7f5b3f7ff700 1 --2- 172.21.15.93:0/1226540127 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b38100d40 0x7f5b38103120 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.241+0000 7f5b3f7ff700 1 --2- 172.21.15.93:0/1226540127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b38105f80 0x7f5b381083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.241+0000 7f5b3f7ff700 1 --2- 172.21.15.93:0/1226540127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b38103660 0x7f5b38105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.241+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1226540127 >> 172.21.15.93:0/1226540127 conn(0x7f5b380fa7c0 msgr2=0x7f5b380fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:39.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.241+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1226540127 shutdown_connections 2023-11-16T16:48:39.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.241+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1226540127 wait complete. 2023-11-16T16:48:39.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.242+0000 7f5b3f7ff700 1 Processor -- start 2023-11-16T16:48:39.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.242+0000 7f5b3f7ff700 1 -- start start 2023-11-16T16:48:39.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.242+0000 7f5b3f7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b38100d40 0x7f5b381982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:39.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.242+0000 7f5b3f7ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b38103660 0x7f5b38198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:39.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.242+0000 7f5b3f7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b38105f80 0x7f5b3819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:39.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3f7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3819fe60 con 0x7f5b38105f80 2023-11-16T16:48:39.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3f7ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3819ffa0 con 0x7f5b38103660 2023-11-16T16:48:39.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3f7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b381a0130 con 0x7f5b38100d40 2023-11-16T16:48:39.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3dd9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b38105f80 0x7f5b3819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:39.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3dd9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b38105f80 0x7f5b3819f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39020/0 (socket says 172.21.15.93:39020) 2023-11-16T16:48:39.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3dd9c700 1 -- 172.21.15.93:0/1101427491 learned_addr learned my addr 172.21.15.93:0/1101427491 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:39.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3d59b700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b38100d40 0x7f5b381982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:39.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3cd9a700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b38103660 0x7f5b38198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:39.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3dd9c700 1 -- 172.21.15.93:0/1101427491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b38100d40 msgr2=0x7f5b381982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:39.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3dd9c700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b38100d40 0x7f5b381982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3dd9c700 1 -- 172.21.15.93:0/1101427491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b38103660 msgr2=0x7f5b38198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:39.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3dd9c700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b38103660 0x7f5b38198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3dd9c700 1 -- 172.21.15.93:0/1101427491 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b340095f0 con 0x7f5b38105f80 2023-11-16T16:48:39.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.243+0000 7f5b3dd9c700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b38105f80 0x7f5b3819f890 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5b30007ee0 tx=0x7f5b3000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:39.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.244+0000 7f5b2e7fc700 1 -- 172.21.15.93:0/1101427491 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30017070 con 0x7f5b38105f80 2023-11-16T16:48:39.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.244+0000 7f5b2e7fc700 1 -- 172.21.15.93:0/1101427491 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b3000f040 con 0x7f5b38105f80 2023-11-16T16:48:39.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.244+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1101427491 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3810bc70 con 0x7f5b38105f80 2023-11-16T16:48:39.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.244+0000 7f5b2e7fc700 1 -- 172.21.15.93:0/1101427491 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30005580 con 0x7f5b38105f80 2023-11-16T16:48:39.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.244+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1101427491 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3810bae0 con 0x7f5b38105f80 2023-11-16T16:48:39.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.245+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1101427491 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b38107b70 con 0x7f5b38105f80 2023-11-16T16:48:39.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.247+0000 7f5b2e7fc700 1 -- 172.21.15.93:0/1101427491 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5b3000db20 con 0x7f5b38105f80 2023-11-16T16:48:39.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.249+0000 7f5b2e7fc700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5b240a11d0 0x7f5b240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:39.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.249+0000 7f5b2e7fc700 1 -- 172.21.15.93:0/1101427491 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f5b300cc3e0 con 0x7f5b38105f80 2023-11-16T16:48:39.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.249+0000 7f5b3d59b700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5b240a11d0 0x7f5b240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:39.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.250+0000 7f5b3d59b700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5b240a11d0 0x7f5b240a3680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400b2e0 tx=0x7f5b34005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:39.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.253+0000 7f5b2e7fc700 1 -- 172.21.15.93:0/1101427491 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5b30095700 con 0x7f5b38105f80 2023-11-16T16:48:39.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.449+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1101427491 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b38061700 con 0x7f5b38105f80 2023-11-16T16:48:39.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.449+0000 7f5b2e7fc700 1 -- 172.21.15.93:0/1101427491 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b30098d20 con 0x7f5b38105f80 2023-11-16T16:48:39.451 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-11-16T16:48:39.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.451+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1101427491 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5b240a11d0 msgr2=0x7f5b240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:39.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.451+0000 7f5b3f7ff700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5b240a11d0 0x7f5b240a3680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400b2e0 tx=0x7f5b34005fd0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.452+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1101427491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b38105f80 msgr2=0x7f5b3819f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:39.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.452+0000 7f5b3f7ff700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b38105f80 0x7f5b3819f890 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5b30007ee0 tx=0x7f5b3000d3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.452+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1101427491 shutdown_connections 2023-11-16T16:48:39.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.452+0000 7f5b3f7ff700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5b240a11d0 0x7f5b240a3680 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.452+0000 7f5b3f7ff700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b38105f80 0x7f5b3819f890 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.452+0000 7f5b3f7ff700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b38100d40 0x7f5b381982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.452+0000 7f5b3f7ff700 1 --2- 172.21.15.93:0/1101427491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b38103660 0x7f5b38198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.452+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1101427491 >> 172.21.15.93:0/1101427491 conn(0x7f5b380fa7c0 msgr2=0x7f5b381019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:39.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.452+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1101427491 shutdown_connections 2023-11-16T16:48:39.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.452+0000 7f5b3f7ff700 1 -- 172.21.15.93:0/1101427491 wait complete. 2023-11-16T16:48:39.472 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:48:39.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.676+0000 7f42ae9ff700 1 -- 172.21.15.93:0/3214643740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a8073550 msgr2=0x7f42a80739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:39.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.676+0000 7f42ae9ff700 1 --2- 172.21.15.93:0/3214643740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a8073550 0x7f42a80739c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4290009990 tx=0x7f4290009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.676+0000 7f42ae9ff700 1 -- 172.21.15.93:0/3214643740 shutdown_connections 2023-11-16T16:48:39.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.676+0000 7f42ae9ff700 1 --2- 172.21.15.93:0/3214643740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42a810c890 0x7f42a810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.676+0000 7f42ae9ff700 1 --2- 172.21.15.93:0/3214643740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a8074b30 0x7f42a8072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.676+0000 7f42ae9ff700 1 --2- 172.21.15.93:0/3214643740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a8073550 0x7f42a80739c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.676+0000 7f42ae9ff700 1 -- 172.21.15.93:0/3214643740 >> 172.21.15.93:0/3214643740 conn(0x7f42a80fb8a0 msgr2=0x7f42a80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:39.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.676+0000 7f42ae9ff700 1 -- 172.21.15.93:0/3214643740 shutdown_connections 2023-11-16T16:48:39.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.676+0000 7f42ae9ff700 1 -- 172.21.15.93:0/3214643740 wait complete. 2023-11-16T16:48:39.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.677+0000 7f42ae9ff700 1 Processor -- start 2023-11-16T16:48:39.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.677+0000 7f42ae9ff700 1 -- start start 2023-11-16T16:48:39.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.677+0000 7f42ae9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a8073550 0x7f42a819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:39.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.677+0000 7f42ae9ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a8074b30 0x7f42a819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:39.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.677+0000 7f42ae9ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42a810c890 0x7f42a81a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:39.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.677+0000 7f42ae9ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a81a41d0 con 0x7f42a8074b30 2023-11-16T16:48:39.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.677+0000 7f42ae9ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a81a4310 con 0x7f42a810c890 2023-11-16T16:48:39.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.677+0000 7f42ae9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a81a44a0 con 0x7f42a8073550 2023-11-16T16:48:39.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f42a7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a8073550 0x7f42a819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:39.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f429ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a8074b30 0x7f42a819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:39.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f42a7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a8073550 0x7f42a819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:34316/0 (socket says 172.21.15.93:34316) 2023-11-16T16:48:39.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f42a7fff700 1 -- 172.21.15.93:0/1895208734 learned_addr learned my addr 172.21.15.93:0/1895208734 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:39.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f42acf9c700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42a810c890 0x7f42a81a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:39.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f429ffff700 1 -- 172.21.15.93:0/1895208734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a8073550 msgr2=0x7f42a819c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:39.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f429ffff700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a8073550 0x7f42a819c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f429ffff700 1 -- 172.21.15.93:0/1895208734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42a810c890 msgr2=0x7f42a81a3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:39.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f429ffff700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42a810c890 0x7f42a81a3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f429ffff700 1 -- 172.21.15.93:0/1895208734 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42900095f0 con 0x7f42a8074b30 2023-11-16T16:48:39.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f429ffff700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a8074b30 0x7f42a819cb90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4290000c00 tx=0x7f4290005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:39.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f42a5ffb700 1 -- 172.21.15.93:0/1895208734 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429001d070 con 0x7f42a8074b30 2023-11-16T16:48:39.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f42a5ffb700 1 -- 172.21.15.93:0/1895208734 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4290022470 con 0x7f42a8074b30 2023-11-16T16:48:39.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f42a5ffb700 1 -- 172.21.15.93:0/1895208734 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290017650 con 0x7f42a8074b30 2023-11-16T16:48:39.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.678+0000 7f42ae9ff700 1 -- 172.21.15.93:0/1895208734 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a8112250 con 0x7f42a8074b30 2023-11-16T16:48:39.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.679+0000 7f42ae9ff700 1 -- 172.21.15.93:0/1895208734 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a81120c0 con 0x7f42a8074b30 2023-11-16T16:48:39.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.680+0000 7f42ae9ff700 1 -- 172.21.15.93:0/1895208734 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a81023e0 con 0x7f42a8074b30 2023-11-16T16:48:39.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.682+0000 7f42a5ffb700 1 -- 172.21.15.93:0/1895208734 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f42900225e0 con 0x7f42a8074b30 2023-11-16T16:48:39.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.684+0000 7f42a5ffb700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f42940a1150 0x7f42940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:39.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.684+0000 7f42a5ffb700 1 -- 172.21.15.93:0/1895208734 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f42900ce0d0 con 0x7f42a8074b30 2023-11-16T16:48:39.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.684+0000 7f42a7fff700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f42940a1150 0x7f42940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:39.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.684+0000 7f42a7fff700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f42940a1150 0x7f42940a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4298005950 tx=0x7f4298009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:39.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.688+0000 7f42a5ffb700 1 -- 172.21.15.93:0/1895208734 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4290097480 con 0x7f42a8074b30 2023-11-16T16:48:39.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.886+0000 7f42ae9ff700 1 -- 172.21.15.93:0/1895208734 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42a80619c0 con 0x7f42a8074b30 2023-11-16T16:48:39.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.887+0000 7f42a5ffb700 1 -- 172.21.15.93:0/1895208734 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f429009aaa0 con 0x7f42a8074b30 2023-11-16T16:48:39.889 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:48:39.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.889+0000 7f42ae9ff700 1 -- 172.21.15.93:0/1895208734 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f42940a1150 msgr2=0x7f42940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:39.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.889+0000 7f42ae9ff700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f42940a1150 0x7f42940a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4298005950 tx=0x7f4298009260 comp rx=0 tx=0).stop 2023-11-16T16:48:39.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.889+0000 7f42ae9ff700 1 -- 172.21.15.93:0/1895208734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a8074b30 msgr2=0x7f42a819cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:39.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.889+0000 7f42ae9ff700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a8074b30 0x7f42a819cb90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4290000c00 tx=0x7f4290005720 comp rx=0 tx=0).stop 2023-11-16T16:48:39.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.889+0000 7f42ae9ff700 1 -- 172.21.15.93:0/1895208734 shutdown_connections 2023-11-16T16:48:39.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.889+0000 7f42ae9ff700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f42940a1150 0x7f42940a3600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.889+0000 7f42ae9ff700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42a810c890 0x7f42a81a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.889+0000 7f42ae9ff700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a8073550 0x7f42a819c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.889+0000 7f42ae9ff700 1 --2- 172.21.15.93:0/1895208734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a8074b30 0x7f42a819cb90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:39.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.889+0000 7f42ae9ff700 1 -- 172.21.15.93:0/1895208734 >> 172.21.15.93:0/1895208734 conn(0x7f42a80fb8a0 msgr2=0x7f42a80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:39.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.890+0000 7f42ae9ff700 1 -- 172.21.15.93:0/1895208734 shutdown_connections 2023-11-16T16:48:39.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:39.890+0000 7f42ae9ff700 1 -- 172.21.15.93:0/1895208734 wait complete. 2023-11-16T16:48:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:39 smithi093 ceph-mon[97863]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:48:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:39 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1931791908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:39 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1101427491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:48:39.910 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:48:39.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:39 smithi159 ceph-mon[131528]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:48:39.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:39 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1931791908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:39.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:39 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1101427491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:48:40.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:39 smithi125 ceph-mon[131076]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:48:40.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:39 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1931791908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:40.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:39 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1101427491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:48:40.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.109+0000 7fd05edff700 1 -- 172.21.15.93:0/1227854346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0581002c0 msgr2=0x7fd0581006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.109+0000 7fd05edff700 1 --2- 172.21.15.93:0/1227854346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0581002c0 0x7fd0581006d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009990 tx=0x7fd04c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.109+0000 7fd05edff700 1 -- 172.21.15.93:0/1227854346 shutdown_connections 2023-11-16T16:48:40.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.109+0000 7fd05edff700 1 --2- 172.21.15.93:0/1227854346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0581002c0 0x7fd0581006d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.109+0000 7fd05edff700 1 --2- 172.21.15.93:0/1227854346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd058101e40 0x7fd058108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.109+0000 7fd05edff700 1 --2- 172.21.15.93:0/1227854346 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0581014b0 0x7fd058101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.109+0000 7fd05edff700 1 -- 172.21.15.93:0/1227854346 >> 172.21.15.93:0/1227854346 conn(0x7fd0580fb8a0 msgr2=0x7fd0580fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:40.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.109+0000 7fd05edff700 1 -- 172.21.15.93:0/1227854346 shutdown_connections 2023-11-16T16:48:40.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.109+0000 7fd05edff700 1 -- 172.21.15.93:0/1227854346 wait complete. 2023-11-16T16:48:40.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.110+0000 7fd05edff700 1 Processor -- start 2023-11-16T16:48:40.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.111+0000 7fd05edff700 1 -- start start 2023-11-16T16:48:40.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.111+0000 7fd05edff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0581002c0 0x7fd058196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:40.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.111+0000 7fd05edff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0581014b0 0x7fd0581965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:40.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.111+0000 7fd05edff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd058101e40 0x7fd05819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:40.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.111+0000 7fd05edff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05819dc10 con 0x7fd0581014b0 2023-11-16T16:48:40.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.111+0000 7fd05edff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05819dd50 con 0x7fd058101e40 2023-11-16T16:48:40.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.111+0000 7fd05edff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05819dee0 con 0x7fd0581002c0 2023-11-16T16:48:40.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.111+0000 7fd057fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0581014b0 0x7fd0581965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:40.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.111+0000 7fd057fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0581014b0 0x7fd0581965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39054/0 (socket says 172.21.15.93:39054) 2023-11-16T16:48:40.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.111+0000 7fd057fff700 1 -- 172.21.15.93:0/3068033777 learned_addr learned my addr 172.21.15.93:0/3068033777 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:40.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.111+0000 7fd05cb9b700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0581002c0 0x7fd058196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:40.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.111+0000 7fd05d39c700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd058101e40 0x7fd05819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:40.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.112+0000 7fd057fff700 1 -- 172.21.15.93:0/3068033777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0581002c0 msgr2=0x7fd058196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.112+0000 7fd057fff700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0581002c0 0x7fd058196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.112+0000 7fd057fff700 1 -- 172.21.15.93:0/3068033777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd058101e40 msgr2=0x7fd05819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.112+0000 7fd057fff700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd058101e40 0x7fd05819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.112+0000 7fd057fff700 1 -- 172.21.15.93:0/3068033777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c0095f0 con 0x7fd0581014b0 2023-11-16T16:48:40.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.112+0000 7fd057fff700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0581014b0 0x7fd0581965d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd04400e9a0 tx=0x7fd04400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:40.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.112+0000 7fd055ffb700 1 -- 172.21.15.93:0/3068033777 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044009790 con 0x7fd0581014b0 2023-11-16T16:48:40.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.112+0000 7fd055ffb700 1 -- 172.21.15.93:0/3068033777 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd044010430 con 0x7fd0581014b0 2023-11-16T16:48:40.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.112+0000 7fd055ffb700 1 -- 172.21.15.93:0/3068033777 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044018680 con 0x7fd0581014b0 2023-11-16T16:48:40.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.112+0000 7fd05edff700 1 -- 172.21.15.93:0/3068033777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05810bc90 con 0x7fd0581014b0 2023-11-16T16:48:40.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.112+0000 7fd05edff700 1 -- 172.21.15.93:0/3068033777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05810bb00 con 0x7fd0581014b0 2023-11-16T16:48:40.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.113+0000 7fd05edff700 1 -- 172.21.15.93:0/3068033777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd058107850 con 0x7fd0581014b0 2023-11-16T16:48:40.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.115+0000 7fd055ffb700 1 -- 172.21.15.93:0/3068033777 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd0440105a0 con 0x7fd0581014b0 2023-11-16T16:48:40.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.116+0000 7fd055ffb700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd0480a11d0 0x7fd0480a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:40.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.116+0000 7fd055ffb700 1 -- 172.21.15.93:0/3068033777 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fd044014070 con 0x7fd0581014b0 2023-11-16T16:48:40.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.117+0000 7fd05cb9b700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd0480a11d0 0x7fd0480a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:40.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.117+0000 7fd05cb9b700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd0480a11d0 0x7fd0480a3680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b2e0 tx=0x7fd04c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:40.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.120+0000 7fd055ffb700 1 -- 172.21.15.93:0/3068033777 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd044051730 con 0x7fd0581014b0 2023-11-16T16:48:40.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.316+0000 7fd05edff700 1 -- 172.21.15.93:0/3068033777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd058061c00 con 0x7fd0581014b0 2023-11-16T16:48:40.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.317+0000 7fd055ffb700 1 -- 172.21.15.93:0/3068033777 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0440961d0 con 0x7fd0581014b0 2023-11-16T16:48:40.318 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-11-16T16:48:40.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.319+0000 7fd05edff700 1 -- 172.21.15.93:0/3068033777 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd0480a11d0 msgr2=0x7fd0480a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.319+0000 7fd05edff700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd0480a11d0 0x7fd0480a3680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b2e0 tx=0x7fd04c005fb0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.319+0000 7fd05edff700 1 -- 172.21.15.93:0/3068033777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0581014b0 msgr2=0x7fd0581965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.319+0000 7fd05edff700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0581014b0 0x7fd0581965d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd04400e9a0 tx=0x7fd04400ecb0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.319+0000 7fd05edff700 1 -- 172.21.15.93:0/3068033777 shutdown_connections 2023-11-16T16:48:40.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.319+0000 7fd05edff700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fd0480a11d0 0x7fd0480a3680 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.319+0000 7fd05edff700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd058101e40 0x7fd05819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.319+0000 7fd05edff700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0581002c0 0x7fd058196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.319+0000 7fd05edff700 1 --2- 172.21.15.93:0/3068033777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0581014b0 0x7fd0581965d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.319+0000 7fd05edff700 1 -- 172.21.15.93:0/3068033777 >> 172.21.15.93:0/3068033777 conn(0x7fd0580fb8a0 msgr2=0x7fd058104990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:40.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.319+0000 7fd05edff700 1 -- 172.21.15.93:0/3068033777 shutdown_connections 2023-11-16T16:48:40.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.319+0000 7fd05edff700 1 -- 172.21.15.93:0/3068033777 wait complete. 2023-11-16T16:48:40.339 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-16T16:48:40.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.538+0000 7ff2d69ff700 1 -- 172.21.15.93:0/701380525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2d0101e50 msgr2=0x7ff2d0108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.538+0000 7ff2d69ff700 1 --2- 172.21.15.93:0/701380525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2d0101e50 0x7ff2d0108710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800b0a0 tx=0x7ff2c800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.538+0000 7ff2d69ff700 1 -- 172.21.15.93:0/701380525 shutdown_connections 2023-11-16T16:48:40.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.538+0000 7ff2d69ff700 1 --2- 172.21.15.93:0/701380525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2d0101e50 0x7ff2d0108710 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.538+0000 7ff2d69ff700 1 --2- 172.21.15.93:0/701380525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d01002d0 0x7ff2d01006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.538+0000 7ff2d69ff700 1 --2- 172.21.15.93:0/701380525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2d01014c0 0x7ff2d0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.538+0000 7ff2d69ff700 1 -- 172.21.15.93:0/701380525 >> 172.21.15.93:0/701380525 conn(0x7ff2d00fb8b0 msgr2=0x7ff2d00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:40.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.538+0000 7ff2d69ff700 1 -- 172.21.15.93:0/701380525 shutdown_connections 2023-11-16T16:48:40.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.538+0000 7ff2d69ff700 1 -- 172.21.15.93:0/701380525 wait complete. 2023-11-16T16:48:40.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.539+0000 7ff2d69ff700 1 Processor -- start 2023-11-16T16:48:40.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.539+0000 7ff2d69ff700 1 -- start start 2023-11-16T16:48:40.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.539+0000 7ff2d69ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2d01002d0 0x7ff2d00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:40.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.539+0000 7ff2d69ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d01014c0 0x7ff2d0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:40.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.539+0000 7ff2d69ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2d0101e50 0x7ff2d0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:40.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.539+0000 7ff2d69ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d0073bc0 con 0x7ff2d0101e50 2023-11-16T16:48:40.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.539+0000 7ff2d69ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d0073d30 con 0x7ff2d01002d0 2023-11-16T16:48:40.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.539+0000 7ff2d69ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d01a01e0 con 0x7ff2d01014c0 2023-11-16T16:48:40.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.540+0000 7ff2d4f9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2d0101e50 0x7ff2d0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:40.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.540+0000 7ff2cffff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2d01002d0 0x7ff2d00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:40.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.540+0000 7ff2d4f9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2d0101e50 0x7ff2d0073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39062/0 (socket says 172.21.15.93:39062) 2023-11-16T16:48:40.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.540+0000 7ff2d4f9c700 1 -- 172.21.15.93:0/38425654 learned_addr learned my addr 172.21.15.93:0/38425654 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:40.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.540+0000 7ff2c7fff700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d01014c0 0x7ff2d0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:40.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.540+0000 7ff2d4f9c700 1 -- 172.21.15.93:0/38425654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d01014c0 msgr2=0x7ff2d0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.540+0000 7ff2d4f9c700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d01014c0 0x7ff2d0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.540+0000 7ff2d4f9c700 1 -- 172.21.15.93:0/38425654 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2d01002d0 msgr2=0x7ff2d00748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.540+0000 7ff2d4f9c700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2d01002d0 0x7ff2d00748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.540+0000 7ff2d4f9c700 1 -- 172.21.15.93:0/38425654 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c0009520 con 0x7ff2d0101e50 2023-11-16T16:48:40.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.540+0000 7ff2d4f9c700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2d0101e50 0x7ff2d0073420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800d010 tx=0x7ff2c800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:40.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.540+0000 7ff2cdffb700 1 -- 172.21.15.93:0/38425654 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c800e040 con 0x7ff2d0101e50 2023-11-16T16:48:40.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.541+0000 7ff2cdffb700 1 -- 172.21.15.93:0/38425654 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c8003e10 con 0x7ff2d0101e50 2023-11-16T16:48:40.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.541+0000 7ff2d69ff700 1 -- 172.21.15.93:0/38425654 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c8009d00 con 0x7ff2d0101e50 2023-11-16T16:48:40.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.541+0000 7ff2cdffb700 1 -- 172.21.15.93:0/38425654 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80052a0 con 0x7ff2d0101e50 2023-11-16T16:48:40.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.541+0000 7ff2d69ff700 1 -- 172.21.15.93:0/38425654 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d010bca0 con 0x7ff2d0101e50 2023-11-16T16:48:40.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.542+0000 7ff2d69ff700 1 -- 172.21.15.93:0/38425654 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d01077f0 con 0x7ff2d0101e50 2023-11-16T16:48:40.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.543+0000 7ff2cdffb700 1 -- 172.21.15.93:0/38425654 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff2c801d480 con 0x7ff2d0101e50 2023-11-16T16:48:40.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.545+0000 7ff2cdffb700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff2bc0a11d0 0x7ff2bc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:40.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.545+0000 7ff2cdffb700 1 -- 172.21.15.93:0/38425654 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7ff2c80cdff0 con 0x7ff2d0101e50 2023-11-16T16:48:40.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.545+0000 7ff2cffff700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff2bc0a11d0 0x7ff2bc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:40.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.546+0000 7ff2cffff700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff2bc0a11d0 0x7ff2bc0a3680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0101320 tx=0x7ff2c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:40.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.549+0000 7ff2cdffb700 1 -- 172.21.15.93:0/38425654 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff2c80973c0 con 0x7ff2d0101e50 2023-11-16T16:48:40.750 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.749+0000 7ff2d69ff700 1 -- 172.21.15.93:0/38425654 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff2d0060f30 con 0x7ff2d0101e50 2023-11-16T16:48:40.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.750+0000 7ff2cdffb700 1 -- 172.21.15.93:0/38425654 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff2c809a9e0 con 0x7ff2d0101e50 2023-11-16T16:48:40.751 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:48:40.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.752+0000 7ff2d69ff700 1 -- 172.21.15.93:0/38425654 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff2bc0a11d0 msgr2=0x7ff2bc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.752+0000 7ff2d69ff700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff2bc0a11d0 0x7ff2bc0a3680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0101320 tx=0x7ff2c0009260 comp rx=0 tx=0).stop 2023-11-16T16:48:40.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.752+0000 7ff2d69ff700 1 -- 172.21.15.93:0/38425654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2d0101e50 msgr2=0x7ff2d0073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.752+0000 7ff2d69ff700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2d0101e50 0x7ff2d0073420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800d010 tx=0x7ff2c800bc80 comp rx=0 tx=0).stop 2023-11-16T16:48:40.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.752+0000 7ff2d69ff700 1 -- 172.21.15.93:0/38425654 shutdown_connections 2023-11-16T16:48:40.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.752+0000 7ff2d69ff700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff2bc0a11d0 0x7ff2bc0a3680 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.752+0000 7ff2d69ff700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2d0101e50 0x7ff2d0073420 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.752+0000 7ff2d69ff700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2d01002d0 0x7ff2d00748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.752+0000 7ff2d69ff700 1 --2- 172.21.15.93:0/38425654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d01014c0 0x7ff2d0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.752+0000 7ff2d69ff700 1 -- 172.21.15.93:0/38425654 >> 172.21.15.93:0/38425654 conn(0x7ff2d00fb8b0 msgr2=0x7ff2d0104930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:40.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.752+0000 7ff2d69ff700 1 -- 172.21.15.93:0/38425654 shutdown_connections 2023-11-16T16:48:40.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.752+0000 7ff2d69ff700 1 -- 172.21.15.93:0/38425654 wait complete. 2023-11-16T16:48:40.772 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:48:40.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:40 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1895208734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:48:40.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:40 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3068033777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:48:40.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.968+0000 7ff15fdff700 1 -- 172.21.15.93:0/4042051341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1581002b0 msgr2=0x7ff1581006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.968+0000 7ff15fdff700 1 --2- 172.21.15.93:0/4042051341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1581002b0 0x7ff1581006c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff154009990 tx=0x7ff154009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.969+0000 7ff15fdff700 1 -- 172.21.15.93:0/4042051341 shutdown_connections 2023-11-16T16:48:40.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.969+0000 7ff15fdff700 1 --2- 172.21.15.93:0/4042051341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1581002b0 0x7ff1581006c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.969+0000 7ff15fdff700 1 --2- 172.21.15.93:0/4042051341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff158101e30 0x7ff1581086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.969+0000 7ff15fdff700 1 --2- 172.21.15.93:0/4042051341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1581014a0 0x7ff1581018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.969+0000 7ff15fdff700 1 -- 172.21.15.93:0/4042051341 >> 172.21.15.93:0/4042051341 conn(0x7ff1580fb890 msgr2=0x7ff1580fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:40.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.969+0000 7ff15fdff700 1 -- 172.21.15.93:0/4042051341 shutdown_connections 2023-11-16T16:48:40.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.969+0000 7ff15fdff700 1 -- 172.21.15.93:0/4042051341 wait complete. 2023-11-16T16:48:40.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.970+0000 7ff15fdff700 1 Processor -- start 2023-11-16T16:48:40.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.970+0000 7ff15fdff700 1 -- start start 2023-11-16T16:48:40.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15fdff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1581002b0 0x7ff1580748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:40.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15fdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1581014a0 0x7ff158072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:40.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15fdff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff158101e30 0x7ff158073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:40.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15fdff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff158073bc0 con 0x7ff1581002b0 2023-11-16T16:48:40.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15fdff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff158073d30 con 0x7ff158101e30 2023-11-16T16:48:40.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15fdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1581a0010 con 0x7ff1581014a0 2023-11-16T16:48:40.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15db9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1581002b0 0x7ff1580748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:40.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15db9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1581002b0 0x7ff1580748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39076/0 (socket says 172.21.15.93:39076) 2023-11-16T16:48:40.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15db9b700 1 -- 172.21.15.93:0/3245043834 learned_addr learned my addr 172.21.15.93:0/3245043834 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:40.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15d39a700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1581014a0 0x7ff158072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:40.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15e39c700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff158101e30 0x7ff158073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:40.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15db9b700 1 -- 172.21.15.93:0/3245043834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1581014a0 msgr2=0x7ff158072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15db9b700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1581014a0 0x7ff158072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15db9b700 1 -- 172.21.15.93:0/3245043834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff158101e30 msgr2=0x7ff158073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:40.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15db9b700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff158101e30 0x7ff158073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:40.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.971+0000 7ff15db9b700 1 -- 172.21.15.93:0/3245043834 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1540095f0 con 0x7ff1581002b0 2023-11-16T16:48:40.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.972+0000 7ff15db9b700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1581002b0 0x7ff1580748a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff154004730 tx=0x7ff154009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:40.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.972+0000 7ff14effd700 1 -- 172.21.15.93:0/3245043834 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15401d070 con 0x7ff1581002b0 2023-11-16T16:48:40.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.972+0000 7ff14effd700 1 -- 172.21.15.93:0/3245043834 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff154004cd0 con 0x7ff1581002b0 2023-11-16T16:48:40.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.972+0000 7ff15fdff700 1 -- 172.21.15.93:0/3245043834 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15810bc80 con 0x7ff1581002b0 2023-11-16T16:48:40.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.972+0000 7ff14effd700 1 -- 172.21.15.93:0/3245043834 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154005530 con 0x7ff1581002b0 2023-11-16T16:48:40.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.972+0000 7ff15fdff700 1 -- 172.21.15.93:0/3245043834 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15810baf0 con 0x7ff1581002b0 2023-11-16T16:48:40.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.973+0000 7ff15fdff700 1 -- 172.21.15.93:0/3245043834 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff158107230 con 0x7ff1581002b0 2023-11-16T16:48:40.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.975+0000 7ff14effd700 1 -- 172.21.15.93:0/3245043834 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff15400fbc0 con 0x7ff1581002b0 2023-11-16T16:48:40.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.976+0000 7ff14effd700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff1440a1150 0x7ff1440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:40.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.976+0000 7ff14effd700 1 -- 172.21.15.93:0/3245043834 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7ff1540ce510 con 0x7ff1581002b0 2023-11-16T16:48:40.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.977+0000 7ff15d39a700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff1440a1150 0x7ff1440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:40.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.977+0000 7ff15d39a700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff1440a1150 0x7ff1440a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff148009bf0 tx=0x7ff148009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:40.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:40.978+0000 7ff14effd700 1 -- 172.21.15.93:0/3245043834 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff1540978c0 con 0x7ff1581002b0 2023-11-16T16:48:40.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:40 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1895208734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:48:40.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:40 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3068033777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:48:41.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:40 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1895208734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:48:41.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:40 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3068033777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:48:41.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.229+0000 7ff15fdff700 1 -- 172.21.15.93:0/3245043834 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff15802cfe0 con 0x7ff1581002b0 2023-11-16T16:48:41.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.229+0000 7ff14effd700 1 -- 172.21.15.93:0/3245043834 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10811 (secure 0 0 0) 0x7ff15409aee0 con 0x7ff1581002b0 2023-11-16T16:48:41.231 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:41.231 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3614592201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3614592201},{"type":"v1","addr":"172.21.15.93:6839","nonce":3614592201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4068789362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4068789362},{"type":"v1","addr":"172.21.15.93:6841","nonce":4068789362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:46:37.124338+0000","modified":"2023-11-16T16:47:10.311966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24541,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/778982589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":778982589},{"type":"v1","addr":"172.21.15.159:6837","nonce":778982589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6839/149300707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":149300707},{"type":"v1","addr":"172.21.15.159:6839","nonce":149300707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/3590027328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":3590027328},{"type":"v1","addr":"172.21.15.125:6839","nonce":3590027328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:48:41.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.232+0000 7ff15fdff700 1 -- 172.21.15.93:0/3245043834 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff1440a1150 msgr2=0x7ff1440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.232+0000 7ff15fdff700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff1440a1150 0x7ff1440a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff148009bf0 tx=0x7ff148009260 comp rx=0 tx=0).stop 2023-11-16T16:48:41.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.232+0000 7ff15fdff700 1 -- 172.21.15.93:0/3245043834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1581002b0 msgr2=0x7ff1580748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.232+0000 7ff15fdff700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1581002b0 0x7ff1580748a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff154004730 tx=0x7ff154009510 comp rx=0 tx=0).stop 2023-11-16T16:48:41.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.232+0000 7ff15fdff700 1 -- 172.21.15.93:0/3245043834 shutdown_connections 2023-11-16T16:48:41.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.232+0000 7ff15fdff700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff1440a1150 0x7ff1440a3600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.232+0000 7ff15fdff700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1581002b0 0x7ff1580748a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.232+0000 7ff15fdff700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff158101e30 0x7ff158073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.232+0000 7ff15fdff700 1 --2- 172.21.15.93:0/3245043834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1581014a0 0x7ff158072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.232+0000 7ff15fdff700 1 -- 172.21.15.93:0/3245043834 >> 172.21.15.93:0/3245043834 conn(0x7ff1580fb890 msgr2=0x7ff1580fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:41.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.232+0000 7ff15fdff700 1 -- 172.21.15.93:0/3245043834 shutdown_connections 2023-11-16T16:48:41.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.232+0000 7ff15fdff700 1 -- 172.21.15.93:0/3245043834 wait complete. 2023-11-16T16:48:41.235 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 23 2023-11-16T16:48:41.253 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:48:41.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.449+0000 7f8f213ff700 1 -- 172.21.15.93:0/2539401892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f1c074b30 msgr2=0x7f8f1c072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.449+0000 7f8f213ff700 1 --2- 172.21.15.93:0/2539401892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f1c074b30 0x7f8f1c072f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009990 tx=0x7f8f0c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.450+0000 7f8f213ff700 1 -- 172.21.15.93:0/2539401892 shutdown_connections 2023-11-16T16:48:41.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.450+0000 7f8f213ff700 1 --2- 172.21.15.93:0/2539401892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f1c074b30 0x7f8f1c072f80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.450+0000 7f8f213ff700 1 --2- 172.21.15.93:0/2539401892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f1c108150 0x7f8f1c10a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.450+0000 7f8f213ff700 1 --2- 172.21.15.93:0/2539401892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1c073550 0x7f8f1c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.450+0000 7f8f213ff700 1 -- 172.21.15.93:0/2539401892 >> 172.21.15.93:0/2539401892 conn(0x7f8f1c0fb870 msgr2=0x7f8f1c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:41.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.450+0000 7f8f213ff700 1 -- 172.21.15.93:0/2539401892 shutdown_connections 2023-11-16T16:48:41.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.450+0000 7f8f213ff700 1 -- 172.21.15.93:0/2539401892 wait complete. 2023-11-16T16:48:41.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.451+0000 7f8f213ff700 1 Processor -- start 2023-11-16T16:48:41.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.451+0000 7f8f213ff700 1 -- start start 2023-11-16T16:48:41.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.451+0000 7f8f213ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1c073550 0x7f8f1c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.451+0000 7f8f213ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f1c074b30 0x7f8f1c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.451+0000 7f8f213ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f1c108150 0x7f8f1c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.451+0000 7f8f213ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c19fda0 con 0x7f8f1c074b30 2023-11-16T16:48:41.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.451+0000 7f8f213ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c19fee0 con 0x7f8f1c073550 2023-11-16T16:48:41.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.451+0000 7f8f213ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c1a0070 con 0x7f8f1c108150 2023-11-16T16:48:41.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.451+0000 7f8f13fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f1c074b30 0x7f8f1c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.451+0000 7f8f1affd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1c073550 0x7f8f1c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.451+0000 7f8f1b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f1c108150 0x7f8f1c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f1affd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1c073550 0x7f8f1c198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:50330/0 (socket says 172.21.15.93:50330) 2023-11-16T16:48:41.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f13fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f1c074b30 0x7f8f1c198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39096/0 (socket says 172.21.15.93:39096) 2023-11-16T16:48:41.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f1affd700 1 -- 172.21.15.93:0/973362304 learned_addr learned my addr 172.21.15.93:0/973362304 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:41.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f1affd700 1 -- 172.21.15.93:0/973362304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f1c108150 msgr2=0x7f8f1c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f1affd700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f1c108150 0x7f8f1c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f1affd700 1 -- 172.21.15.93:0/973362304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f1c074b30 msgr2=0x7f8f1c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f1affd700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f1c074b30 0x7f8f1c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f1affd700 1 -- 172.21.15.93:0/973362304 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c0095f0 con 0x7f8f1c073550 2023-11-16T16:48:41.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f13fff700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f1c074b30 0x7f8f1c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:41.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f1affd700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1c073550 0x7f8f1c198220 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c004710 tx=0x7f8f0c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:41.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f18ff9700 1 -- 172.21.15.93:0/973362304 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c01d070 con 0x7f8f1c073550 2023-11-16T16:48:41.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f213ff700 1 -- 172.21.15.93:0/973362304 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1c10de50 con 0x7f8f1c073550 2023-11-16T16:48:41.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f18ff9700 1 -- 172.21.15.93:0/973362304 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f0c021410 con 0x7f8f1c073550 2023-11-16T16:48:41.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.452+0000 7f8f18ff9700 1 -- 172.21.15.93:0/973362304 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c017650 con 0x7f8f1c073550 2023-11-16T16:48:41.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.453+0000 7f8f213ff700 1 -- 172.21.15.93:0/973362304 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1c10dcc0 con 0x7f8f1c073550 2023-11-16T16:48:41.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.454+0000 7f8f213ff700 1 -- 172.21.15.93:0/973362304 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1c106800 con 0x7f8f1c073550 2023-11-16T16:48:41.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.456+0000 7f8f18ff9700 1 -- 172.21.15.93:0/973362304 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8f0c021580 con 0x7f8f1c073550 2023-11-16T16:48:41.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.457+0000 7f8f18ff9700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f080a1150 0x7f8f080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.457+0000 7f8f18ff9700 1 -- 172.21.15.93:0/973362304 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f8f0c0ce830 con 0x7f8f1c073550 2023-11-16T16:48:41.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.457+0000 7f8f13fff700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f080a1150 0x7f8f080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.458+0000 7f8f13fff700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f080a1150 0x7f8f080a3600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c074850 tx=0x7f8f04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:41.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.459+0000 7f8f18ff9700 1 -- 172.21.15.93:0/973362304 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8f0c097be0 con 0x7f8f1c073550 2023-11-16T16:48:41.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.663+0000 7f8f213ff700 1 -- 172.21.15.93:0/973362304 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f1c02cfe0 con 0x7f8f1c073550 2023-11-16T16:48:41.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.666+0000 7f8f18ff9700 1 -- 172.21.15.93:0/973362304 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24795 (secure 0 0 0) 0x7f8f0c09b200 con 0x7f8f1c073550 2023-11-16T16:48:41.668 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:48:41.668 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":92,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:47:53.331398+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":1,"to":2}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:48:41.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.668+0000 7f8f213ff700 1 -- 172.21.15.93:0/973362304 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f080a1150 msgr2=0x7f8f080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.668+0000 7f8f213ff700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f080a1150 0x7f8f080a3600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c074850 tx=0x7f8f04009260 comp rx=0 tx=0).stop 2023-11-16T16:48:41.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.668+0000 7f8f213ff700 1 -- 172.21.15.93:0/973362304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1c073550 msgr2=0x7f8f1c198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.668+0000 7f8f213ff700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1c073550 0x7f8f1c198220 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c004710 tx=0x7f8f0c009580 comp rx=0 tx=0).stop 2023-11-16T16:48:41.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.668+0000 7f8f213ff700 1 -- 172.21.15.93:0/973362304 shutdown_connections 2023-11-16T16:48:41.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.668+0000 7f8f213ff700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f080a1150 0x7f8f080a3600 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.668+0000 7f8f213ff700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f1c108150 0x7f8f1c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.668+0000 7f8f213ff700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1c073550 0x7f8f1c198220 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.668+0000 7f8f213ff700 1 --2- 172.21.15.93:0/973362304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f1c074b30 0x7f8f1c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.668+0000 7f8f213ff700 1 -- 172.21.15.93:0/973362304 >> 172.21.15.93:0/973362304 conn(0x7f8f1c0fb870 msgr2=0x7f8f1c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:41.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.668+0000 7f8f213ff700 1 -- 172.21.15.93:0/973362304 shutdown_connections 2023-11-16T16:48:41.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.669+0000 7f8f213ff700 1 -- 172.21.15.93:0/973362304 wait complete. 2023-11-16T16:48:41.689 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-16T16:46:37.124338+0000', 'modified': '2023-11-16T16:47:10.311966+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24566, 'mds_1': 24541, 'mds_2': 24533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24506': {'gid': 24506, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.125:6835/2117404468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6834', 'nonce': 2117404468}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 2117404468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/3072074959', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 3072074959}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3072074959}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6837/778982589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 778982589}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 778982589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24533': {'gid': 24533, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.125:6837/1957598038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6836', 'nonce': 1957598038}, {'type': 'v1', 'addr': '172.21.15.125:6837', 'nonce': 1957598038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.159:6839/149300707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 149300707}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 149300707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6839/3590027328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6838', 'nonce': 3590027328}, {'type': 'v1', 'addr': '172.21.15.125:6839', 'nonce': 3590027328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-16T16:48:41.689 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-16T16:48:41.689 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-11-16T16:48:41.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.886+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2401316746 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e14074b30 msgr2=0x7f7e14072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.886+0000 7f7e1a1ff700 1 --2- 172.21.15.93:0/2401316746 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e14074b30 0x7f7e14072f80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009990 tx=0x7f7e04009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.886+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2401316746 shutdown_connections 2023-11-16T16:48:41.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.886+0000 7f7e1a1ff700 1 --2- 172.21.15.93:0/2401316746 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e14074b30 0x7f7e14072f80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.886+0000 7f7e1a1ff700 1 --2- 172.21.15.93:0/2401316746 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e141080f0 0x7f7e1410a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.886+0000 7f7e1a1ff700 1 --2- 172.21.15.93:0/2401316746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e14073550 0x7f7e140739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.886+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2401316746 >> 172.21.15.93:0/2401316746 conn(0x7f7e140fb810 msgr2=0x7f7e140fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:41.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.886+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2401316746 shutdown_connections 2023-11-16T16:48:41.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.886+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2401316746 wait complete. 2023-11-16T16:48:41.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.887+0000 7f7e1a1ff700 1 Processor -- start 2023-11-16T16:48:41.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.887+0000 7f7e1a1ff700 1 -- start start 2023-11-16T16:48:41.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e1a1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e14073550 0x7f7e14198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e1a1ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e14074b30 0x7f7e14198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e1a1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e141080f0 0x7f7e1419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e1a1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1419fd90 con 0x7f7e141080f0 2023-11-16T16:48:41.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e1a1ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1419fed0 con 0x7f7e14074b30 2023-11-16T16:48:41.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e1a1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e141a0060 con 0x7f7e14073550 2023-11-16T16:48:41.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e13fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e141080f0 0x7f7e1419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e137fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e14073550 0x7f7e14198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e14074b30 0x7f7e14198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e137fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e14073550 0x7f7e14198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:34396/0 (socket says 172.21.15.93:34396) 2023-11-16T16:48:41.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e137fe700 1 -- 172.21.15.93:0/2094192248 learned_addr learned my addr 172.21.15.93:0/2094192248 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:41.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e137fe700 1 -- 172.21.15.93:0/2094192248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e14074b30 msgr2=0x7f7e14198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e137fe700 1 --2- 172.21.15.93:0/2094192248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e14074b30 0x7f7e14198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e137fe700 1 -- 172.21.15.93:0/2094192248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e141080f0 msgr2=0x7f7e1419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e137fe700 1 --2- 172.21.15.93:0/2094192248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e141080f0 0x7f7e1419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.888+0000 7f7e137fe700 1 -- 172.21.15.93:0/2094192248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e040095f0 con 0x7f7e14073550 2023-11-16T16:48:41.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.889+0000 7f7e137fe700 1 --2- 172.21.15.93:0/2094192248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e14073550 0x7f7e14198210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7e04004710 tx=0x7f7e04009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:41.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.889+0000 7f7e10ff9700 1 -- 172.21.15.93:0/2094192248 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0401d070 con 0x7f7e14073550 2023-11-16T16:48:41.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.889+0000 7f7e10ff9700 1 -- 172.21.15.93:0/2094192248 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e04021410 con 0x7f7e14073550 2023-11-16T16:48:41.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.889+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2094192248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1410de00 con 0x7f7e14073550 2023-11-16T16:48:41.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.889+0000 7f7e10ff9700 1 -- 172.21.15.93:0/2094192248 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04017650 con 0x7f7e14073550 2023-11-16T16:48:41.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.889+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2094192248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1410dc70 con 0x7f7e14073550 2023-11-16T16:48:41.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.892+0000 7f7e10ff9700 1 -- 172.21.15.93:0/2094192248 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7e04021580 con 0x7f7e14073550 2023-11-16T16:48:41.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.893+0000 7f7e2db8fb80 1 Processor -- start 2023-11-16T16:48:41.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.893+0000 7f7e2db8fb80 1 -- start start 2023-11-16T16:48:41.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.893+0000 7f7e10ff9700 1 --2- 172.21.15.93:0/2094192248 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e2db8fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b26fbb0 0x55b94b26ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e2db8fb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b270500 0x55b94b270ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e10ff9700 1 -- 172.21.15.93:0/2094192248 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f7e040ce140 con 0x7f7e14073550 2023-11-16T16:48:41.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e2db8fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b2710c0 0x55b94b271530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e2db8fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94b175010 con 0x55b94b2710c0 2023-11-16T16:48:41.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e2db8fb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94b1c7850 con 0x55b94b270500 2023-11-16T16:48:41.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e2db8fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94b1c0ce0 con 0x55b94b26fbb0 2023-11-16T16:48:41.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e13fff700 1 --2- 172.21.15.93:0/2094192248 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b270500 0x55b94b270ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b270500 0x55b94b270ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:50362/0 (socket says 172.21.15.93:50362) 2023-11-16T16:48:41.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e13fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b2710c0 0x55b94b271530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e12ffd700 1 -- 172.21.15.93:0/1170390705 learned_addr learned my addr 172.21.15.93:0/1170390705 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:41.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.894+0000 7f7e137fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b26fbb0 0x55b94b26ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.895+0000 7f7e12ffd700 1 -- 172.21.15.93:0/1170390705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b26fbb0 msgr2=0x55b94b26ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:41.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.895+0000 7f7e12ffd700 1 --2- 172.21.15.93:0/1170390705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b26fbb0 0x55b94b26ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.895+0000 7f7e12ffd700 1 -- 172.21.15.93:0/1170390705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b2710c0 msgr2=0x55b94b271530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:41.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.895+0000 7f7e12ffd700 1 --2- 172.21.15.93:0/1170390705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b2710c0 0x55b94b271530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.895+0000 7f7e12ffd700 1 -- 172.21.15.93:0/1170390705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b94b0425e0 con 0x55b94b270500 2023-11-16T16:48:41.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.895+0000 7f7e13fff700 1 --2- 172.21.15.93:0/2094192248 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009ec0 tx=0x7f7e0c0092f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:41.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.895+0000 7f7e12ffd700 1 --2- 172.21.15.93:0/1170390705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b270500 0x55b94b270ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009d30 tx=0x7f7e08011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6a518e6c77b36e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:41.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.895+0000 7f7df9ffb700 1 -- 172.21.15.93:0/1170390705 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08004220 con 0x55b94b270500 2023-11-16T16:48:41.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.895+0000 7f7df9ffb700 1 -- 172.21.15.93:0/1170390705 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e08004380 con 0x55b94b270500 2023-11-16T16:48:41.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.895+0000 7f7df9ffb700 1 -- 172.21.15.93:0/1170390705 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08019810 con 0x55b94b270500 2023-11-16T16:48:41.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.896+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/1170390705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b270500 msgr2=0x55b94b270ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.896+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/1170390705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b270500 0x55b94b270ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009d30 tx=0x7f7e08011720 comp rx=0 tx=0).stop 2023-11-16T16:48:41.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.896+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/1170390705 shutdown_connections 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.896+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/1170390705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b2710c0 0x55b94b271530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.896+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/1170390705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b26fbb0 0x55b94b26ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.896+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/1170390705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b270500 0x55b94b270ae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.896+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/1170390705 >> 172.21.15.93:0/1170390705 conn(0x55b94b265310 msgr2=0x55b94b26a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.896+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/1170390705 shutdown_connections 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.896+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/1170390705 wait complete. 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.896+0000 7f7e2db8fb80 1 -- start start 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e2db8fb80 1 Processor -- start 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e2db8fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b26fbb0 0x55b94b279520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e2db8fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b270500 0x55b94b279a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e2db8fb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b2710c0 0x55b94b280aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e2db8fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94b162890 con 0x55b94b26fbb0 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e2db8fb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94b162530 con 0x55b94b2710c0 2023-11-16T16:48:41.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e2db8fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94b0448d0 con 0x55b94b270500 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e2db8fb80 10 client.0 _ll_register_callbacks cb 0x55b94b180730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e137fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b26fbb0 0x55b94b279520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e137fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b26fbb0 0x55b94b279520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39130/0 (socket says 172.21.15.93:39130) 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e137fe700 1 -- 172.21.15.93:0/2554526011 learned_addr learned my addr 172.21.15.93:0/2554526011 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e13fff700 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b2710c0 0x55b94b280aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.897+0000 7f7e12ffd700 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b270500 0x55b94b279a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.898+0000 7f7e137fe700 1 -- 172.21.15.93:0/2554526011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b270500 msgr2=0x55b94b279a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.898+0000 7f7e137fe700 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b270500 0x55b94b279a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.898+0000 7f7e137fe700 1 -- 172.21.15.93:0/2554526011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b2710c0 msgr2=0x55b94b280aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.898+0000 7f7e137fe700 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b2710c0 0x55b94b280aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.898+0000 7f7e137fe700 1 -- 172.21.15.93:0/2554526011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e08009960 con 0x55b94b26fbb0 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.898+0000 7f7e13fff700 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b2710c0 0x55b94b280aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:41.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.898+0000 7f7e137fe700 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b26fbb0 0x55b94b279520 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7e0409b140 tx=0x7f7e04021b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548b786bcef2773 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:41.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.898+0000 7f7df2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:48:41.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.898+0000 7f7df2ffd700 1 -- 172.21.15.93:0/2554526011 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0409a8a0 con 0x55b94b26fbb0 2023-11-16T16:48:41.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.898+0000 7f7df2ffd700 1 -- 172.21.15.93:0/2554526011 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f7e0409f1e0 con 0x55b94b26fbb0 2023-11-16T16:48:41.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.898+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2554526011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b94b262ad0 con 0x55b94b26fbb0 2023-11-16T16:48:41.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.899+0000 7f7df2ffd700 1 -- 172.21.15.93:0/2554526011 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e040a3570 con 0x55b94b26fbb0 2023-11-16T16:48:41.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.899+0000 7f7e2db8fb80 10 client.15267 fetch_fsmap learned FSMap version 23 2023-11-16T16:48:41.903 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:41 smithi093 ceph-mon[97863]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:41.903 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:41 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/38425654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:48:41.903 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:41 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3245043834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:41.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.899+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2554526011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b94b187140 con 0x55b94b26fbb0 2023-11-16T16:48:41.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.899+0000 7f7df2ffd700 1 -- 172.21.15.93:0/2554526011 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e040c9080 con 0x55b94b26fbb0 2023-11-16T16:48:41.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.899+0000 7f7df2ffd700 1 -- 172.21.15.93:0/2554526011 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e0409a110 con 0x55b94b26fbb0 2023-11-16T16:48:41.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.899+0000 7f7e2db8fb80 10 client.15267 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:48:41.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.899+0000 7f7e2db8fb80 10 client.15267 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-16T16:48:41.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.899+0000 7f7e2db8fb80 20 client.15267 populate_metadata read hostname 'smithi093' 2023-11-16T16:48:41.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.899+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55b94b2843d0 0x55b94b2839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.899+0000 7f7e2db8fb80 4 client.15267 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:41.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.899+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2554526011 --> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b94b183050 con 0x55b94b2843d0 2023-11-16T16:48:41.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.900+0000 7f7e12ffd700 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55b94b2843d0 0x55b94b2839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.900+0000 7f7df2ffd700 10 client.15267 ms_handle_connect on v2:172.21.15.125:6836/1957598038 2023-11-16T16:48:41.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.900+0000 7f7e12ffd700 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55b94b2843d0 0x55b94b2839d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1c130823dad90bf server_cookie=782efdde49800157 in_seq=0 out_seq=0 2023-11-16T16:48:41.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7df2ffd700 1 -- 172.21.15.93:0/2554526011 <== mds.2 v2:172.21.15.125:6836/1957598038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b94b183050 con 0x55b94b2843d0 2023-11-16T16:48:41.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7df2ffd700 10 client.15267 handle_command_reply: tid=0 2023-11-16T16:48:41.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7e2db8fb80 1 client.15267 shutdown 2023-11-16T16:48:41.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2554526011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b26fbb0 msgr2=0x55b94b279520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b26fbb0 0x55b94b279520 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7e0409b140 tx=0x7f7e04021b60 comp rx=0 tx=0).stop 2023-11-16T16:48:41.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2554526011 shutdown_connections 2023-11-16T16:48:41.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55b94b2843d0 0x55b94b2839d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b26fbb0 0x55b94b279520 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b2710c0 0x55b94b280aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7df2ffd700 0 client.15267 ms_handle_reset on v2:172.21.15.125:6836/1957598038 2023-11-16T16:48:41.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2554526011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b270500 0x55b94b279a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2554526011 >> 172.21.15.93:0/2554526011 conn(0x55b94b265310 msgr2=0x55b94b269d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:41.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2554526011 shutdown_connections 2023-11-16T16:48:41.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.901+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2554526011 wait complete. 2023-11-16T16:48:41.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.902+0000 7f7e2db8fb80 20 client.15267 trim_cache size 0 max 16384 2023-11-16T16:48:41.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.919+0000 7f7e2db8fb80 1 Processor -- start 2023-11-16T16:48:41.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.919+0000 7f7e2db8fb80 1 -- start start 2023-11-16T16:48:41.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.919+0000 7f7e2db8fb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b265310 0x55b94b273f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.919+0000 7f7e2db8fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b26fbb0 0x55b94b274470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.919+0000 7f7e2db8fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 0x55b94b280a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.919+0000 7f7e2db8fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94b2840e0 con 0x55b94b270500 2023-11-16T16:48:41.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.919+0000 7f7e2db8fb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94b16a730 con 0x55b94b265310 2023-11-16T16:48:41.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.919+0000 7f7e2db8fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94b0cced0 con 0x55b94b26fbb0 2023-11-16T16:48:41.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.920+0000 7f7e13fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 0x55b94b280a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.920+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b26fbb0 0x55b94b274470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.920+0000 7f7e13fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 0x55b94b280a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39132/0 (socket says 172.21.15.93:39132) 2023-11-16T16:48:41.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.920+0000 7f7e13fff700 1 -- 172.21.15.93:0/2712632972 learned_addr learned my addr 172.21.15.93:0/2712632972 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:41.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.920+0000 7f7e137fe700 1 --2- 172.21.15.93:0/2712632972 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b265310 0x55b94b273f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.920+0000 7f7e12ffd700 1 -- 172.21.15.93:0/2712632972 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b265310 msgr2=0x55b94b273f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:41.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.920+0000 7f7e12ffd700 1 --2- 172.21.15.93:0/2712632972 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b265310 0x55b94b273f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.920+0000 7f7e12ffd700 1 -- 172.21.15.93:0/2712632972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 msgr2=0x55b94b280a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:41.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.920+0000 7f7e12ffd700 1 --2- 172.21.15.93:0/2712632972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 0x55b94b280a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.920+0000 7f7e12ffd700 1 -- 172.21.15.93:0/2712632972 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b94b186e90 con 0x55b94b26fbb0 2023-11-16T16:48:41.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.921+0000 7f7e12ffd700 1 --2- 172.21.15.93:0/2712632972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b26fbb0 0x55b94b274470 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009230 tx=0x7f7e08011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=225339918284bf11 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:41.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.921+0000 7f7dfa7fc700 1 -- 172.21.15.93:0/2712632972 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e080044f0 con 0x55b94b26fbb0 2023-11-16T16:48:41.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.921+0000 7f7dfa7fc700 1 -- 172.21.15.93:0/2712632972 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e08003680 con 0x55b94b26fbb0 2023-11-16T16:48:41.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.921+0000 7f7dfa7fc700 1 -- 172.21.15.93:0/2712632972 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0802aae0 con 0x55b94b26fbb0 2023-11-16T16:48:41.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.921+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2712632972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b26fbb0 msgr2=0x55b94b274470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.921+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2712632972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b26fbb0 0x55b94b274470 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009230 tx=0x7f7e08011720 comp rx=0 tx=0).stop 2023-11-16T16:48:41.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.921+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2712632972 shutdown_connections 2023-11-16T16:48:41.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.921+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2712632972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 0x55b94b280a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.921+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2712632972 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b265310 0x55b94b273f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.922+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2712632972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b26fbb0 0x55b94b274470 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.922+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2712632972 >> 172.21.15.93:0/2712632972 conn(0x55b94b2785d0 msgr2=0x55b94b2839d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.922+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2712632972 shutdown_connections 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.922+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2712632972 wait complete. 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.922+0000 7f7e2db8fb80 1 -- start start 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.922+0000 7f7e2db8fb80 1 Processor -- start 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.923+0000 7f7e2db8fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b265310 0x55b94b27b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.923+0000 7f7e2db8fb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b26fbb0 0x55b94b2809a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.923+0000 7f7e2db8fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 0x55b94b27b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.923+0000 7f7e2db8fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94b2d8270 con 0x55b94b270500 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.923+0000 7f7e2db8fb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94b2d83b0 con 0x55b94b26fbb0 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.923+0000 7f7e2db8fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94b2d84f0 con 0x55b94b265310 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.923+0000 7f7e2db8fb80 10 client.0 _ll_register_callbacks cb 0x7f7e0409afc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.923+0000 7f7e13fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 0x55b94b27b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.923+0000 7f7e137fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b265310 0x55b94b27b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.923+0000 7f7e13fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 0x55b94b27b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39138/0 (socket says 172.21.15.93:39138) 2023-11-16T16:48:41.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.923+0000 7f7e13fff700 1 -- 172.21.15.93:0/2055930067 learned_addr learned my addr 172.21.15.93:0/2055930067 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:41.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.923+0000 7f7e12ffd700 1 --2- 172.21.15.93:0/2055930067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b26fbb0 0x55b94b2809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.924+0000 7f7e13fff700 1 -- 172.21.15.93:0/2055930067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b265310 msgr2=0x55b94b27b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:41.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.924+0000 7f7e13fff700 1 --2- 172.21.15.93:0/2055930067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b265310 0x55b94b27b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.924+0000 7f7e13fff700 1 -- 172.21.15.93:0/2055930067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b26fbb0 msgr2=0x55b94b2809a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:41.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.924+0000 7f7e13fff700 1 --2- 172.21.15.93:0/2055930067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b26fbb0 0x55b94b2809a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.924+0000 7f7e13fff700 1 -- 172.21.15.93:0/2055930067 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b94b0425e0 con 0x55b94b270500 2023-11-16T16:48:41.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.924+0000 7f7e13fff700 1 --2- 172.21.15.93:0/2055930067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 0x55b94b27b9d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c01b700 tx=0x7f7e0c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9980aef13c846a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:41.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.924+0000 7f7df2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:48:41.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.924+0000 7f7df2ffd700 1 -- 172.21.15.93:0/2055930067 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00e070 con 0x55b94b270500 2023-11-16T16:48:41.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.924+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2055930067 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e08004380 con 0x55b94b270500 2023-11-16T16:48:41.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.924+0000 7f7df2ffd700 1 -- 172.21.15.93:0/2055930067 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f7e0c021040 con 0x55b94b270500 2023-11-16T16:48:41.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.925+0000 7f7df2ffd700 1 -- 172.21.15.93:0/2055930067 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0c008570 con 0x55b94b270500 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.925+0000 7f7e2db8fb80 10 client.15273 fetch_fsmap learned FSMap version 23 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.925+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2055930067 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b94afd7c10 con 0x55b94b270500 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.925+0000 7f7df2ffd700 1 -- 172.21.15.93:0/2055930067 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e08004380 con 0x55b94b270500 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.925+0000 7f7df2ffd700 1 -- 172.21.15.93:0/2055930067 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e0c0326e0 con 0x55b94b270500 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.925+0000 7f7e2db8fb80 10 client.15273 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.925+0000 7f7e2db8fb80 10 client.15273 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.925+0000 7f7e2db8fb80 20 client.15273 populate_metadata read hostname 'smithi093' 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.925+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2055930067 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55b94b2710c0 0x55b94b2d9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.925+0000 7f7e2db8fb80 4 client.15273 mds_command: new command op to 24533 tid=0[{"prefix": "status"}] 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.925+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2055930067 --> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b94b18f610 con 0x55b94b2710c0 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.925+0000 7f7e137fe700 1 --2- 172.21.15.93:0/2055930067 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55b94b2710c0 0x55b94b2d9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.926+0000 7f7df2ffd700 10 client.15273 ms_handle_connect on v2:172.21.15.125:6836/1957598038 2023-11-16T16:48:41.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.926+0000 7f7e137fe700 1 --2- 172.21.15.93:0/2055930067 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55b94b2710c0 0x55b94b2d9c50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e75b6a0ea7c849e server_cookie=a90f13971564a940 in_seq=0 out_seq=0 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.926+0000 7f7df2ffd700 1 -- 172.21.15.93:0/2055930067 <== mds.2 v2:172.21.15.125:6836/1957598038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b94b18f610 con 0x55b94b2710c0 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.926+0000 7f7df2ffd700 10 client.15273 handle_command_reply: tid=0 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.926+0000 7f7e2db8fb80 1 client.15273 shutdown 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.926+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2055930067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 msgr2=0x55b94b27b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.926+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2055930067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 0x55b94b27b9d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c01b700 tx=0x7f7e0c00b040 comp rx=0 tx=0).stop 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.926+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2055930067 shutdown_connections 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2055930067 >> [v2:172.21.15.125:6836/1957598038,v1:172.21.15.125:6837/1957598038] conn(0x55b94b2710c0 0x55b94b2d9c50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2055930067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b94b270500 0x55b94b27b9d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7df2ffd700 0 client.15273 ms_handle_reset on v2:172.21.15.125:6836/1957598038 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2055930067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b94b265310 0x55b94b27b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e2db8fb80 1 --2- 172.21.15.93:0/2055930067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b94b26fbb0 0x55b94b2809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2055930067 >> 172.21.15.93:0/2055930067 conn(0x55b94b2785d0 msgr2=0x55b94b277420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:41.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2055930067 shutdown_connections 2023-11-16T16:48:41.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e2db8fb80 1 -- 172.21.15.93:0/2055930067 wait complete. 2023-11-16T16:48:41.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e2db8fb80 20 client.15273 trim_cache size 0 max 16384 2023-11-16T16:48:41.932 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:48:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "6a59c1a8-849e-11ee-8dbe-212e2dc638e7", 2023-11-16T16:48:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 2, 2023-11-16T16:48:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24533, 2023-11-16T16:48:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-11-16T16:48:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-11-16T16:48:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-11-16T16:48:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 92.286975351999999, 2023-11-16T16:48:41.933 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 23, 2023-11-16T16:48:41.933 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 92, 2023-11-16T16:48:41.933 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 90, 2023-11-16T16:48:41.933 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 143.45297432000001 2023-11-16T16:48:41.933 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:48:41.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2094192248 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7dfc0a1150 msgr2=0x7f7dfc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e1a1ff700 1 --2- 172.21.15.93:0/2094192248 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009ec0 tx=0x7f7e0c0092f0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2094192248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e14073550 msgr2=0x7f7e14198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:41.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e1a1ff700 1 --2- 172.21.15.93:0/2094192248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e14073550 0x7f7e14198210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7e04004710 tx=0x7f7e04009580 comp rx=0 tx=0).stop 2023-11-16T16:48:41.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2094192248 shutdown_connections 2023-11-16T16:48:41.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e1a1ff700 1 --2- 172.21.15.93:0/2094192248 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e1a1ff700 1 --2- 172.21.15.93:0/2094192248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e141080f0 0x7f7e1419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e1a1ff700 1 --2- 172.21.15.93:0/2094192248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e14073550 0x7f7e14198210 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e1a1ff700 1 --2- 172.21.15.93:0/2094192248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e14074b30 0x7f7e14198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:41.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.927+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2094192248 >> 172.21.15.93:0/2094192248 conn(0x7f7e140fb810 msgr2=0x7f7e140fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:41.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.928+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2094192248 shutdown_connections 2023-11-16T16:48:41.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:41.928+0000 7f7e1a1ff700 1 -- 172.21.15.93:0/2094192248 wait complete. 2023-11-16T16:48:41.947 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-11-16T16:48:41.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:41 smithi159 ceph-mon[131528]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:41.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:41 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/38425654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:48:41.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:41 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3245043834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:42.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:41 smithi125 ceph-mon[131076]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:42.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:41 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/38425654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:48:42.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:41 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3245043834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:42.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.144+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1702194184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4e4073550 msgr2=0x7fe4e40739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.144+0000 7fe4ea1ff700 1 --2- 172.21.15.93:0/1702194184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4e4073550 0x7fe4e40739c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80098f0 tx=0x7fe4d8009c00 comp rx=0 tx=0).stop 2023-11-16T16:48:42.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.144+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1702194184 shutdown_connections 2023-11-16T16:48:42.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.144+0000 7fe4ea1ff700 1 --2- 172.21.15.93:0/1702194184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e41080f0 0x7fe4e410a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.144+0000 7fe4ea1ff700 1 --2- 172.21.15.93:0/1702194184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4e4074b30 0x7fe4e4072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.144+0000 7fe4ea1ff700 1 --2- 172.21.15.93:0/1702194184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4e4073550 0x7fe4e40739c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.144+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1702194184 >> 172.21.15.93:0/1702194184 conn(0x7fe4e40fb810 msgr2=0x7fe4e40fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:42.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.144+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1702194184 shutdown_connections 2023-11-16T16:48:42.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.144+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1702194184 wait complete. 2023-11-16T16:48:42.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.145+0000 7fe4ea1ff700 1 Processor -- start 2023-11-16T16:48:42.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.145+0000 7fe4ea1ff700 1 -- start start 2023-11-16T16:48:42.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.145+0000 7fe4ea1ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4e4073550 0x7fe4e4198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.145+0000 7fe4ea1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4e4074b30 0x7fe4e4198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.145+0000 7fe4ea1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e41080f0 0x7fe4e419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4ea1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e419fdd0 con 0x7fe4e41080f0 2023-11-16T16:48:42.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4ea1ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e419ff10 con 0x7fe4e4073550 2023-11-16T16:48:42.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4ea1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e41a00a0 con 0x7fe4e4074b30 2023-11-16T16:48:42.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4e2ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4e4074b30 0x7fe4e4198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4e3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e41080f0 0x7fe4e419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4e2ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4e4074b30 0x7fe4e4198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:34450/0 (socket says 172.21.15.93:34450) 2023-11-16T16:48:42.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4e2ffd700 1 -- 172.21.15.93:0/1086093820 learned_addr learned my addr 172.21.15.93:0/1086093820 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:42.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4e37fe700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4e4073550 0x7fe4e4198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4e3fff700 1 -- 172.21.15.93:0/1086093820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4e4074b30 msgr2=0x7fe4e4198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4e3fff700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4e4074b30 0x7fe4e4198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4e3fff700 1 -- 172.21.15.93:0/1086093820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4e4073550 msgr2=0x7fe4e4198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4e3fff700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4e4073550 0x7fe4e4198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4e3fff700 1 -- 172.21.15.93:0/1086093820 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d8009520 con 0x7fe4e41080f0 2023-11-16T16:48:42.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.146+0000 7fe4e3fff700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e41080f0 0x7fe4e419f800 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00efc0 tx=0x7fe4dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:42.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.147+0000 7fe4e0ff9700 1 -- 172.21.15.93:0/1086093820 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc018070 con 0x7fe4e41080f0 2023-11-16T16:48:42.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.147+0000 7fe4e0ff9700 1 -- 172.21.15.93:0/1086093820 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4dc00f040 con 0x7fe4e41080f0 2023-11-16T16:48:42.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.147+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1086093820 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e410de10 con 0x7fe4e41080f0 2023-11-16T16:48:42.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.147+0000 7fe4e0ff9700 1 -- 172.21.15.93:0/1086093820 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc00ccb0 con 0x7fe4e41080f0 2023-11-16T16:48:42.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.147+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1086093820 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e410dc80 con 0x7fe4e41080f0 2023-11-16T16:48:42.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.150+0000 7fe4e0ff9700 1 -- 172.21.15.93:0/1086093820 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe4dc003a00 con 0x7fe4e41080f0 2023-11-16T16:48:42.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.151+0000 7fe4fdba4b80 1 Processor -- start 2023-11-16T16:48:42.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.151+0000 7fe4e0ff9700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe4cc0a1150 0x7fe4cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.151+0000 7fe4fdba4b80 1 -- start start 2023-11-16T16:48:42.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4e0ff9700 1 -- 172.21.15.93:0/1086093820 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe4dc0cc6f0 con 0x7fe4e41080f0 2023-11-16T16:48:42.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4fdba4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x557440933fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4fdba4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 0x557440934ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4e2ffd700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe4cc0a1150 0x7fe4cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4fdba4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x557440935530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4fdba4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557440839010 con 0x557440934500 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4fdba4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744088b850 con 0x5574409350c0 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4fdba4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557440884ce0 con 0x557440933bb0 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 0x557440934ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4e37fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x557440933fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4e3fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x557440935530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 0x557440934ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39166/0 (socket says 172.21.15.93:39166) 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.152+0000 7fe4e2ffd700 1 -- 172.21.15.93:0/2297609198 learned_addr learned my addr 172.21.15.93:0/2297609198 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.153+0000 7fe4e3fff700 1 -- 172.21.15.93:0/2297609198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 msgr2=0x557440933fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.153+0000 7fe4e3fff700 1 --2- 172.21.15.93:0/2297609198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x557440933fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.153+0000 7fe4e3fff700 1 -- 172.21.15.93:0/2297609198 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 msgr2=0x557440934ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.153+0000 7fe4e3fff700 1 --2- 172.21.15.93:0/2297609198 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 0x557440934ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.153+0000 7fe4e3fff700 1 -- 172.21.15.93:0/2297609198 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574407065e0 con 0x5574409350c0 2023-11-16T16:48:42.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.153+0000 7fe4e37fe700 1 --2- 172.21.15.93:0/2297609198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x557440933fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:42.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.153+0000 7fe4e2ffd700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe4cc0a1150 0x7fe4cc0a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009fd0 tx=0x7fe4d800bfd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:42.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.153+0000 7fe4e3fff700 1 --2- 172.21.15.93:0/2297609198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x557440935530 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0c9e20 tx=0x7fe4dc0993d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4e447393a721bed server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:42.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.153+0000 7fe4c9ffb700 1 -- 172.21.15.93:0/2297609198 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc0997c0 con 0x5574409350c0 2023-11-16T16:48:42.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.153+0000 7fe4c9ffb700 1 -- 172.21.15.93:0/2297609198 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4dc09d7f0 con 0x5574409350c0 2023-11-16T16:48:42.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.153+0000 7fe4c9ffb700 1 -- 172.21.15.93:0/2297609198 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc004bf0 con 0x5574409350c0 2023-11-16T16:48:42.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.154+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/2297609198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 msgr2=0x557440935530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.154+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/2297609198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x557440935530 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0c9e20 tx=0x7fe4dc0993d0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.154+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/2297609198 shutdown_connections 2023-11-16T16:48:42.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.154+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/2297609198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x557440935530 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.154+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/2297609198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x557440933fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.154+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/2297609198 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 0x557440934ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.154+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/2297609198 >> 172.21.15.93:0/2297609198 conn(0x557440929310 msgr2=0x55744092e620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:42.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.154+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/2297609198 shutdown_connections 2023-11-16T16:48:42.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.154+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/2297609198 wait complete. 2023-11-16T16:48:42.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.155+0000 7fe4fdba4b80 1 -- start start 2023-11-16T16:48:42.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.155+0000 7fe4fdba4b80 1 Processor -- start 2023-11-16T16:48:42.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4fdba4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x55744093d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4fdba4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557440934500 0x55744093db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4fdba4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5574409350c0 0x557440944ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4fdba4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557440826890 con 0x5574409350c0 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4fdba4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557440826530 con 0x557440934500 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4fdba4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574407088d0 con 0x557440933bb0 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4fdba4b80 10 client.0 _ll_register_callbacks cb 0x557440844730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4e3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5574409350c0 0x557440944ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4e2ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557440934500 0x55744093db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4e37fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x55744093d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4e37fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x55744093d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:34474/0 (socket says 172.21.15.93:34474) 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4e37fe700 1 -- 172.21.15.93:0/1764912122 learned_addr learned my addr 172.21.15.93:0/1764912122 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4e2ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557440934500 0x55744093db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:50434/0 (socket says 172.21.15.93:50434) 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4e2ffd700 1 -- 172.21.15.93:0/1764912122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 msgr2=0x55744093d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4e2ffd700 1 --2- 172.21.15.93:0/1764912122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x55744093d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4e2ffd700 1 -- 172.21.15.93:0/1764912122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5574409350c0 msgr2=0x557440944ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4e2ffd700 1 --2- 172.21.15.93:0/1764912122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5574409350c0 0x557440944ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4e2ffd700 1 -- 172.21.15.93:0/1764912122 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e410dc80 con 0x557440934500 2023-11-16T16:48:42.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.156+0000 7fe4e3fff700 1 --2- 172.21.15.93:0/1764912122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5574409350c0 0x557440944ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:42.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.157+0000 7fe4e2ffd700 1 --2- 172.21.15.93:0/1764912122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557440934500 0x55744093db60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80098c0 tx=0x7fe4d8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75fc161f1a32033a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:42.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.157+0000 7fe4c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:48:42.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.157+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1764912122 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8033070 con 0x557440934500 2023-11-16T16:48:42.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.157+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1764912122 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe4d80056b0 con 0x557440934500 2023-11-16T16:48:42.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.157+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/1764912122 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557440926ad0 con 0x557440934500 2023-11-16T16:48:42.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.157+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1764912122 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4d8003680 con 0x557440934500 2023-11-16T16:48:42.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.157+0000 7fe4fdba4b80 10 client.25322 fetch_fsmap learned FSMap version 23 2023-11-16T16:48:42.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.157+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/1764912122 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55744084b140 con 0x557440934500 2023-11-16T16:48:42.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.158+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1764912122 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557440926ad0 con 0x557440934500 2023-11-16T16:48:42.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.158+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1764912122 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe4d8036030 con 0x557440934500 2023-11-16T16:48:42.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.158+0000 7fe4fdba4b80 10 client.25322 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.158+0000 7fe4fdba4b80 10 client.25322 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.158+0000 7fe4fdba4b80 20 client.25322 populate_metadata read hostname 'smithi093' 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.158+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/1764912122 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x557440948750 0x557440947be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.158+0000 7fe4fdba4b80 4 client.25322 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.158+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/1764912122 --> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557440847050 con 0x557440948750 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.158+0000 7fe4e37fe700 1 --2- 172.21.15.93:0/1764912122 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x557440948750 0x557440947be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.159+0000 7fe4e37fe700 1 --2- 172.21.15.93:0/1764912122 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x557440948750 0x557440947be0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e713b684621c0d51 server_cookie=271312338dd448c4 in_seq=0 out_seq=0 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.159+0000 7fe4c2ffd700 10 client.25322 ms_handle_connect on v2:172.21.15.159:6838/149300707 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1764912122 <== mds.1 v2:172.21.15.159:6838/149300707 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557440847050 con 0x557440948750 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4c2ffd700 10 client.25322 handle_command_reply: tid=0 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4fdba4b80 1 client.25322 shutdown 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/1764912122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557440934500 msgr2=0x55744093db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/1764912122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557440934500 0x55744093db60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80098c0 tx=0x7fe4d8009c00 comp rx=0 tx=0).stop 2023-11-16T16:48:42.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/1764912122 shutdown_connections 2023-11-16T16:48:42.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/1764912122 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x557440948750 0x557440947be0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/1764912122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5574409350c0 0x557440944ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4c2ffd700 0 client.25322 ms_handle_reset on v2:172.21.15.159:6838/149300707 2023-11-16T16:48:42.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/1764912122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x55744093d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/1764912122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557440934500 0x55744093db60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/1764912122 >> 172.21.15.93:0/1764912122 conn(0x557440929310 msgr2=0x55744092dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:42.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/1764912122 shutdown_connections 2023-11-16T16:48:42.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/1764912122 wait complete. 2023-11-16T16:48:42.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.160+0000 7fe4fdba4b80 20 client.25322 trim_cache size 0 max 16384 2023-11-16T16:48:42.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.178+0000 7fe4fdba4b80 1 Processor -- start 2023-11-16T16:48:42.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.178+0000 7fe4fdba4b80 1 -- start start 2023-11-16T16:48:42.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.178+0000 7fe4fdba4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440933bb0 0x557440938000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.178+0000 7fe4fdba4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440934500 0x557440938540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.178+0000 7fe4fdba4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x557440944ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.178+0000 7fe4fdba4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574409301b0 con 0x557440933bb0 2023-11-16T16:48:42.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.178+0000 7fe4fdba4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557440935930 con 0x5574409350c0 2023-11-16T16:48:42.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.178+0000 7fe4fdba4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557440847050 con 0x557440934500 2023-11-16T16:48:42.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.178+0000 7fe4e37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440933bb0 0x557440938000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.179+0000 7fe4e37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440933bb0 0x557440938000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39180/0 (socket says 172.21.15.93:39180) 2023-11-16T16:48:42.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.179+0000 7fe4e37fe700 1 -- 172.21.15.93:0/2267952881 learned_addr learned my addr 172.21.15.93:0/2267952881 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:42.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.179+0000 7fe4e3fff700 1 --2- 172.21.15.93:0/2267952881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x557440944ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.179+0000 7fe4e2ffd700 1 --2- 172.21.15.93:0/2267952881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440934500 0x557440938540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.179+0000 7fe4e37fe700 1 -- 172.21.15.93:0/2267952881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440934500 msgr2=0x557440938540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.179+0000 7fe4e37fe700 1 --2- 172.21.15.93:0/2267952881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440934500 0x557440938540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.179+0000 7fe4e37fe700 1 -- 172.21.15.93:0/2267952881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 msgr2=0x557440944ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.179+0000 7fe4e37fe700 1 --2- 172.21.15.93:0/2267952881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x557440944ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.179+0000 7fe4e37fe700 1 -- 172.21.15.93:0/2267952881 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55744084ae90 con 0x557440933bb0 2023-11-16T16:48:42.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.179+0000 7fe4e37fe700 1 --2- 172.21.15.93:0/2267952881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440933bb0 0x557440938000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55744093e1b0 tx=0x7fe4d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a96239263bf382 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:42.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.179+0000 7fe4ca7fc700 1 -- 172.21.15.93:0/2267952881 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4017040 con 0x557440933bb0 2023-11-16T16:48:42.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.180+0000 7fe4ca7fc700 1 -- 172.21.15.93:0/2267952881 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4d4018040 con 0x557440933bb0 2023-11-16T16:48:42.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.180+0000 7fe4ca7fc700 1 -- 172.21.15.93:0/2267952881 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4012e40 con 0x557440933bb0 2023-11-16T16:48:42.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.180+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/2267952881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440933bb0 msgr2=0x557440938000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.180+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/2267952881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440933bb0 0x557440938000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55744093e1b0 tx=0x7fe4d4009260 comp rx=0 tx=0).stop 2023-11-16T16:48:42.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.180+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/2267952881 shutdown_connections 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.180+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/2267952881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440933bb0 0x557440938000 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.180+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/2267952881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x557440944ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.180+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/2267952881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440934500 0x557440938540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.180+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/2267952881 >> 172.21.15.93:0/2267952881 conn(0x557440929310 msgr2=0x557440947be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.180+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/2267952881 shutdown_connections 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.180+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/2267952881 wait complete. 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4fdba4b80 1 -- start start 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4fdba4b80 1 Processor -- start 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4fdba4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x55744093f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4fdba4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 0x55744093fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4fdba4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x55744099c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4fdba4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557440944c30 con 0x557440934500 2023-11-16T16:48:42.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4fdba4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557440944da0 con 0x5574409350c0 2023-11-16T16:48:42.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4fdba4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744099caf0 con 0x557440933bb0 2023-11-16T16:48:42.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4fdba4b80 10 client.0 _ll_register_callbacks cb 0x557440844730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:42.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 0x55744093fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 0x55744093fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39186/0 (socket says 172.21.15.93:39186) 2023-11-16T16:48:42.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4e37fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x55744093f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.181+0000 7fe4e2ffd700 1 -- 172.21.15.93:0/3994208370 learned_addr learned my addr 172.21.15.93:0/3994208370 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:42.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4e3fff700 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x55744099c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4e37fe700 1 -- 172.21.15.93:0/3994208370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 msgr2=0x55744099c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4e37fe700 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x55744099c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4e37fe700 1 -- 172.21.15.93:0/3994208370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 msgr2=0x55744093fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4e37fe700 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 0x55744093fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4e37fe700 1 -- 172.21.15.93:0/3994208370 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d4009cd0 con 0x557440933bb0 2023-11-16T16:48:42.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4e2ffd700 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 0x55744093fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:42.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4e3fff700 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x55744099c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:42.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4e37fe700 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x55744093f6c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x557440926b20 tx=0x7fe4d4012970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2df447c949598f2f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:42.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-11-16T16:48:42.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/3994208370 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4012dd0 con 0x557440933bb0 2023-11-16T16:48:42.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/3994208370 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4dc09d7f0 con 0x557440933bb0 2023-11-16T16:48:42.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.182+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/3994208370 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe4d40210f0 con 0x557440933bb0 2023-11-16T16:48:42.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.183+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/3994208370 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4d40212b0 con 0x557440933bb0 2023-11-16T16:48:42.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.183+0000 7fe4fdba4b80 10 client.25321 fetch_fsmap learned FSMap version 23 2023-11-16T16:48:42.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.183+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/3994208370 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55744069bc10 con 0x557440933bb0 2023-11-16T16:48:42.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.183+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/3994208370 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4dc09d7f0 con 0x557440933bb0 2023-11-16T16:48:42.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.183+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/3994208370 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe4d4004870 con 0x557440933bb0 2023-11-16T16:48:42.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.183+0000 7fe4fdba4b80 10 client.25321 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:48:42.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.183+0000 7fe4fdba4b80 10 client.25321 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-16T16:48:42.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.183+0000 7fe4fdba4b80 20 client.25321 populate_metadata read hostname 'smithi093' 2023-11-16T16:48:42.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.183+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x5574409a1780 0x55744099e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.183+0000 7fe4fdba4b80 4 client.25321 mds_command: new command op to 24541 tid=0[{"prefix": "status"}] 2023-11-16T16:48:42.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.183+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/3994208370 --> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557440853610 con 0x5574409a1780 2023-11-16T16:48:42.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.184+0000 7fe4e2ffd700 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x5574409a1780 0x55744099e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.184+0000 7fe4c2ffd700 10 client.25321 ms_handle_connect on v2:172.21.15.159:6838/149300707 2023-11-16T16:48:42.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.184+0000 7fe4e2ffd700 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x5574409a1780 0x55744099e0d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8715c52d305f360 server_cookie=8073942bc63d6660 in_seq=0 out_seq=0 2023-11-16T16:48:42.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.184+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/3994208370 <== mds.1 v2:172.21.15.159:6838/149300707 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557440853610 con 0x5574409a1780 2023-11-16T16:48:42.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.184+0000 7fe4c2ffd700 10 client.25321 handle_command_reply: tid=0 2023-11-16T16:48:42.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.184+0000 7fe4fdba4b80 1 client.25321 shutdown 2023-11-16T16:48:42.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.184+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/3994208370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 msgr2=0x55744093f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.184+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x55744093f6c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x557440926b20 tx=0x7fe4d4012970 comp rx=0 tx=0).stop 2023-11-16T16:48:42.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/3994208370 shutdown_connections 2023-11-16T16:48:42.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.159:6838/149300707,v1:172.21.15.159:6839/149300707] conn(0x5574409a1780 0x55744099e0d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574409350c0 0x55744099c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557440933bb0 0x55744093f6c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4c2ffd700 0 client.25321 ms_handle_reset on v2:172.21.15.159:6838/149300707 2023-11-16T16:48:42.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4fdba4b80 1 --2- 172.21.15.93:0/3994208370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557440934500 0x55744093fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/3994208370 >> 172.21.15.93:0/3994208370 conn(0x557440929310 msgr2=0x557440938000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:42.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/3994208370 shutdown_connections 2023-11-16T16:48:42.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4fdba4b80 1 -- 172.21.15.93:0/3994208370 wait complete. 2023-11-16T16:48:42.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4fdba4b80 20 client.25321 trim_cache size 0 max 16384 2023-11-16T16:48:42.191 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:48:42.191 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "6a59c1a8-849e-11ee-8dbe-212e2dc638e7", 2023-11-16T16:48:42.191 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 1, 2023-11-16T16:48:42.191 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24541, 2023-11-16T16:48:42.192 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-11-16T16:48:42.192 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-11-16T16:48:42.192 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-11-16T16:48:42.192 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 93.551789876000001, 2023-11-16T16:48:42.192 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 23, 2023-11-16T16:48:42.192 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 92, 2023-11-16T16:48:42.192 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 90, 2023-11-16T16:48:42.192 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 141.43970125999999 2023-11-16T16:48:42.192 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:48:42.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1086093820 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe4cc0a1150 msgr2=0x7fe4cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4ea1ff700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe4cc0a1150 0x7fe4cc0a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009fd0 tx=0x7fe4d800bfd0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1086093820 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e41080f0 msgr2=0x7fe4e419f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4ea1ff700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e41080f0 0x7fe4e419f800 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00efc0 tx=0x7fe4dc00c3a0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1086093820 shutdown_connections 2023-11-16T16:48:42.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4ea1ff700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe4cc0a1150 0x7fe4cc0a3600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4ea1ff700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e41080f0 0x7fe4e419f800 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4ea1ff700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4e4073550 0x7fe4e4198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4ea1ff700 1 --2- 172.21.15.93:0/1086093820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4e4074b30 0x7fe4e4198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.185+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1086093820 >> 172.21.15.93:0/1086093820 conn(0x7fe4e40fb810 msgr2=0x7fe4e40fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:42.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.186+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1086093820 shutdown_connections 2023-11-16T16:48:42.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.186+0000 7fe4ea1ff700 1 -- 172.21.15.93:0/1086093820 wait complete. 2023-11-16T16:48:42.205 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-16T16:48:42.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.402+0000 7efdce7ff700 1 -- 172.21.15.93:0/3047055531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdc8101420 msgr2=0x7efdc8101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.402+0000 7efdce7ff700 1 --2- 172.21.15.93:0/3047055531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdc8101420 0x7efdc8101870 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009990 tx=0x7efdbc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.402+0000 7efdce7ff700 1 -- 172.21.15.93:0/3047055531 shutdown_connections 2023-11-16T16:48:42.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.402+0000 7efdce7ff700 1 --2- 172.21.15.93:0/3047055531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdc8101db0 0x7efdc8108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.402+0000 7efdce7ff700 1 --2- 172.21.15.93:0/3047055531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdc8100230 0x7efdc8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.402+0000 7efdce7ff700 1 --2- 172.21.15.93:0/3047055531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdc8101420 0x7efdc8101870 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.402+0000 7efdce7ff700 1 -- 172.21.15.93:0/3047055531 >> 172.21.15.93:0/3047055531 conn(0x7efdc80fb810 msgr2=0x7efdc80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:42.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.402+0000 7efdce7ff700 1 -- 172.21.15.93:0/3047055531 shutdown_connections 2023-11-16T16:48:42.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.402+0000 7efdce7ff700 1 -- 172.21.15.93:0/3047055531 wait complete. 2023-11-16T16:48:42.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.403+0000 7efdce7ff700 1 Processor -- start 2023-11-16T16:48:42.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.403+0000 7efdce7ff700 1 -- start start 2023-11-16T16:48:42.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdce7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdc8100230 0x7efdc81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdce7ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdc8101420 0x7efdc8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdce7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdc8101db0 0x7efdc819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdce7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc819fd60 con 0x7efdc8101db0 2023-11-16T16:48:42.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdce7ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc819fea0 con 0x7efdc8101420 2023-11-16T16:48:42.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdce7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc81a0030 con 0x7efdc8100230 2023-11-16T16:48:42.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdccd9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdc8101db0 0x7efdc819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdc7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdc8100230 0x7efdc81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdccd9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdc8101db0 0x7efdc819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39214/0 (socket says 172.21.15.93:39214) 2023-11-16T16:48:42.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdccd9c700 1 -- 172.21.15.93:0/4205150520 learned_addr learned my addr 172.21.15.93:0/4205150520 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:42.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdc77fe700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdc8101420 0x7efdc8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdccd9c700 1 -- 172.21.15.93:0/4205150520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdc8100230 msgr2=0x7efdc81981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdccd9c700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdc8100230 0x7efdc81981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdccd9c700 1 -- 172.21.15.93:0/4205150520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdc8101420 msgr2=0x7efdc8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdccd9c700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdc8101420 0x7efdc8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.404+0000 7efdccd9c700 1 -- 172.21.15.93:0/4205150520 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb8009520 con 0x7efdc8101db0 2023-11-16T16:48:42.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.405+0000 7efdccd9c700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdc8101db0 0x7efdc819f790 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efdc000ef30 tx=0x7efdc000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:42.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.405+0000 7efdc57fa700 1 -- 172.21.15.93:0/4205150520 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0018070 con 0x7efdc8101db0 2023-11-16T16:48:42.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.405+0000 7efdc57fa700 1 -- 172.21.15.93:0/4205150520 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdc000f040 con 0x7efdc8101db0 2023-11-16T16:48:42.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.405+0000 7efdce7ff700 1 -- 172.21.15.93:0/4205150520 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc0095f0 con 0x7efdc8101db0 2023-11-16T16:48:42.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.405+0000 7efdc57fa700 1 -- 172.21.15.93:0/4205150520 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc000cd00 con 0x7efdc8101db0 2023-11-16T16:48:42.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.405+0000 7efdce7ff700 1 -- 172.21.15.93:0/4205150520 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc810bbf0 con 0x7efdc8101db0 2023-11-16T16:48:42.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.408+0000 7efdc57fa700 1 -- 172.21.15.93:0/4205150520 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efdc0003a00 con 0x7efdc8101db0 2023-11-16T16:48:42.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.409+0000 7efde208ab80 1 Processor -- start 2023-11-16T16:48:42.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.409+0000 7efdc57fa700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efdb00a11d0 0x7efdb00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efde208ab80 1 -- start start 2023-11-16T16:48:42.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efdc57fa700 1 -- 172.21.15.93:0/4205150520 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7efdc00cc7a0 con 0x7efdc8101db0 2023-11-16T16:48:42.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efdc77fe700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efdb00a11d0 0x7efdb00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efde208ab80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd122bb0 0x561ccd122fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efde208ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd123ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efde208ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd1240c0 0x561ccd124530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efde208ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ccd028010 con 0x561ccd123500 2023-11-16T16:48:42.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efde208ab80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ccd07a850 con 0x561ccd122bb0 2023-11-16T16:48:42.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efde208ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ccd073ce0 con 0x561ccd1240c0 2023-11-16T16:48:42.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efdc77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd123ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efdc77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd123ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39224/0 (socket says 172.21.15.93:39224) 2023-11-16T16:48:42.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efdc77fe700 1 -- 172.21.15.93:0/3086916415 learned_addr learned my addr 172.21.15.93:0/3086916415 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:42.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efdc7fff700 1 --2- 172.21.15.93:0/3086916415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd122bb0 0x561ccd122fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.410+0000 7efdccd9c700 1 --2- 172.21.15.93:0/3086916415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd1240c0 0x561ccd124530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.411+0000 7efdc77fe700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efdb00a11d0 0x7efdb00a3680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0098e0 tx=0x7efdbc005490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:42.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.411+0000 7efdc7fff700 1 -- 172.21.15.93:0/3086916415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd1240c0 msgr2=0x561ccd124530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.411+0000 7efdc7fff700 1 --2- 172.21.15.93:0/3086916415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd1240c0 0x561ccd124530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.411+0000 7efdc7fff700 1 -- 172.21.15.93:0/3086916415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 msgr2=0x561ccd123ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.411+0000 7efdc7fff700 1 --2- 172.21.15.93:0/3086916415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd123ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.411+0000 7efdc7fff700 1 -- 172.21.15.93:0/3086916415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cccef55e0 con 0x561ccd122bb0 2023-11-16T16:48:42.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.411+0000 7efdc77fe700 1 --2- 172.21.15.93:0/3086916415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd123ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:42.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.411+0000 7efdc7fff700 1 --2- 172.21.15.93:0/3086916415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd122bb0 0x561ccd122fc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009de0 tx=0x7efdb8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3296f521df16899 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:42.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.411+0000 7efdae7fc700 1 -- 172.21.15.93:0/3086916415 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8004250 con 0x561ccd122bb0 2023-11-16T16:48:42.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.411+0000 7efdae7fc700 1 -- 172.21.15.93:0/3086916415 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb8004cf0 con 0x561ccd122bb0 2023-11-16T16:48:42.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.411+0000 7efdae7fc700 1 -- 172.21.15.93:0/3086916415 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb80055c0 con 0x561ccd122bb0 2023-11-16T16:48:42.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.412+0000 7efde208ab80 1 -- 172.21.15.93:0/3086916415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd122bb0 msgr2=0x561ccd122fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.412+0000 7efde208ab80 1 --2- 172.21.15.93:0/3086916415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd122bb0 0x561ccd122fc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009de0 tx=0x7efdb8011720 comp rx=0 tx=0).stop 2023-11-16T16:48:42.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.412+0000 7efde208ab80 1 -- 172.21.15.93:0/3086916415 shutdown_connections 2023-11-16T16:48:42.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.412+0000 7efde208ab80 1 --2- 172.21.15.93:0/3086916415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd1240c0 0x561ccd124530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.412+0000 7efde208ab80 1 --2- 172.21.15.93:0/3086916415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd122bb0 0x561ccd122fc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.412+0000 7efde208ab80 1 --2- 172.21.15.93:0/3086916415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd123ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.412+0000 7efde208ab80 1 -- 172.21.15.93:0/3086916415 >> 172.21.15.93:0/3086916415 conn(0x561ccd118310 msgr2=0x561ccd11d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:42.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.412+0000 7efde208ab80 1 -- 172.21.15.93:0/3086916415 shutdown_connections 2023-11-16T16:48:42.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.412+0000 7efde208ab80 1 -- 172.21.15.93:0/3086916415 wait complete. 2023-11-16T16:48:42.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.412+0000 7efde208ab80 1 -- start start 2023-11-16T16:48:42.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efde208ab80 1 Processor -- start 2023-11-16T16:48:42.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efde208ab80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd122bb0 0x561ccd12c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efde208ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd12ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efde208ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd1240c0 0x561ccd133ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efde208ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ccd015890 con 0x561ccd123500 2023-11-16T16:48:42.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efde208ab80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ccd015530 con 0x561ccd122bb0 2023-11-16T16:48:42.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efde208ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cccef78d0 con 0x561ccd1240c0 2023-11-16T16:48:42.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efde208ab80 10 client.0 _ll_register_callbacks cb 0x561ccd033730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:42.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efdc77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd12ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efdc7fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd122bb0 0x561ccd12c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efdccd9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd1240c0 0x561ccd133ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efdc77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd12ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39226/0 (socket says 172.21.15.93:39226) 2023-11-16T16:48:42.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.413+0000 7efdc77fe700 1 -- 172.21.15.93:0/2171389583 learned_addr learned my addr 172.21.15.93:0/2171389583 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:42.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.414+0000 7efdc77fe700 1 -- 172.21.15.93:0/2171389583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd1240c0 msgr2=0x561ccd133ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.414+0000 7efdc77fe700 1 --2- 172.21.15.93:0/2171389583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd1240c0 0x561ccd133ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.414+0000 7efdc77fe700 1 -- 172.21.15.93:0/2171389583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd122bb0 msgr2=0x561ccd12c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.414+0000 7efdc77fe700 1 --2- 172.21.15.93:0/2171389583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd122bb0 0x561ccd12c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.414+0000 7efdc77fe700 1 -- 172.21.15.93:0/2171389583 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb8009a40 con 0x561ccd123500 2023-11-16T16:48:42.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.414+0000 7efdc7fff700 1 --2- 172.21.15.93:0/2171389583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd122bb0 0x561ccd12c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:42.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.414+0000 7efdc77fe700 1 --2- 172.21.15.93:0/2171389583 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd12ca80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efdbc006010 tx=0x7efdbc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bea049e3218cd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:42.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.414+0000 7efda77fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:48:42.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.414+0000 7efda77fe700 1 -- 172.21.15.93:0/2171389583 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc01d030 con 0x561ccd123500 2023-11-16T16:48:42.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.414+0000 7efda77fe700 1 -- 172.21.15.93:0/2171389583 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7efdbc004850 con 0x561ccd123500 2023-11-16T16:48:42.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.414+0000 7efde208ab80 1 -- 172.21.15.93:0/2171389583 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ccd115ad0 con 0x561ccd123500 2023-11-16T16:48:42.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.414+0000 7efda77fe700 1 -- 172.21.15.93:0/2171389583 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdbc004a50 con 0x561ccd123500 2023-11-16T16:48:42.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.415+0000 7efde208ab80 10 client.15303 fetch_fsmap learned FSMap version 23 2023-11-16T16:48:42.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.415+0000 7efde208ab80 1 -- 172.21.15.93:0/2171389583 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561ccd03a140 con 0x561ccd123500 2023-11-16T16:48:42.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.415+0000 7efda77fe700 1 -- 172.21.15.93:0/2171389583 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561ccd115ad0 con 0x561ccd123500 2023-11-16T16:48:42.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.415+0000 7efda77fe700 1 -- 172.21.15.93:0/2171389583 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efdbc018400 con 0x561ccd123500 2023-11-16T16:48:42.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.415+0000 7efde208ab80 10 client.15303 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:48:42.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.415+0000 7efde208ab80 10 client.15303 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-16T16:48:42.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.415+0000 7efde208ab80 20 client.15303 populate_metadata read hostname 'smithi093' 2023-11-16T16:48:42.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.415+0000 7efde208ab80 1 --2- 172.21.15.93:0/2171389583 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561ccd1376a0 0x561ccd136b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.415+0000 7efde208ab80 4 client.15303 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:42.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.415+0000 7efde208ab80 1 -- 172.21.15.93:0/2171389583 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ccd036050 con 0x561ccd1376a0 2023-11-16T16:48:42.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.416+0000 7efdc7fff700 1 --2- 172.21.15.93:0/2171389583 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561ccd1376a0 0x561ccd136b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.416+0000 7efda77fe700 10 client.15303 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:48:42.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.416+0000 7efdc7fff700 1 --2- 172.21.15.93:0/2171389583 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561ccd1376a0 0x561ccd136b00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f859858f99db56 server_cookie=6f1ea9c04fbc4d5f in_seq=0 out_seq=0 2023-11-16T16:48:42.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efda77fe700 1 -- 172.21.15.93:0/2171389583 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561ccd036050 con 0x561ccd1376a0 2023-11-16T16:48:42.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efda77fe700 10 client.15303 handle_command_reply: tid=0 2023-11-16T16:48:42.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efde208ab80 1 client.15303 shutdown 2023-11-16T16:48:42.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efde208ab80 1 -- 172.21.15.93:0/2171389583 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 msgr2=0x561ccd12ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efde208ab80 1 --2- 172.21.15.93:0/2171389583 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd12ca80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efdbc006010 tx=0x7efdbc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efde208ab80 1 -- 172.21.15.93:0/2171389583 shutdown_connections 2023-11-16T16:48:42.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efde208ab80 1 --2- 172.21.15.93:0/2171389583 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561ccd1376a0 0x561ccd136b00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efde208ab80 1 --2- 172.21.15.93:0/2171389583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd1240c0 0x561ccd133ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efde208ab80 1 --2- 172.21.15.93:0/2171389583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd122bb0 0x561ccd12c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efde208ab80 1 --2- 172.21.15.93:0/2171389583 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd12ca80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efda77fe700 0 client.15303 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:48:42.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efde208ab80 1 -- 172.21.15.93:0/2171389583 >> 172.21.15.93:0/2171389583 conn(0x561ccd118310 msgr2=0x561ccd11cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:42.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efde208ab80 1 -- 172.21.15.93:0/2171389583 shutdown_connections 2023-11-16T16:48:42.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.417+0000 7efde208ab80 1 -- 172.21.15.93:0/2171389583 wait complete. 2023-11-16T16:48:42.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.418+0000 7efde208ab80 20 client.15303 trim_cache size 0 max 16384 2023-11-16T16:48:42.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.435+0000 7efde208ab80 1 Processor -- start 2023-11-16T16:48:42.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.435+0000 7efde208ab80 1 -- start start 2023-11-16T16:48:42.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efde208ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd122bb0 0x561ccd140430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efde208ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd123500 0x561ccd1339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efde208ab80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd1240c0 0x561ccd12a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efde208ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ccd1249b0 con 0x561ccd122bb0 2023-11-16T16:48:42.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efde208ab80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ccd00fe40 con 0x561ccd1240c0 2023-11-16T16:48:42.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efde208ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cccef9a30 con 0x561ccd123500 2023-11-16T16:48:42.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efdc7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd122bb0 0x561ccd140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efdc7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd122bb0 0x561ccd140430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39238/0 (socket says 172.21.15.93:39238) 2023-11-16T16:48:42.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efdc7fff700 1 -- 172.21.15.93:0/2591664614 learned_addr learned my addr 172.21.15.93:0/2591664614 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:42.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efdc77fe700 1 --2- 172.21.15.93:0/2591664614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd123500 0x561ccd1339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efdccd9c700 1 --2- 172.21.15.93:0/2591664614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd1240c0 0x561ccd12a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efdc7fff700 1 -- 172.21.15.93:0/2591664614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd123500 msgr2=0x561ccd1339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efdc7fff700 1 --2- 172.21.15.93:0/2591664614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd123500 0x561ccd1339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efdc7fff700 1 -- 172.21.15.93:0/2591664614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd1240c0 msgr2=0x561ccd12a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efdc7fff700 1 --2- 172.21.15.93:0/2591664614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd1240c0 0x561ccd12a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efdc7fff700 1 -- 172.21.15.93:0/2591664614 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ccd039e90 con 0x561ccd122bb0 2023-11-16T16:48:42.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.436+0000 7efdc7fff700 1 --2- 172.21.15.93:0/2591664614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd122bb0 0x561ccd140430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efdb8005950 tx=0x7efdb8005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db9d632705484b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:42.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.437+0000 7efdaeffd700 1 -- 172.21.15.93:0/2591664614 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8029070 con 0x561ccd122bb0 2023-11-16T16:48:42.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.437+0000 7efdaeffd700 1 -- 172.21.15.93:0/2591664614 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb8004e60 con 0x561ccd122bb0 2023-11-16T16:48:42.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.437+0000 7efdaeffd700 1 -- 172.21.15.93:0/2591664614 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8003800 con 0x561ccd122bb0 2023-11-16T16:48:42.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.437+0000 7efde208ab80 1 -- 172.21.15.93:0/2591664614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd122bb0 msgr2=0x561ccd140430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.437+0000 7efde208ab80 1 --2- 172.21.15.93:0/2591664614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd122bb0 0x561ccd140430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efdb8005950 tx=0x7efdb8005630 comp rx=0 tx=0).stop 2023-11-16T16:48:42.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.437+0000 7efde208ab80 1 -- 172.21.15.93:0/2591664614 shutdown_connections 2023-11-16T16:48:42.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.437+0000 7efde208ab80 1 --2- 172.21.15.93:0/2591664614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd122bb0 0x561ccd140430 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.437+0000 7efde208ab80 1 --2- 172.21.15.93:0/2591664614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd1240c0 0x561ccd12a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.437+0000 7efde208ab80 1 --2- 172.21.15.93:0/2591664614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd123500 0x561ccd1339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.437+0000 7efde208ab80 1 -- 172.21.15.93:0/2591664614 >> 172.21.15.93:0/2591664614 conn(0x561ccd118310 msgr2=0x561ccd11d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.437+0000 7efde208ab80 1 -- 172.21.15.93:0/2591664614 shutdown_connections 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.437+0000 7efde208ab80 1 -- 172.21.15.93:0/2591664614 wait complete. 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.438+0000 7efde208ab80 1 -- start start 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.438+0000 7efde208ab80 1 Processor -- start 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.438+0000 7efde208ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd122bb0 0x561ccd12e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.438+0000 7efde208ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd133920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.438+0000 7efde208ab80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd1240c0 0x561ccd12e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.438+0000 7efde208ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ccd12eff0 con 0x561ccd123500 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.438+0000 7efde208ab80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ccd18b560 con 0x561ccd1240c0 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.438+0000 7efde208ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ccd18b6f0 con 0x561ccd122bb0 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.438+0000 7efde208ab80 10 client.0 _ll_register_callbacks cb 0x561ccd033730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efdc77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd133920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efdc77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd133920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39242/0 (socket says 172.21.15.93:39242) 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efdc77fe700 1 -- 172.21.15.93:0/1370296104 learned_addr learned my addr 172.21.15.93:0/1370296104 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efdc7fff700 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd122bb0 0x561ccd12e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efdccd9c700 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd1240c0 0x561ccd12e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efdc77fe700 1 -- 172.21.15.93:0/1370296104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd122bb0 msgr2=0x561ccd12e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efdc77fe700 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd122bb0 0x561ccd12e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efdc77fe700 1 -- 172.21.15.93:0/1370296104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd1240c0 msgr2=0x561ccd12e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efdc77fe700 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd1240c0 0x561ccd12e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efdc77fe700 1 -- 172.21.15.93:0/1370296104 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb801d040 con 0x561ccd123500 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efdc77fe700 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd133920 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efdbc006010 tx=0x7efdbc018b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339246724255accc server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efda77fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efda77fe700 1 -- 172.21.15.93:0/1370296104 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc01d040 con 0x561ccd123500 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efda77fe700 1 -- 172.21.15.93:0/1370296104 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7efdbc004850 con 0x561ccd123500 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.439+0000 7efde208ab80 1 -- 172.21.15.93:0/1370296104 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdb8004cf0 con 0x561ccd123500 2023-11-16T16:48:42.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.440+0000 7efda77fe700 1 -- 172.21.15.93:0/1370296104 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdbc004a10 con 0x561ccd123500 2023-11-16T16:48:42.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.440+0000 7efde208ab80 10 client.15309 fetch_fsmap learned FSMap version 23 2023-11-16T16:48:42.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.440+0000 7efde208ab80 1 -- 172.21.15.93:0/1370296104 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561ccce8ac10 con 0x561ccd123500 2023-11-16T16:48:42.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.440+0000 7efda77fe700 1 -- 172.21.15.93:0/1370296104 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb8004cf0 con 0x561ccd123500 2023-11-16T16:48:42.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.440+0000 7efda77fe700 1 -- 172.21.15.93:0/1370296104 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efdbc038040 con 0x561ccd123500 2023-11-16T16:48:42.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.440+0000 7efde208ab80 10 client.15309 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:48:42.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.440+0000 7efde208ab80 10 client.15309 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-16T16:48:42.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.440+0000 7efde208ab80 20 client.15309 populate_metadata read hostname 'smithi093' 2023-11-16T16:48:42.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.440+0000 7efde208ab80 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561ccd1900b0 0x561ccd18c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:42.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.440+0000 7efde208ab80 4 client.15309 mds_command: new command op to 24566 tid=0[{"prefix": "status"}] 2023-11-16T16:48:42.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.440+0000 7efde208ab80 1 -- 172.21.15.93:0/1370296104 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561ccd042610 con 0x561ccd1900b0 2023-11-16T16:48:42.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.440+0000 7efdc7fff700 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561ccd1900b0 0x561ccd18c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:42.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efda77fe700 10 client.15309 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:48:42.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efdc7fff700 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561ccd1900b0 0x561ccd18c9c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59e325cbb801b3c5 server_cookie=5e730d3ead0a78b1 in_seq=0 out_seq=0 2023-11-16T16:48:42.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efda77fe700 1 -- 172.21.15.93:0/1370296104 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x561ccd042610 con 0x561ccd1900b0 2023-11-16T16:48:42.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efda77fe700 10 client.15309 handle_command_reply: tid=0 2023-11-16T16:48:42.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efde208ab80 1 client.15309 shutdown 2023-11-16T16:48:42.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efde208ab80 1 -- 172.21.15.93:0/1370296104 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 msgr2=0x561ccd133920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efde208ab80 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd133920 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efdbc006010 tx=0x7efdbc018b00 comp rx=0 tx=0).stop 2023-11-16T16:48:42.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efde208ab80 1 -- 172.21.15.93:0/1370296104 shutdown_connections 2023-11-16T16:48:42.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efde208ab80 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561ccd1900b0 0x561ccd18c9c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efde208ab80 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ccd1240c0 0x561ccd12e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efde208ab80 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ccd122bb0 0x561ccd12e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efde208ab80 1 --2- 172.21.15.93:0/1370296104 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ccd123500 0x561ccd133920 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efda77fe700 0 client.15309 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:48:42.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.441+0000 7efde208ab80 1 -- 172.21.15.93:0/1370296104 >> 172.21.15.93:0/1370296104 conn(0x561ccd118310 msgr2=0x561ccd140430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:42.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efde208ab80 1 -- 172.21.15.93:0/1370296104 shutdown_connections 2023-11-16T16:48:42.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efde208ab80 1 -- 172.21.15.93:0/1370296104 wait complete. 2023-11-16T16:48:42.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efde208ab80 20 client.15309 trim_cache size 0 max 16384 2023-11-16T16:48:42.448 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:48:42.448 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "6a59c1a8-849e-11ee-8dbe-212e2dc638e7", 2023-11-16T16:48:42.448 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 0, 2023-11-16T16:48:42.448 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24566, 2023-11-16T16:48:42.448 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-11-16T16:48:42.448 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-11-16T16:48:42.449 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-11-16T16:48:42.449 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 125.314684103, 2023-11-16T16:48:42.449 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 23, 2023-11-16T16:48:42.449 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 92, 2023-11-16T16:48:42.449 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 90, 2023-11-16T16:48:42.449 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 133.07777797 2023-11-16T16:48:42.449 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:48:42.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efdce7ff700 1 -- 172.21.15.93:0/4205150520 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efdb00a11d0 msgr2=0x7efdb00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efdce7ff700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efdb00a11d0 0x7efdb00a3680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0098e0 tx=0x7efdbc005490 comp rx=0 tx=0).stop 2023-11-16T16:48:42.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efdce7ff700 1 -- 172.21.15.93:0/4205150520 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdc8101db0 msgr2=0x7efdc819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:42.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efdce7ff700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdc8101db0 0x7efdc819f790 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efdc000ef30 tx=0x7efdc000c3a0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efdce7ff700 1 -- 172.21.15.93:0/4205150520 shutdown_connections 2023-11-16T16:48:42.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efdce7ff700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7efdb00a11d0 0x7efdb00a3680 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efdce7ff700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdc8101db0 0x7efdc819f790 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efdce7ff700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdc8100230 0x7efdc81981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efdce7ff700 1 --2- 172.21.15.93:0/4205150520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdc8101420 0x7efdc8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:42.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efdce7ff700 1 -- 172.21.15.93:0/4205150520 >> 172.21.15.93:0/4205150520 conn(0x7efdc80fb810 msgr2=0x7efdc80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:42.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efdce7ff700 1 -- 172.21.15.93:0/4205150520 shutdown_connections 2023-11-16T16:48:42.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:42.442+0000 7efdce7ff700 1 -- 172.21.15.93:0/4205150520 wait complete. 2023-11-16T16:48:42.462 INFO:teuthology.run_tasks:Running task kclient... 2023-11-16T16:48:42.474 INFO:tasks.kclient:Mounting kernel clients... 2023-11-16T16:48:42.474 INFO:tasks.kclient:config is {'client.0': {}} 2023-11-16T16:48:42.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:42.475 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-11-16T16:48:42.518 INFO:teuthology.orchestra.run.smithi093.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-16T16:48:42.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:42.519 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-16T16:48:42.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:42.630 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link delete ceph-brx 2023-11-16T16:48:42.722 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:42 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/973362304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:42.732 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-11-16T16:48:42.732 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-16T16:48:42.733 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-16T16:48:42.733 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph fs ls 2023-11-16T16:48:42.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:42 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/973362304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:43.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:42 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/973362304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:48:43.575 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:48:43.886 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:43 smithi093 ceph-mon[97863]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:43.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:43 smithi159 ceph-mon[131528]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:44.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:43 smithi125 ceph-mon[131076]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:45.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.075+0000 7fbc9890e700 1 -- 172.21.15.93:0/1633628885 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc94072e90 msgr2=0x7fbc94073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:45.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.075+0000 7fbc9890e700 1 --2- 172.21.15.93:0/1633628885 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc94072e90 0x7fbc94073260 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009990 tx=0x7fbc84009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:45.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.076+0000 7fbc9890e700 1 -- 172.21.15.93:0/1633628885 shutdown_connections 2023-11-16T16:48:45.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.076+0000 7fbc9890e700 1 --2- 172.21.15.93:0/1633628885 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc94072e90 0x7fbc94073260 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:45.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.076+0000 7fbc9890e700 1 --2- 172.21.15.93:0/1633628885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc9410cb20 0x7fbc9410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:45.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.076+0000 7fbc9890e700 1 --2- 172.21.15.93:0/1633628885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc940737a0 0x7fbc9410c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:45.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.076+0000 7fbc9890e700 1 -- 172.21.15.93:0/1633628885 >> 172.21.15.93:0/1633628885 conn(0x7fbc940fbdc0 msgr2=0x7fbc940fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:45.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.076+0000 7fbc9890e700 1 -- 172.21.15.93:0/1633628885 shutdown_connections 2023-11-16T16:48:45.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.076+0000 7fbc9890e700 1 -- 172.21.15.93:0/1633628885 wait complete. 2023-11-16T16:48:45.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.077+0000 7fbc9890e700 1 Processor -- start 2023-11-16T16:48:45.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.077+0000 7fbc9890e700 1 -- start start 2023-11-16T16:48:45.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.077+0000 7fbc9890e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc94072e90 0x7fbc941985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:45.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.077+0000 7fbc9890e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc940737a0 0x7fbc94198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:45.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9890e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc9410cb20 0x7fbc9419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:45.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9890e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9419d600 con 0x7fbc940737a0 2023-11-16T16:48:45.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9890e700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9419d770 con 0x7fbc9410cb20 2023-11-16T16:48:45.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9890e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9419d930 con 0x7fbc94072e90 2023-11-16T16:48:45.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc94072e90 0x7fbc941985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:45.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc94072e90 0x7fbc941985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:34550/0 (socket says 172.21.15.93:34550) 2023-11-16T16:48:45.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9259c700 1 -- 172.21.15.93:0/2486212004 learned_addr learned my addr 172.21.15.93:0/2486212004 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:48:45.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc91d9b700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc940737a0 0x7fbc94198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:45.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc92d9d700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc9410cb20 0x7fbc9419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:45.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9259c700 1 -- 172.21.15.93:0/2486212004 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc9410cb20 msgr2=0x7fbc9419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:45.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9259c700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc9410cb20 0x7fbc9419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:45.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9259c700 1 -- 172.21.15.93:0/2486212004 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc940737a0 msgr2=0x7fbc94198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:45.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9259c700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc940737a0 0x7fbc94198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:45.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9259c700 1 -- 172.21.15.93:0/2486212004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc840095f0 con 0x7fbc94072e90 2023-11-16T16:48:45.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc91d9b700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc940737a0 0x7fbc94198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:45.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.078+0000 7fbc9259c700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc94072e90 0x7fbc941985b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc84005910 tx=0x7fbc84004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:45.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.079+0000 7fbc837fe700 1 -- 172.21.15.93:0/2486212004 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8401d070 con 0x7fbc94072e90 2023-11-16T16:48:45.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.079+0000 7fbc9890e700 1 -- 172.21.15.93:0/2486212004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc94112550 con 0x7fbc94072e90 2023-11-16T16:48:45.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.079+0000 7fbc837fe700 1 -- 172.21.15.93:0/2486212004 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc84022470 con 0x7fbc94072e90 2023-11-16T16:48:45.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.079+0000 7fbc837fe700 1 -- 172.21.15.93:0/2486212004 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84017650 con 0x7fbc94072e90 2023-11-16T16:48:45.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.079+0000 7fbc9890e700 1 -- 172.21.15.93:0/2486212004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc941123c0 con 0x7fbc94072e90 2023-11-16T16:48:45.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.080+0000 7fbc9890e700 1 -- 172.21.15.93:0/2486212004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc941126e0 con 0x7fbc94072e90 2023-11-16T16:48:45.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.082+0000 7fbc837fe700 1 -- 172.21.15.93:0/2486212004 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbc840225e0 con 0x7fbc94072e90 2023-11-16T16:48:45.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.083+0000 7fbc837fe700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbc7c0a1150 0x7fbc7c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:45.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.083+0000 7fbc837fe700 1 -- 172.21.15.93:0/2486212004 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fbc840cdff0 con 0x7fbc94072e90 2023-11-16T16:48:45.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.083+0000 7fbc91d9b700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbc7c0a1150 0x7fbc7c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:45.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.084+0000 7fbc91d9b700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbc7c0a1150 0x7fbc7c0a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbc94199bd0 tx=0x7fbc88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:45.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.086+0000 7fbc837fe700 1 -- 172.21.15.93:0/2486212004 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbc840973a0 con 0x7fbc94072e90 2023-11-16T16:48:45.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.332+0000 7fbc9890e700 1 -- 172.21.15.93:0/2486212004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbc94107440 con 0x7fbc94072e90 2023-11-16T16:48:45.334 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.332+0000 7fbc837fe700 1 -- 172.21.15.93:0/2486212004 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+88 (secure 0 0 0) 0x7fbc8409a9c0 con 0x7fbc94072e90 2023-11-16T16:48:45.334 INFO:teuthology.orchestra.run.smithi093.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-16T16:48:45.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.335+0000 7fbc9890e700 1 -- 172.21.15.93:0/2486212004 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbc7c0a1150 msgr2=0x7fbc7c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:45.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.335+0000 7fbc9890e700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbc7c0a1150 0x7fbc7c0a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbc94199bd0 tx=0x7fbc88009260 comp rx=0 tx=0).stop 2023-11-16T16:48:45.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.335+0000 7fbc9890e700 1 -- 172.21.15.93:0/2486212004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc94072e90 msgr2=0x7fbc941985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:45.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.335+0000 7fbc9890e700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc94072e90 0x7fbc941985b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc84005910 tx=0x7fbc84004710 comp rx=0 tx=0).stop 2023-11-16T16:48:45.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.335+0000 7fbc9890e700 1 -- 172.21.15.93:0/2486212004 shutdown_connections 2023-11-16T16:48:45.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.335+0000 7fbc9890e700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbc7c0a1150 0x7fbc7c0a3600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:45.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.335+0000 7fbc9890e700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc9410cb20 0x7fbc9419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:45.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.335+0000 7fbc9890e700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc94072e90 0x7fbc941985b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:45.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.335+0000 7fbc9890e700 1 --2- 172.21.15.93:0/2486212004 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc940737a0 0x7fbc94198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:45.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.335+0000 7fbc9890e700 1 -- 172.21.15.93:0/2486212004 >> 172.21.15.93:0/2486212004 conn(0x7fbc940fbdc0 msgr2=0x7fbc94104580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:45.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.335+0000 7fbc9890e700 1 -- 172.21.15.93:0/2486212004 shutdown_connections 2023-11-16T16:48:45.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:48:45.335+0000 7fbc9890e700 1 -- 172.21.15.93:0/2486212004 wait complete. 2023-11-16T16:48:45.862 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-16T16:48:45.862 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-16T16:48:45.862 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi093.front.sepia.ceph.com 2023-11-16T16:48:45.862 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-16T16:48:45.862 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-16T16:48:45.862 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-16T16:48:45.862 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-16T16:48:45.862 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-16T16:48:45.863 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-16T16:48:45.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:45.863 DEBUG:teuthology.orchestra.run.smithi093:> ip addr 2023-11-16T16:48:45.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:45 smithi093 ceph-mon[97863]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:45.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:45 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2486212004' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-16T16:48:45.907 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-16T16:48:45.908 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-16T16:48:45.908 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2023-11-16T16:48:45.908 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-16T16:48:45.908 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2023-11-16T16:48:45.908 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-16T16:48:45.908 INFO:teuthology.orchestra.run.smithi093.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-16T16:48:45.908 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1c brd ff:ff:ff:ff:ff:ff 2023-11-16T16:48:45.908 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f0 2023-11-16T16:48:45.908 INFO:teuthology.orchestra.run.smithi093.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-16T16:48:45.908 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1d brd ff:ff:ff:ff:ff:ff 2023-11-16T16:48:45.908 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f1 2023-11-16T16:48:45.908 INFO:teuthology.orchestra.run.smithi093.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-16T16:48:45.909 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:34 brd ff:ff:ff:ff:ff:ff 2023-11-16T16:48:45.909 INFO:teuthology.orchestra.run.smithi093.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-16T16:48:45.909 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:35 brd ff:ff:ff:ff:ff:ff 2023-11-16T16:48:45.909 INFO:teuthology.orchestra.run.smithi093.stdout: inet 172.21.15.93/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-16T16:48:45.909 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft 41469sec preferred_lft 41469sec 2023-11-16T16:48:45.909 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::2b1e:922d:25d9:39b2/64 scope link noprefixroute 2023-11-16T16:48:45.909 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-16T16:48:45.910 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-16T16:48:45.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:45.910 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-16T16:48:45.910 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-16T16:48:45.910 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add name ceph-brx type bridge 2023-11-16T16:48:45.910 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr flush dev ceph-brx 2023-11-16T16:48:45.910 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set ceph-brx up 2023-11-16T16:48:45.911 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-16T16:48:45.911 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-16T16:48:45.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:45 smithi159 ceph-mon[131528]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:45.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:45 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2486212004' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-16T16:48:46.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:45 smithi125 ceph-mon[131076]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:48:46.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:45 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2486212004' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-16T16:48:46.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:46.156 DEBUG:teuthology.orchestra.run.smithi093:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-16T16:48:46.223 INFO:teuthology.orchestra.run.smithi093.stdout:1 2023-11-16T16:48:46.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:46.227 DEBUG:teuthology.orchestra.run.smithi093:> route 2023-11-16T16:48:46.273 INFO:teuthology.orchestra.run.smithi093.stdout:Kernel IP routing table 2023-11-16T16:48:46.273 INFO:teuthology.orchestra.run.smithi093.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-16T16:48:46.273 INFO:teuthology.orchestra.run.smithi093.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-16T16:48:46.273 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-16T16:48:46.273 INFO:teuthology.orchestra.run.smithi093.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-16T16:48:46.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:46.274 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-16T16:48:46.274 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-16T16:48:46.274 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-16T16:48:46.275 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-16T16:48:46.275 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-16T16:48:46.275 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-16T16:48:46.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:46.485 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-11-16T16:48:46.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:46.529 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list-id 2023-11-16T16:48:46.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:46.614 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-16T16:48:46.614 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-16T16:48:46.614 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-16T16:48:46.614 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-16T16:48:46.614 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-16T16:48:46.797 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-16T16:48:46.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:46.797 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-16T16:48:46.798 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-16T16:48:46.798 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-16T16:48:46.798 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-16T16:48:46.798 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-16T16:48:46.798 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-16T16:48:46.798 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-16T16:48:46.798 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-16T16:48:47.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:47.042 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-16T16:48:47.042 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-16T16:48:47.042 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set brx.0 up 2023-11-16T16:48:47.043 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set dev brx.0 master ceph-brx 2023-11-16T16:48:47.043 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-16T16:48:47.215 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-16T16:48:47.215 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-16T16:48:47.259 INFO:teuthology.orchestra.run.smithi093.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-16T16:48:47.259 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-16T16:48:47.259 DEBUG:teuthology.orchestra.run.smithi093:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-16T16:48:47.342 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-11-16T16:48:47.342 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-11-16T16:48:47.342 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/a7076a0f-3e60-4d59-9a3c-534477d252be 2023-11-16T16:48:47.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:47.342 DEBUG:teuthology.orchestra.run.smithi093:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/a7076a0f-3e60-4d59-9a3c-534477d252be /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2023-11-16T16:48:47.487 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2023-11-16T16:48:47.487 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2023-11-16T16:48:47.487 INFO:teuthology.orchestra.run.smithi093.stdout:invalid new device string format 2023-11-16T16:48:47.743 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:47 smithi093 ceph-mon[97863]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:47.926 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2023-11-16T16:48:47.927 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2023-11-16T16:48:47.927 INFO:teuthology.orchestra.run.smithi093.stdout:invalid new device string format 2023-11-16T16:48:47.927 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: resolved to: "172.21.15.93:3300,172.21.15.125:3300,172.21.15.159:3300" 2023-11-16T16:48:47.927 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.93:3300,172.21.15.125:3300,172.21.15.159:3300:/volumes/_nogroup/sv_1/a7076a0f-3e60-4d59-9a3c-534477d252be 2023-11-16T16:48:47.927 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=6a59c1a8-849e-11ee-8dbe-212e2dc638e7" will pass to kernel 2023-11-16T16:48:47.932 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-16T16:48:47.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:47.933 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-16T16:48:47.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:47 smithi159 ceph-mon[131528]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:48.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:48.005 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-16T16:48:48.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:47 smithi125 ceph-mon[131076]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:48:48.091 INFO:teuthology.orchestra.run.smithi093.stdout:client25348 2023-11-16T16:48:48.092 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:48:48.092 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-16T16:48:48.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:48.136 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-16T16:48:48.262 INFO:teuthology.orchestra.run.smithi093.stdout:client25348 2023-11-16T16:48:48.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:48.263 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-16T16:48:48.349 INFO:teuthology.orchestra.run.smithi093.stdout:client25348 2023-11-16T16:48:48.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:48:48.349 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7.client25348/status') 2023-11-16T16:48:48.456 INFO:teuthology.orchestra.run.smithi093.stdout:instance: client.25348 (3)172.21.15.93:0/1765435391 2023-11-16T16:48:48.456 INFO:teuthology.orchestra.run.smithi093.stdout:blocklisted: false 2023-11-16T16:48:48.456 INFO:teuthology.orchestra.run.smithi093.stderr:0+1 records in 2023-11-16T16:48:48.456 INFO:teuthology.orchestra.run.smithi093.stderr:0+1 records out 2023-11-16T16:48:48.456 INFO:teuthology.orchestra.run.smithi093.stderr:71 bytes copied, 6.5698e-05 s, 1.1 MB/s 2023-11-16T16:48:48.459 INFO:teuthology.run_tasks:Running task check-counter... 2023-11-16T16:48:48.469 INFO:teuthology.run_tasks:Running task background_exec... 2023-11-16T16:48:48.636 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-11-16T16:48:48.636 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-11-16T16:48:48.642 INFO:teuthology.run_tasks:Running task workunit... 2023-11-16T16:48:48.656 INFO:tasks.workunit:Pulling workunits from ref 5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:48:48.656 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-11-16T16:48:48.656 INFO:tasks.workunit:timeout=3h 2023-11-16T16:48:48.656 INFO:tasks.workunit:cleanup=True 2023-11-16T16:48:48.656 DEBUG:teuthology.orchestra.run.smithi093:> stat -- /home/ubuntu/cephtest/mnt.0 2023-11-16T16:48:48.702 INFO:teuthology.orchestra.run.smithi093.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-11-16T16:48:48.702 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-11-16T16:48:48.702 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-11-16T16:48:48.702 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:48:48.702 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:cephfs_t:s0 2023-11-16T16:48:48.702 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-11-16 16:47:11.770197311 +0000 2023-11-16T16:48:48.702 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-11-16 16:47:11.770197311 +0000 2023-11-16T16:48:48.703 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-11-16 16:48:47.998543258 +0000 2023-11-16T16:48:48.703 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: 2023-11-16 16:47:11.770197311 +0000 2023-11-16T16:48:48.703 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-11-16T16:48:48.703 DEBUG:teuthology.orchestra.run.smithi093:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-11-16T16:48:48.827 DEBUG:teuthology.orchestra.run.smithi093:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:48:48.873 INFO:tasks.workunit.client.0.smithi093.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-11-16T16:48:49.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:49 smithi093 ceph-mon[97863]: pgmap v351: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-16T16:48:49.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:49 smithi159 ceph-mon[131528]: pgmap v351: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-16T16:48:50.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:49 smithi125 ceph-mon[131076]: pgmap v351: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-16T16:48:51.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:51 smithi093 ceph-mon[97863]: pgmap v352: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s 2023-11-16T16:48:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:51 smithi159 ceph-mon[131528]: pgmap v352: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s 2023-11-16T16:48:51.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:51 smithi125 ceph-mon[131076]: pgmap v352: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s 2023-11-16T16:48:53.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:53 smithi093 ceph-mon[97863]: pgmap v353: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-16T16:48:53.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:53 smithi159 ceph-mon[131528]: pgmap v353: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-16T16:48:54.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:53 smithi125 ceph-mon[131076]: pgmap v353: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-16T16:48:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:55 smithi093 ceph-mon[97863]: pgmap v354: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-16T16:48:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:55 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:48:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:55 smithi159 ceph-mon[131528]: pgmap v354: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-16T16:48:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:55 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:48:56.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:55 smithi125 ceph-mon[131076]: pgmap v354: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-16T16:48:56.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:55 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:48:57.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:57 smithi093 ceph-mon[97863]: pgmap v355: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-11-16T16:48:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:57 smithi159 ceph-mon[131528]: pgmap v355: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-11-16T16:48:58.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:57 smithi125 ceph-mon[131076]: pgmap v355: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-11-16T16:48:59.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:59 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:59 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:59 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:59 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:59 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:48:59 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:59 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:59 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:59 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:59 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:59 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:48:59 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:59 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:59 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:59 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:59 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:59 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:48:59.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:48:59 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:49:00.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:00 smithi159 ceph-mon[131528]: pgmap v356: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-11-16T16:49:00.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:49:00.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:49:00.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:49:00.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:00 smithi125 ceph-mon[131076]: pgmap v356: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-11-16T16:49:00.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:49:00.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:49:00.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:49:00.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:00 smithi093 ceph-mon[97863]: pgmap v356: 129 pgs: 129 active+clean; 649 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-11-16T16:49:00.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:49:00.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:49:00.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:49:01.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:01 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:49:01.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:01 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:49:01.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:01 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:49:01.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:01 smithi159 ceph-mon[131528]: Service mds.all has no related daemon already placed. 2023-11-16T16:49:01.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:01 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:49:01.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:01 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:49:01.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:01 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:49:01.578 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:01 smithi125 ceph-mon[131076]: Service mds.all has no related daemon already placed. 2023-11-16T16:49:01.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:01 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:49:01.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:01 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:49:01.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:01 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:49:01.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:01 smithi093 ceph-mon[97863]: Service mds.all has no related daemon already placed. 2023-11-16T16:49:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.891+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3837369553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f88100240 msgr2=0x7f8f88100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.891+0000 7f8f8dfff700 1 --2- 172.21.15.93:0/3837369553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f88100240 0x7f8f88100650 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009990 tx=0x7f8f78009ca0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.891+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3837369553 shutdown_connections 2023-11-16T16:49:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.891+0000 7f8f8dfff700 1 --2- 172.21.15.93:0/3837369553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f88100240 0x7f8f88100650 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.891+0000 7f8f8dfff700 1 --2- 172.21.15.93:0/3837369553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f88101dc0 0x7f8f88108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.891+0000 7f8f8dfff700 1 --2- 172.21.15.93:0/3837369553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f88101430 0x7f8f88101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.891+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3837369553 >> 172.21.15.93:0/3837369553 conn(0x7f8f880fb840 msgr2=0x7f8f880fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.891+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3837369553 shutdown_connections 2023-11-16T16:49:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.891+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3837369553 wait complete. 2023-11-16T16:49:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.892+0000 7f8f8dfff700 1 Processor -- start 2023-11-16T16:49:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.892+0000 7f8f8dfff700 1 -- start start 2023-11-16T16:49:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f8dfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f88100240 0x7f8f88198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f8dfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f88101430 0x7f8f88198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f8dfff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f88101dc0 0x7f8f8819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f8dfff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8819fdc0 con 0x7f8f88101430 2023-11-16T16:49:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f8dfff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8819ff00 con 0x7f8f88101dc0 2023-11-16T16:49:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f8dfff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f881a0090 con 0x7f8f88100240 2023-11-16T16:49:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f88101430 0x7f8f88198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f88101430 0x7f8f88198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:60076/0 (socket says 172.21.15.93:60076) 2023-11-16T16:49:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f86ffd700 1 -- 172.21.15.93:0/3025185498 learned_addr learned my addr 172.21.15.93:0/3025185498 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f877fe700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f88100240 0x7f8f88198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f87fff700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f88101dc0 0x7f8f8819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f86ffd700 1 -- 172.21.15.93:0/3025185498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f88100240 msgr2=0x7f8f88198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f88100240 0x7f8f88198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f86ffd700 1 -- 172.21.15.93:0/3025185498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f88101dc0 msgr2=0x7f8f8819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f88101dc0 0x7f8f8819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.893+0000 7f8f86ffd700 1 -- 172.21.15.93:0/3025185498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f780095f0 con 0x7f8f88101430 2023-11-16T16:49:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.894+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f88101430 0x7f8f88198780 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00e9a0 tx=0x7f8f7c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.894+0000 7f8f84ff9700 1 -- 172.21.15.93:0/3025185498 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c009790 con 0x7f8f88101430 2023-11-16T16:49:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.894+0000 7f8f84ff9700 1 -- 172.21.15.93:0/3025185498 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f7c010430 con 0x7f8f88101430 2023-11-16T16:49:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.894+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3025185498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8810bbf0 con 0x7f8f88101430 2023-11-16T16:49:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.894+0000 7f8f84ff9700 1 -- 172.21.15.93:0/3025185498 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c018680 con 0x7f8f88101430 2023-11-16T16:49:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.894+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3025185498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8810ba60 con 0x7f8f88101430 2023-11-16T16:49:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.897+0000 7f8f84ff9700 1 -- 172.21.15.93:0/3025185498 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8f7c0105a0 con 0x7f8f88101430 2023-11-16T16:49:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.898+0000 7f8f84ff9700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f700a11d0 0x7f8f700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.898+0000 7f8fa18f6b80 1 Processor -- start 2023-11-16T16:49:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.899+0000 7f8f84ff9700 1 -- 172.21.15.93:0/3025185498 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f8f7c014070 con 0x7f8f88101430 2023-11-16T16:49:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.899+0000 7f8fa18f6b80 1 -- start start 2023-11-16T16:49:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.899+0000 7f8f877fe700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f700a11d0 0x7f8f700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.899+0000 7f8fa18f6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f1ab6b0 0x55d33f34f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.899+0000 7f8fa18f6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f3505d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.899+0000 7f8fa18f6b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 0x55d33f350f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.899+0000 7f8fa18f6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f255000 con 0x55d33f34fdb0 2023-11-16T16:49:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.899+0000 7f8fa18f6b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f2a7840 con 0x55d33f350b10 2023-11-16T16:49:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.899+0000 7f8fa18f6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f2a0cd0 con 0x55d33f1ab6b0 2023-11-16T16:49:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.899+0000 7f8f86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f3505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.899+0000 7f8f86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f3505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:60084/0 (socket says 172.21.15.93:60084) 2023-11-16T16:49:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.899+0000 7f8f86ffd700 1 -- 172.21.15.93:0/2809209979 learned_addr learned my addr 172.21.15.93:0/2809209979 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.900+0000 7f8f877fe700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f700a11d0 0x7f8f700a3680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800b2e0 tx=0x7f8f78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.900+0000 7f8f877fe700 1 --2- 172.21.15.93:0/2809209979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f1ab6b0 0x55d33f34f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.900+0000 7f8f87fff700 1 --2- 172.21.15.93:0/2809209979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 0x55d33f350f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.900+0000 7f8f86ffd700 1 -- 172.21.15.93:0/2809209979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f1ab6b0 msgr2=0x55d33f34f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.900+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/2809209979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f1ab6b0 0x55d33f34f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.900+0000 7f8f86ffd700 1 -- 172.21.15.93:0/2809209979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 msgr2=0x55d33f350f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.900+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/2809209979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 0x55d33f350f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.900+0000 7f8f86ffd700 1 -- 172.21.15.93:0/2809209979 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d33f1225d0 con 0x55d33f34fdb0 2023-11-16T16:49:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.900+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/2809209979 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f3505d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c099e20 tx=0x7f8f7c099870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5075cc33898e02d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.900+0000 7f8f6dffb700 1 -- 172.21.15.93:0/2809209979 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0b1070 con 0x55d33f34fdb0 2023-11-16T16:49:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.900+0000 7f8f6dffb700 1 -- 172.21.15.93:0/2809209979 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f7c09dec0 con 0x55d33f34fdb0 2023-11-16T16:49:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.900+0000 7f8f6dffb700 1 -- 172.21.15.93:0/2809209979 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0995e0 con 0x55d33f34fdb0 2023-11-16T16:49:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.901+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2809209979 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 msgr2=0x55d33f3505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.901+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/2809209979 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f3505d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c099e20 tx=0x7f8f7c099870 comp rx=0 tx=0).stop 2023-11-16T16:49:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.901+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2809209979 shutdown_connections 2023-11-16T16:49:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.901+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/2809209979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 0x55d33f350f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.901+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/2809209979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f1ab6b0 0x55d33f34f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.901+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/2809209979 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f3505d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.901+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2809209979 >> 172.21.15.93:0/2809209979 conn(0x55d33f344f50 msgr2=0x55d33f34a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.901+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2809209979 shutdown_connections 2023-11-16T16:49:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.901+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2809209979 wait complete. 2023-11-16T16:49:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.902+0000 7f8fa18f6b80 1 -- start start 2023-11-16T16:49:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.902+0000 7f8fa18f6b80 1 Processor -- start 2023-11-16T16:49:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.902+0000 7f8fa18f6b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f1ab6b0 0x55d33f358c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.902+0000 7f8fa18f6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f3591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.902+0000 7f8fa18f6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f350b10 0x55d33f3601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.902+0000 7f8fa18f6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f242880 con 0x55d33f34fdb0 2023-11-16T16:49:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8fa18f6b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f242520 con 0x55d33f1ab6b0 2023-11-16T16:49:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8fa18f6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f1248c0 con 0x55d33f350b10 2023-11-16T16:49:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8fa18f6b80 10 client.0 _ll_register_callbacks cb 0x55d33f260720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8f86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f3591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8f877fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f1ab6b0 0x55d33f358c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8f86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f3591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:60100/0 (socket says 172.21.15.93:60100) 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8f86ffd700 1 -- 172.21.15.93:0/1684887504 learned_addr learned my addr 172.21.15.93:0/1684887504 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8f87fff700 1 --2- 172.21.15.93:0/1684887504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f350b10 0x55d33f3601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8f86ffd700 1 -- 172.21.15.93:0/1684887504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f350b10 msgr2=0x55d33f3601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/1684887504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f350b10 0x55d33f3601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8f86ffd700 1 -- 172.21.15.93:0/1684887504 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f1ab6b0 msgr2=0x55d33f358c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/1684887504 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f1ab6b0 0x55d33f358c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8f86ffd700 1 -- 172.21.15.93:0/1684887504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8810ba60 con 0x55d33f34fdb0 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/1684887504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f3591a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c010cc0 tx=0x7f8f7c0a2080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dd4f767ba388a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.903+0000 7f8f66ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.904+0000 7f8f66ffd700 1 -- 172.21.15.93:0/1684887504 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0b1030 con 0x55d33f34fdb0 2023-11-16T16:49:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.904+0000 7f8f66ffd700 1 -- 172.21.15.93:0/1684887504 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f8f7c0b2070 con 0x55d33f34fdb0 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.904+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1684887504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d33f342710 con 0x55d33f34fdb0 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.904+0000 7f8f66ffd700 1 -- 172.21.15.93:0/1684887504 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f7c0a22e0 con 0x55d33f34fdb0 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.904+0000 7f8fa18f6b80 10 client.15327 fetch_fsmap learned FSMap version 23 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.904+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1684887504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d33f267130 con 0x55d33f34fdb0 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.904+0000 7f8f66ffd700 1 -- 172.21.15.93:0/1684887504 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f7c07c070 con 0x55d33f34fdb0 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.904+0000 7f8f66ffd700 1 -- 172.21.15.93:0/1684887504 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8f7c0ca240 con 0x55d33f34fdb0 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.904+0000 7f8fa18f6b80 10 client.15327 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.904+0000 7f8fa18f6b80 10 client.15327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.904+0000 7f8fa18f6b80 20 client.15327 populate_metadata read hostname 'smithi093' 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.905+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/1684887504 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d33f364720 0x55d33f366b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.905+0000 7f8fa18f6b80 4 client.15327 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.905+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1684887504 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d33f263040 con 0x55d33f364720 2023-11-16T16:49:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.905+0000 7f8f877fe700 1 --2- 172.21.15.93:0/1684887504 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d33f364720 0x55d33f366b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.905+0000 7f8f877fe700 1 --2- 172.21.15.93:0/1684887504 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d33f364720 0x55d33f366b30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e674e3906e62517 server_cookie=3e33d0c7fed174ea in_seq=0 out_seq=0 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.905+0000 7f8f66ffd700 10 client.15327 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.906+0000 7f8f66ffd700 1 -- 172.21.15.93:0/1684887504 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d33f263040 con 0x55d33f364720 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.906+0000 7f8f66ffd700 10 client.15327 handle_command_reply: tid=0 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.906+0000 7f8fa18f6b80 1 client.15327 shutdown 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.906+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1684887504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 msgr2=0x55d33f3591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.906+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/1684887504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f3591a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c010cc0 tx=0x7f8f7c0a2080 comp rx=0 tx=0).stop 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.906+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1684887504 shutdown_connections 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.906+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/1684887504 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d33f364720 0x55d33f366b30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.906+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/1684887504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f350b10 0x55d33f3601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.906+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/1684887504 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f1ab6b0 0x55d33f358c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.906+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/1684887504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f3591a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.906+0000 7f8f66ffd700 0 client.15327 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.906+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1684887504 >> 172.21.15.93:0/1684887504 conn(0x55d33f344f50 msgr2=0x55d33f3497e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.907+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1684887504 shutdown_connections 2023-11-16T16:49:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.907+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1684887504 wait complete. 2023-11-16T16:49:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.907+0000 7f8fa18f6b80 20 client.15327 trim_cache size 0 max 16384 2023-11-16T16:49:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8fa18f6b80 1 Processor -- start 2023-11-16T16:49:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8fa18f6b80 1 -- start start 2023-11-16T16:49:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8fa18f6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f1ab6b0 0x55d33f34a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8fa18f6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f360170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8fa18f6b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 0x55d33f36b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8fa18f6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f350180 con 0x55d33f34fdb0 2023-11-16T16:49:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8fa18f6b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f34bbb0 con 0x55d33f350b10 2023-11-16T16:49:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8fa18f6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f34a610 con 0x55d33f1ab6b0 2023-11-16T16:49:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8f86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f360170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8f86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f360170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:60102/0 (socket says 172.21.15.93:60102) 2023-11-16T16:49:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8f86ffd700 1 -- 172.21.15.93:0/1416827671 learned_addr learned my addr 172.21.15.93:0/1416827671 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8f87fff700 1 --2- 172.21.15.93:0/1416827671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 0x55d33f36b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.923+0000 7f8f877fe700 1 --2- 172.21.15.93:0/1416827671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f1ab6b0 0x55d33f34a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.924+0000 7f8f86ffd700 1 -- 172.21.15.93:0/1416827671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f1ab6b0 msgr2=0x55d33f34a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.924+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/1416827671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f1ab6b0 0x55d33f34a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.924+0000 7f8f86ffd700 1 -- 172.21.15.93:0/1416827671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 msgr2=0x55d33f36b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.924+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/1416827671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 0x55d33f36b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.924+0000 7f8f86ffd700 1 -- 172.21.15.93:0/1416827671 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d33f266e80 con 0x55d33f34fdb0 2023-11-16T16:49:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.924+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/1416827671 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f360170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c096900 tx=0x7f8f7c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7e4c3d678644a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.924+0000 7f8f6e7fc700 1 -- 172.21.15.93:0/1416827671 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0b1070 con 0x55d33f34fdb0 2023-11-16T16:49:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.924+0000 7f8f6e7fc700 1 -- 172.21.15.93:0/1416827671 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f7c07c070 con 0x55d33f34fdb0 2023-11-16T16:49:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.924+0000 7f8f6e7fc700 1 -- 172.21.15.93:0/1416827671 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c099150 con 0x55d33f34fdb0 2023-11-16T16:49:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.925+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1416827671 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 msgr2=0x55d33f360170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.925+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/1416827671 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f360170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c096900 tx=0x7f8f7c01f000 comp rx=0 tx=0).stop 2023-11-16T16:49:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.925+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1416827671 shutdown_connections 2023-11-16T16:49:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.925+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/1416827671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 0x55d33f36b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.925+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/1416827671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f1ab6b0 0x55d33f34a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.925+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/1416827671 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f34fdb0 0x55d33f360170 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.925+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1416827671 >> 172.21.15.93:0/1416827671 conn(0x55d33f344f50 msgr2=0x55d33f3579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.925+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1416827671 shutdown_connections 2023-11-16T16:49:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.925+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/1416827671 wait complete. 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.925+0000 7f8fa18f6b80 1 -- start start 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.925+0000 7f8fa18f6b80 1 Processor -- start 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8fa18f6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f1ab6b0 0x55d33f35c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8fa18f6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f34fdb0 0x55d33f35c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8fa18f6b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 0x55d33f35f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8fa18f6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f35f830 con 0x55d33f1ab6b0 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8fa18f6b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f35f9a0 con 0x55d33f350b10 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8fa18f6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f35fb60 con 0x55d33f34fdb0 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8fa18f6b80 10 client.0 _ll_register_callbacks cb 0x7f8f80009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8f877fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f1ab6b0 0x55d33f35c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8f877fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f1ab6b0 0x55d33f35c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:60104/0 (socket says 172.21.15.93:60104) 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8f877fe700 1 -- 172.21.15.93:0/2758958819 learned_addr learned my addr 172.21.15.93:0/2758958819 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f34fdb0 0x55d33f35c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8f87fff700 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 0x55d33f35f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8f877fe700 1 -- 172.21.15.93:0/2758958819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f34fdb0 msgr2=0x55d33f35c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8f877fe700 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f34fdb0 0x55d33f35c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8f877fe700 1 -- 172.21.15.93:0/2758958819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 msgr2=0x55d33f35f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8f877fe700 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 0x55d33f35f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.926+0000 7f8f877fe700 1 -- 172.21.15.93:0/2758958819 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d33f267130 con 0x55d33f1ab6b0 2023-11-16T16:49:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.927+0000 7f8f877fe700 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f1ab6b0 0x55d33f35c330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f88101290 tx=0x7f8f78009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4672441dd5e417e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.927+0000 7f8f66ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:49:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.927+0000 7f8f66ffd700 1 -- 172.21.15.93:0/2758958819 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78035ba0 con 0x55d33f1ab6b0 2023-11-16T16:49:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.927+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2758958819 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f7c09dec0 con 0x55d33f1ab6b0 2023-11-16T16:49:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.927+0000 7f8f66ffd700 1 -- 172.21.15.93:0/2758958819 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f8f78035d00 con 0x55d33f1ab6b0 2023-11-16T16:49:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.927+0000 7f8f66ffd700 1 -- 172.21.15.93:0/2758958819 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f78004990 con 0x55d33f1ab6b0 2023-11-16T16:49:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.927+0000 7f8fa18f6b80 10 client.15333 fetch_fsmap learned FSMap version 23 2023-11-16T16:49:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.927+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2758958819 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d33f0b7c00 con 0x55d33f1ab6b0 2023-11-16T16:49:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.927+0000 7f8f66ffd700 1 -- 172.21.15.93:0/2758958819 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f7c09dec0 con 0x55d33f1ab6b0 2023-11-16T16:49:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.928+0000 7f8f66ffd700 1 -- 172.21.15.93:0/2758958819 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8f7800f460 con 0x55d33f1ab6b0 2023-11-16T16:49:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.928+0000 7f8fa18f6b80 10 client.15333 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:49:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.928+0000 7f8fa18f6b80 10 client.15333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.928+0000 7f8fa18f6b80 20 client.15333 populate_metadata read hostname 'smithi093' 2023-11-16T16:49:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.928+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d33f3b9340 0x55d33f3bb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.928+0000 7f8fa18f6b80 4 client.15333 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:49:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.928+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2758958819 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d33f26f600 con 0x55d33f3b9340 2023-11-16T16:49:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.928+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d33f3b9340 0x55d33f3bb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.928+0000 7f8f66ffd700 10 client.15333 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.928+0000 7f8f86ffd700 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d33f3b9340 0x55d33f3bb750 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da6d15bdf5abf376 server_cookie=d8e22504ea53b167 in_seq=0 out_seq=0 2023-11-16T16:49:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8f66ffd700 1 -- 172.21.15.93:0/2758958819 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d33f26f600 con 0x55d33f3b9340 2023-11-16T16:49:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8f66ffd700 10 client.15333 handle_command_reply: tid=0 2023-11-16T16:49:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8fa18f6b80 1 client.15333 shutdown 2023-11-16T16:49:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2758958819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f1ab6b0 msgr2=0x55d33f35c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f1ab6b0 0x55d33f35c330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f88101290 tx=0x7f8f78009ca0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2758958819 shutdown_connections 2023-11-16T16:49:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d33f3b9340 0x55d33f3bb750 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33f1ab6b0 0x55d33f35c330 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d33f350b10 0x55d33f35f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8fa18f6b80 1 --2- 172.21.15.93:0/2758958819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33f34fdb0 0x55d33f35c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2758958819 >> 172.21.15.93:0/2758958819 conn(0x55d33f344f50 msgr2=0x55d33f36b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8f66ffd700 0 client.15333 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2758958819 shutdown_connections 2023-11-16T16:49:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.941+0000 7f8fa18f6b80 1 -- 172.21.15.93:0/2758958819 wait complete. 2023-11-16T16:49:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.942+0000 7f8fa18f6b80 20 client.15333 trim_cache size 0 max 16384 2023-11-16T16:49:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:49:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:49:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:49:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:49:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.942+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3025185498 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f700a11d0 msgr2=0x7f8f700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.942+0000 7f8f8dfff700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f700a11d0 0x7f8f700a3680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800b2e0 tx=0x7f8f78005fb0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.942+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3025185498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f88101430 msgr2=0x7f8f88198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.942+0000 7f8f8dfff700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f88101430 0x7f8f88198780 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00e9a0 tx=0x7f8f7c00ecb0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.942+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3025185498 shutdown_connections 2023-11-16T16:49:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.942+0000 7f8f8dfff700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8f700a11d0 0x7f8f700a3680 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.942+0000 7f8f8dfff700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f88101dc0 0x7f8f8819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.942+0000 7f8f8dfff700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f88100240 0x7f8f88198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.942+0000 7f8f8dfff700 1 --2- 172.21.15.93:0/3025185498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f88101430 0x7f8f88198780 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.942+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3025185498 >> 172.21.15.93:0/3025185498 conn(0x7f8f880fb840 msgr2=0x7f8f88106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.943+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3025185498 shutdown_connections 2023-11-16T16:49:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:01.943+0000 7f8f8dfff700 1 -- 172.21.15.93:0/3025185498 wait complete. 2023-11-16T16:49:02.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:02 smithi159 ceph-mon[131528]: pgmap v357: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-11-16T16:49:02.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:02 smithi125 ceph-mon[131076]: pgmap v357: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-11-16T16:49:02.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:02 smithi093 ceph-mon[97863]: pgmap v357: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-11-16T16:49:04.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:04 smithi159 ceph-mon[131528]: pgmap v358: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2023-11-16T16:49:04.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:04 smithi125 ceph-mon[131076]: pgmap v358: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2023-11-16T16:49:04.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:04 smithi093 ceph-mon[97863]: pgmap v358: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2023-11-16T16:49:06.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:06 smithi159 ceph-mon[131528]: pgmap v359: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2023-11-16T16:49:06.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:06 smithi125 ceph-mon[131076]: pgmap v359: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2023-11-16T16:49:06.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:06 smithi093 ceph-mon[97863]: pgmap v359: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 7 op/s 2023-11-16T16:49:08.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:08 smithi159 ceph-mon[131528]: pgmap v360: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-11-16T16:49:08.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:08 smithi125 ceph-mon[131076]: pgmap v360: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-11-16T16:49:08.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:08 smithi093 ceph-mon[97863]: pgmap v360: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-11-16T16:49:10.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:10 smithi159 ceph-mon[131528]: pgmap v361: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-11-16T16:49:10.577 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:10 smithi125 ceph-mon[131076]: pgmap v361: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-11-16T16:49:10.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:10 smithi093 ceph-mon[97863]: pgmap v361: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-11-16T16:49:11.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:11 smithi093 ceph-mon[97863]: pgmap v362: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-11-16T16:49:11.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:11 smithi159 ceph-mon[131528]: pgmap v362: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-11-16T16:49:11.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:11 smithi125 ceph-mon[131076]: pgmap v362: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-11-16T16:49:13.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:13 smithi093 ceph-mon[97863]: pgmap v363: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-11-16T16:49:13.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:13 smithi159 ceph-mon[131528]: pgmap v363: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-11-16T16:49:14.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:13 smithi125 ceph-mon[131076]: pgmap v363: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-11-16T16:49:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.097+0000 7f1847fff700 1 -- 172.21.15.93:0/3797844789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1848101250 msgr2=0x7f18481016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.097+0000 7f1847fff700 1 --2- 172.21.15.93:0/3797844789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1848101250 0x7f18481016a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f183c009990 tx=0x7f183c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.097+0000 7f1847fff700 1 -- 172.21.15.93:0/3797844789 shutdown_connections 2023-11-16T16:49:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.097+0000 7f1847fff700 1 --2- 172.21.15.93:0/3797844789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1848101be0 0x7f18481084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.097+0000 7f1847fff700 1 --2- 172.21.15.93:0/3797844789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1848100060 0x7f1848100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.097+0000 7f1847fff700 1 --2- 172.21.15.93:0/3797844789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1848101250 0x7f18481016a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.097+0000 7f1847fff700 1 -- 172.21.15.93:0/3797844789 >> 172.21.15.93:0/3797844789 conn(0x7f18480fb640 msgr2=0x7f18480fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.097+0000 7f1847fff700 1 -- 172.21.15.93:0/3797844789 shutdown_connections 2023-11-16T16:49:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.097+0000 7f1847fff700 1 -- 172.21.15.93:0/3797844789 wait complete. 2023-11-16T16:49:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.099+0000 7f1847fff700 1 Processor -- start 2023-11-16T16:49:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.099+0000 7f1847fff700 1 -- start start 2023-11-16T16:49:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.099+0000 7f1847fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1848100060 0x7f18481980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.099+0000 7f1847fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1848101250 0x7f18481985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.099+0000 7f1847fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1848101be0 0x7f184819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.099+0000 7f1847fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184819fc20 con 0x7f1848100060 2023-11-16T16:49:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.099+0000 7f1847fff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184819fd60 con 0x7f1848101250 2023-11-16T16:49:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.099+0000 7f1847fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184819fef0 con 0x7f1848101be0 2023-11-16T16:49:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.099+0000 7f1846ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1848100060 0x7f18481980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.100+0000 7f1846ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1848100060 0x7f18481980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47860/0 (socket says 172.21.15.93:47860) 2023-11-16T16:49:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.100+0000 7f1846ffd700 1 -- 172.21.15.93:0/4043754152 learned_addr learned my addr 172.21.15.93:0/4043754152 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.100+0000 7f18467fc700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1848101250 0x7f18481985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.100+0000 7f18477fe700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1848101be0 0x7f184819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.100+0000 7f18467fc700 1 -- 172.21.15.93:0/4043754152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1848101be0 msgr2=0x7f184819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.100+0000 7f18467fc700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1848101be0 0x7f184819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.100+0000 7f18467fc700 1 -- 172.21.15.93:0/4043754152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1848100060 msgr2=0x7f18481980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.100+0000 7f18467fc700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1848100060 0x7f18481980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.100+0000 7f18467fc700 1 -- 172.21.15.93:0/4043754152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1838009520 con 0x7f1848101250 2023-11-16T16:49:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.100+0000 7f18467fc700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1848101250 0x7f18481985e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f183c0098e0 tx=0x7f183c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.101+0000 7f184c9fd700 1 -- 172.21.15.93:0/4043754152 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c01d070 con 0x7f1848101250 2023-11-16T16:49:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.101+0000 7f1847fff700 1 -- 172.21.15.93:0/4043754152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183c0095f0 con 0x7f1848101250 2023-11-16T16:49:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.101+0000 7f184c9fd700 1 -- 172.21.15.93:0/4043754152 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f183c021410 con 0x7f1848101250 2023-11-16T16:49:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.101+0000 7f184c9fd700 1 -- 172.21.15.93:0/4043754152 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c017650 con 0x7f1848101250 2023-11-16T16:49:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.101+0000 7f1847fff700 1 -- 172.21.15.93:0/4043754152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184810ba10 con 0x7f1848101250 2023-11-16T16:49:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.104+0000 7f184c9fd700 1 -- 172.21.15.93:0/4043754152 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f183c021580 con 0x7f1848101250 2023-11-16T16:49:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.105+0000 7f1861362b80 1 Processor -- start 2023-11-16T16:49:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f184c9fd700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f18300a11d0 0x7f18300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f1861362b80 1 -- start start 2023-11-16T16:49:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f184c9fd700 1 -- 172.21.15.93:0/4043754152 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f183c0cdfa0 con 0x7f1848101250 2023-11-16T16:49:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f1861362b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 0x555a416f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f1861362b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a416f1fb0 0x555a416f2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f1861362b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f2d60 0x555a416f3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f18477fe700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f18300a11d0 0x7f18300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f1861362b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a415f7000 con 0x555a4154d6b0 2023-11-16T16:49:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f1861362b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a41649840 con 0x555a416f1fb0 2023-11-16T16:49:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f1861362b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a41642cd0 con 0x555a416f2d60 2023-11-16T16:49:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f1846ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 0x555a416f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f1846ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 0x555a416f1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47872/0 (socket says 172.21.15.93:47872) 2023-11-16T16:49:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f1846ffd700 1 -- 172.21.15.93:0/835866603 learned_addr learned my addr 172.21.15.93:0/835866603 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.106+0000 7f18477fe700 1 --2- 172.21.15.93:0/835866603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f2d60 0x555a416f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.107+0000 7f1846ffd700 1 -- 172.21.15.93:0/835866603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f2d60 msgr2=0x555a416f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.107+0000 7f1846ffd700 1 --2- 172.21.15.93:0/835866603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f2d60 0x555a416f3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.107+0000 7f1846ffd700 1 -- 172.21.15.93:0/835866603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a416f1fb0 msgr2=0x555a416f2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.107+0000 7f18467fc700 1 --2- 172.21.15.93:0/835866603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a416f1fb0 0x555a416f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.107+0000 7f1846ffd700 1 --2- 172.21.15.93:0/835866603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a416f1fb0 0x555a416f2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.107+0000 7f1846ffd700 1 -- 172.21.15.93:0/835866603 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a414c45d0 con 0x555a4154d6b0 2023-11-16T16:49:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.107+0000 7f18467fc700 1 --2- 172.21.15.93:0/835866603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a416f1fb0 0x555a416f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:49:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.107+0000 7f18477fe700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f18300a11d0 0x7f18300a3680 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1840009ec0 tx=0x7f184000fc40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.107+0000 7f1846ffd700 1 --2- 172.21.15.93:0/835866603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 0x555a416f1a70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1838009f20 tx=0x7f1838011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a17eaa5c8354b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.107+0000 7f182dffb700 1 -- 172.21.15.93:0/835866603 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838004a30 con 0x555a4154d6b0 2023-11-16T16:49:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.107+0000 7f182dffb700 1 -- 172.21.15.93:0/835866603 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18380054e0 con 0x555a4154d6b0 2023-11-16T16:49:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.107+0000 7f182dffb700 1 -- 172.21.15.93:0/835866603 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838019930 con 0x555a4154d6b0 2023-11-16T16:49:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.108+0000 7f1861362b80 1 -- 172.21.15.93:0/835866603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 msgr2=0x555a416f1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.108+0000 7f1861362b80 1 --2- 172.21.15.93:0/835866603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 0x555a416f1a70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1838009f20 tx=0x7f1838011720 comp rx=0 tx=0).stop 2023-11-16T16:49:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.108+0000 7f1861362b80 1 -- 172.21.15.93:0/835866603 shutdown_connections 2023-11-16T16:49:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.108+0000 7f1861362b80 1 --2- 172.21.15.93:0/835866603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 0x555a416f1a70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.108+0000 7f1861362b80 1 --2- 172.21.15.93:0/835866603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f2d60 0x555a416f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.108+0000 7f1861362b80 1 --2- 172.21.15.93:0/835866603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a416f1fb0 0x555a416f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.108+0000 7f1861362b80 1 -- 172.21.15.93:0/835866603 >> 172.21.15.93:0/835866603 conn(0x555a416e71a0 msgr2=0x555a416ec2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.108+0000 7f1861362b80 1 -- 172.21.15.93:0/835866603 shutdown_connections 2023-11-16T16:49:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.108+0000 7f1861362b80 1 -- 172.21.15.93:0/835866603 wait complete. 2023-11-16T16:49:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.109+0000 7f1861362b80 1 -- start start 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.109+0000 7f1861362b80 1 Processor -- start 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1861362b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 0x555a416fae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1861362b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a416f1fb0 0x555a416fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1861362b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f2d60 0x555a41702400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1861362b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a415e4880 con 0x555a4154d6b0 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1861362b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a415e4520 con 0x555a416f1fb0 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1861362b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a414c68c0 con 0x555a416f2d60 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1861362b80 10 client.0 _ll_register_callbacks cb 0x555a41602720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1846ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 0x555a416fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1846ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 0x555a416fae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47880/0 (socket says 172.21.15.93:47880) 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f18477fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f2d60 0x555a41702400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1846ffd700 1 -- 172.21.15.93:0/678239137 learned_addr learned my addr 172.21.15.93:0/678239137 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f18467fc700 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a416f1fb0 0x555a416fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1846ffd700 1 -- 172.21.15.93:0/678239137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f2d60 msgr2=0x555a41702400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1846ffd700 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f2d60 0x555a41702400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.110+0000 7f1846ffd700 1 -- 172.21.15.93:0/678239137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a416f1fb0 msgr2=0x555a416fb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.111+0000 7f1846ffd700 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a416f1fb0 0x555a416fb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.111+0000 7f1846ffd700 1 -- 172.21.15.93:0/678239137 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1838009bd0 con 0x555a4154d6b0 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.111+0000 7f18477fe700 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f2d60 0x555a41702400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.111+0000 7f18467fc700 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a416f1fb0 0x555a416fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.111+0000 7f1846ffd700 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 0x555a416fae80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1838009ef0 tx=0x7f1838009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd99b46515dadc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.111+0000 7f1826ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.111+0000 7f1826ffd700 1 -- 172.21.15.93:0/678239137 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838004960 con 0x555a4154d6b0 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.111+0000 7f1826ffd700 1 -- 172.21.15.93:0/678239137 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f1838004ac0 con 0x555a4154d6b0 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.111+0000 7f1861362b80 1 -- 172.21.15.93:0/678239137 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a416e4960 con 0x555a4154d6b0 2023-11-16T16:49:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.111+0000 7f1826ffd700 1 -- 172.21.15.93:0/678239137 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1838019670 con 0x555a4154d6b0 2023-11-16T16:49:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.111+0000 7f1861362b80 10 client.15345 fetch_fsmap learned FSMap version 23 2023-11-16T16:49:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.111+0000 7f1861362b80 1 -- 172.21.15.93:0/678239137 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555a41609130 con 0x555a4154d6b0 2023-11-16T16:49:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.112+0000 7f1826ffd700 1 -- 172.21.15.93:0/678239137 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a416e4960 con 0x555a4154d6b0 2023-11-16T16:49:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.112+0000 7f1826ffd700 1 -- 172.21.15.93:0/678239137 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1838024850 con 0x555a4154d6b0 2023-11-16T16:49:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.112+0000 7f1861362b80 10 client.15345 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:49:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.112+0000 7f1861362b80 10 client.15345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.112+0000 7f1861362b80 20 client.15345 populate_metadata read hostname 'smithi093' 2023-11-16T16:49:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.112+0000 7f1861362b80 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x555a41706af0 0x555a41708f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.112+0000 7f1861362b80 4 client.15345 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:49:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.112+0000 7f1861362b80 1 -- 172.21.15.93:0/678239137 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a41605040 con 0x555a41706af0 2023-11-16T16:49:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.112+0000 7f18467fc700 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x555a41706af0 0x555a41708f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.113+0000 7f18467fc700 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x555a41706af0 0x555a41708f00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7845344a2ded852 server_cookie=a69af16df257cbe2 in_seq=0 out_seq=0 2023-11-16T16:49:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.113+0000 7f1826ffd700 10 client.15345 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1826ffd700 1 -- 172.21.15.93:0/678239137 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555a41605040 con 0x555a41706af0 2023-11-16T16:49:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1826ffd700 10 client.15345 handle_command_reply: tid=0 2023-11-16T16:49:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1861362b80 1 client.15345 shutdown 2023-11-16T16:49:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1861362b80 1 -- 172.21.15.93:0/678239137 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 msgr2=0x555a416fae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1861362b80 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 0x555a416fae80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1838009ef0 tx=0x7f1838009900 comp rx=0 tx=0).stop 2023-11-16T16:49:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1861362b80 1 -- 172.21.15.93:0/678239137 shutdown_connections 2023-11-16T16:49:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1861362b80 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x555a41706af0 0x555a41708f00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1861362b80 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a4154d6b0 0x555a416fae80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1861362b80 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f2d60 0x555a41702400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1861362b80 1 --2- 172.21.15.93:0/678239137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a416f1fb0 0x555a416fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1861362b80 1 -- 172.21.15.93:0/678239137 >> 172.21.15.93:0/678239137 conn(0x555a416e71a0 msgr2=0x555a416eba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1826ffd700 0 client.15345 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1861362b80 1 -- 172.21.15.93:0/678239137 shutdown_connections 2023-11-16T16:49:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1861362b80 1 -- 172.21.15.93:0/678239137 wait complete. 2023-11-16T16:49:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.114+0000 7f1861362b80 20 client.15345 trim_cache size 0 max 16384 2023-11-16T16:49:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.138+0000 7f1861362b80 1 Processor -- start 2023-11-16T16:49:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.138+0000 7f1861362b80 1 -- start start 2023-11-16T16:49:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.138+0000 7f1861362b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 0x555a416f3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.138+0000 7f1861362b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f1fb0 0x555a417023c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.138+0000 7f1861362b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a416f2d60 0x555a4170d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.138+0000 7f1861362b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a416f2640 con 0x555a416f2d60 2023-11-16T16:49:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.138+0000 7f1861362b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a416f67e0 con 0x555a4154d6b0 2023-11-16T16:49:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.138+0000 7f1861362b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4170dda0 con 0x555a416f1fb0 2023-11-16T16:49:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.138+0000 7f1846ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 0x555a416f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.139+0000 7f1846ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 0x555a416f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:33614/0 (socket says 172.21.15.93:33614) 2023-11-16T16:49:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.139+0000 7f1846ffd700 1 -- 172.21.15.93:0/2323820103 learned_addr learned my addr 172.21.15.93:0/2323820103 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.139+0000 7f18477fe700 1 --2- 172.21.15.93:0/2323820103 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a416f2d60 0x555a4170d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.139+0000 7f18467fc700 1 --2- 172.21.15.93:0/2323820103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f1fb0 0x555a417023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.139+0000 7f1846ffd700 1 -- 172.21.15.93:0/2323820103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f1fb0 msgr2=0x555a417023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.139+0000 7f1846ffd700 1 --2- 172.21.15.93:0/2323820103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f1fb0 0x555a417023c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.139+0000 7f1846ffd700 1 -- 172.21.15.93:0/2323820103 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a416f2d60 msgr2=0x555a4170d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.139+0000 7f1846ffd700 1 --2- 172.21.15.93:0/2323820103 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a416f2d60 0x555a4170d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.139+0000 7f1846ffd700 1 -- 172.21.15.93:0/2323820103 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a41608e80 con 0x555a4154d6b0 2023-11-16T16:49:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.139+0000 7f1846ffd700 1 --2- 172.21.15.93:0/2323820103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 0x555a416f3130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1838009ba0 tx=0x7f1838005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8f9d8fd325a338 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.139+0000 7f182e7fc700 1 -- 172.21.15.93:0/2323820103 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838004b60 con 0x555a4154d6b0 2023-11-16T16:49:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.140+0000 7f182e7fc700 1 -- 172.21.15.93:0/2323820103 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1838031070 con 0x555a4154d6b0 2023-11-16T16:49:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.140+0000 7f182e7fc700 1 -- 172.21.15.93:0/2323820103 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838019780 con 0x555a4154d6b0 2023-11-16T16:49:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.140+0000 7f1861362b80 1 -- 172.21.15.93:0/2323820103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 msgr2=0x555a416f3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.140+0000 7f1861362b80 1 --2- 172.21.15.93:0/2323820103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 0x555a416f3130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1838009ba0 tx=0x7f1838005a00 comp rx=0 tx=0).stop 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.140+0000 7f1861362b80 1 -- 172.21.15.93:0/2323820103 shutdown_connections 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.140+0000 7f1861362b80 1 --2- 172.21.15.93:0/2323820103 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a416f2d60 0x555a4170d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.140+0000 7f1861362b80 1 --2- 172.21.15.93:0/2323820103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 0x555a416f3130 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.140+0000 7f1861362b80 1 --2- 172.21.15.93:0/2323820103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f1fb0 0x555a417023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.140+0000 7f1861362b80 1 -- 172.21.15.93:0/2323820103 >> 172.21.15.93:0/2323820103 conn(0x555a416e71a0 msgr2=0x555a416f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.140+0000 7f1861362b80 1 -- 172.21.15.93:0/2323820103 shutdown_connections 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.140+0000 7f1861362b80 1 -- 172.21.15.93:0/2323820103 wait complete. 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.141+0000 7f1861362b80 1 -- start start 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.141+0000 7f1861362b80 1 Processor -- start 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.141+0000 7f1861362b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 0x555a416fd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.141+0000 7f1861362b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f1fb0 0x555a416fda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.141+0000 7f1861362b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a416f2d60 0x555a41700a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.141+0000 7f1861362b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a41700fb0 con 0x555a416f2d60 2023-11-16T16:49:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.141+0000 7f1861362b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a41701120 con 0x555a4154d6b0 2023-11-16T16:49:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.141+0000 7f1861362b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a417012e0 con 0x555a416f1fb0 2023-11-16T16:49:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.141+0000 7f1861362b80 10 client.0 _ll_register_callbacks cb 0x7f184001e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f18467fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f1fb0 0x555a416fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f18467fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f1fb0 0x555a416fda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:42272/0 (socket says 172.21.15.93:42272) 2023-11-16T16:49:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f1846ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 0x555a416fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f18467fc700 1 -- 172.21.15.93:0/2971675714 learned_addr learned my addr 172.21.15.93:0/2971675714 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f18477fe700 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a416f2d60 0x555a41700a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f18467fc700 1 -- 172.21.15.93:0/2971675714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 msgr2=0x555a416fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f18467fc700 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 0x555a416fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f18467fc700 1 -- 172.21.15.93:0/2971675714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a416f2d60 msgr2=0x555a41700a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f18467fc700 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a416f2d60 0x555a41700a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f18467fc700 1 -- 172.21.15.93:0/2971675714 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a41609130 con 0x555a416f1fb0 2023-11-16T16:49:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f1846ffd700 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 0x555a416fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:49:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f18467fc700 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f1fb0 0x555a416fda50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f183c09ad10 tx=0x7f183c0094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90fdb95bbf3408dd server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.142+0000 7f1826ffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-11-16T16:49:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.143+0000 7f1826ffd700 1 -- 172.21.15.93:0/2971675714 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c09b470 con 0x555a416f1fb0 2023-11-16T16:49:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.143+0000 7f1861362b80 1 -- 172.21.15.93:0/2971675714 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18380054e0 con 0x555a416f1fb0 2023-11-16T16:49:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.143+0000 7f1826ffd700 1 -- 172.21.15.93:0/2971675714 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f183c0a33d0 con 0x555a416f1fb0 2023-11-16T16:49:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.143+0000 7f1826ffd700 1 -- 172.21.15.93:0/2971675714 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f183c0b9150 con 0x555a416f1fb0 2023-11-16T16:49:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.143+0000 7f1861362b80 10 client.25384 fetch_fsmap learned FSMap version 23 2023-11-16T16:49:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.143+0000 7f1861362b80 1 -- 172.21.15.93:0/2971675714 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555a41459c00 con 0x555a416f1fb0 2023-11-16T16:49:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.143+0000 7f1826ffd700 1 -- 172.21.15.93:0/2971675714 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18380054e0 con 0x555a416f1fb0 2023-11-16T16:49:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.143+0000 7f1826ffd700 1 -- 172.21.15.93:0/2971675714 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f183c0a3040 con 0x555a416f1fb0 2023-11-16T16:49:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.144+0000 7f1861362b80 10 client.25384 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:49:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.144+0000 7f1861362b80 10 client.25384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.144+0000 7f1861362b80 20 client.25384 populate_metadata read hostname 'smithi093' 2023-11-16T16:49:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.144+0000 7f1861362b80 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x555a4175a9a0 0x555a4175cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.144+0000 7f1861362b80 4 client.25384 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:49:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.144+0000 7f1861362b80 1 -- 172.21.15.93:0/2971675714 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a41611600 con 0x555a4175a9a0 2023-11-16T16:49:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.144+0000 7f1846ffd700 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x555a4175a9a0 0x555a4175cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.144+0000 7f1846ffd700 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x555a4175a9a0 0x555a4175cdb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36735e6ac6aac715 server_cookie=f4dc204cec2c8a8f in_seq=0 out_seq=0 2023-11-16T16:49:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.144+0000 7f1826ffd700 10 client.25384 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.148+0000 7f1826ffd700 1 -- 172.21.15.93:0/2971675714 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a41611600 con 0x555a4175a9a0 2023-11-16T16:49:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.148+0000 7f1826ffd700 10 client.25384 handle_command_reply: tid=0 2023-11-16T16:49:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.148+0000 7f1861362b80 1 client.25384 shutdown 2023-11-16T16:49:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.148+0000 7f1861362b80 1 -- 172.21.15.93:0/2971675714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f1fb0 msgr2=0x555a416fda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.148+0000 7f1861362b80 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f1fb0 0x555a416fda50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f183c09ad10 tx=0x7f183c0094b0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.148+0000 7f1861362b80 1 -- 172.21.15.93:0/2971675714 shutdown_connections 2023-11-16T16:49:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.148+0000 7f1861362b80 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x555a4175a9a0 0x555a4175cdb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.148+0000 7f1861362b80 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a416f2d60 0x555a41700a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.148+0000 7f1861362b80 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a4154d6b0 0x555a416fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.149+0000 7f1826ffd700 0 client.25384 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.148+0000 7f1861362b80 1 --2- 172.21.15.93:0/2971675714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a416f1fb0 0x555a416fda50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.149+0000 7f1861362b80 1 -- 172.21.15.93:0/2971675714 >> 172.21.15.93:0/2971675714 conn(0x555a416e71a0 msgr2=0x555a416f9140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.149+0000 7f1861362b80 1 -- 172.21.15.93:0/2971675714 shutdown_connections 2023-11-16T16:49:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.149+0000 7f1861362b80 1 -- 172.21.15.93:0/2971675714 wait complete. 2023-11-16T16:49:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.149+0000 7f1861362b80 20 client.25384 trim_cache size 0 max 16384 2023-11-16T16:49:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:49:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:49:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:49:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:49:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.149+0000 7f1847fff700 1 -- 172.21.15.93:0/4043754152 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f18300a11d0 msgr2=0x7f18300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.149+0000 7f1847fff700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f18300a11d0 0x7f18300a3680 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1840009ec0 tx=0x7f184000fc40 comp rx=0 tx=0).stop 2023-11-16T16:49:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.149+0000 7f1847fff700 1 -- 172.21.15.93:0/4043754152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1848101250 msgr2=0x7f18481985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.149+0000 7f1847fff700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1848101250 0x7f18481985e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f183c0098e0 tx=0x7f183c005740 comp rx=0 tx=0).stop 2023-11-16T16:49:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.150+0000 7f1847fff700 1 -- 172.21.15.93:0/4043754152 shutdown_connections 2023-11-16T16:49:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.150+0000 7f1847fff700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f18300a11d0 0x7f18300a3680 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.150+0000 7f1847fff700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1848100060 0x7f18481980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.150+0000 7f1847fff700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1848101be0 0x7f184819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.150+0000 7f1847fff700 1 --2- 172.21.15.93:0/4043754152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1848101250 0x7f18481985e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.150+0000 7f1847fff700 1 -- 172.21.15.93:0/4043754152 >> 172.21.15.93:0/4043754152 conn(0x7f18480fb640 msgr2=0x7f1848106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.150+0000 7f1847fff700 1 -- 172.21.15.93:0/4043754152 shutdown_connections 2023-11-16T16:49:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:15.150+0000 7f1847fff700 1 -- 172.21.15.93:0/4043754152 wait complete. 2023-11-16T16:49:15.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:15 smithi093 ceph-mon[97863]: pgmap v364: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-11-16T16:49:15.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:15 smithi159 ceph-mon[131528]: pgmap v364: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-11-16T16:49:16.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:15 smithi125 ceph-mon[131076]: pgmap v364: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-11-16T16:49:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:17 smithi093 ceph-mon[97863]: pgmap v365: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 853 B/s wr, 9 op/s 2023-11-16T16:49:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:17 smithi159 ceph-mon[131528]: pgmap v365: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 853 B/s wr, 9 op/s 2023-11-16T16:49:18.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:17 smithi125 ceph-mon[131076]: pgmap v365: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 853 B/s wr, 9 op/s 2023-11-16T16:49:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:19 smithi093 ceph-mon[97863]: pgmap v366: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:49:19.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:19 smithi159 ceph-mon[131528]: pgmap v366: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:49:20.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:19 smithi125 ceph-mon[131076]: pgmap v366: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:49:21.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:21 smithi093 ceph-mon[97863]: pgmap v367: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-16T16:49:21.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:21 smithi159 ceph-mon[131528]: pgmap v367: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-16T16:49:21.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:21 smithi125 ceph-mon[131076]: pgmap v367: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-16T16:49:23.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:23 smithi093 ceph-mon[97863]: pgmap v368: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:49:32.980 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:25 smithi093 ceph-mon[97863]: pgmap v369: 129 pgs: 129 active+clean; 664 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-11-16T16:49:32.980 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:27 smithi093 ceph-mon[97863]: pgmap v370: 129 pgs: 129 active+clean; 664 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-11-16T16:49:32.982 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:23 smithi125 ceph-mon[131076]: pgmap v368: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:49:32.982 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:25 smithi125 ceph-mon[131076]: pgmap v369: 129 pgs: 129 active+clean; 664 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-11-16T16:49:32.982 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:27 smithi125 ceph-mon[131076]: pgmap v370: 129 pgs: 129 active+clean; 664 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-11-16T16:49:32.982 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:29 smithi125 ceph-mon[131076]: pgmap v371: 129 pgs: 129 active+clean; 664 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-16T16:49:32.982 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:31 smithi125 ceph-mon[131076]: pgmap v372: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-11-16T16:49:32.982 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:23 smithi159 ceph-mon[131528]: pgmap v368: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:49:32.983 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:25 smithi159 ceph-mon[131528]: pgmap v369: 129 pgs: 129 active+clean; 664 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-11-16T16:49:32.983 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:27 smithi159 ceph-mon[131528]: pgmap v370: 129 pgs: 129 active+clean; 664 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-11-16T16:49:32.983 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:29 smithi159 ceph-mon[131528]: pgmap v371: 129 pgs: 129 active+clean; 664 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-16T16:49:32.983 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:31 smithi159 ceph-mon[131528]: pgmap v372: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-11-16T16:49:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.312+0000 7fdf0f59d700 1 -- 172.21.15.93:0/4017302643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf08100060 msgr2=0x7fdf08100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.312+0000 7fdf0f59d700 1 --2- 172.21.15.93:0/4017302643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf08100060 0x7fdf08100470 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009990 tx=0x7fdf04009ca0 comp rx=0 tx=0).stop 2023-11-16T16:49:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.312+0000 7fdf0f59d700 1 -- 172.21.15.93:0/4017302643 shutdown_connections 2023-11-16T16:49:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.312+0000 7fdf0f59d700 1 --2- 172.21.15.93:0/4017302643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf08100060 0x7fdf08100470 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.312+0000 7fdf0f59d700 1 --2- 172.21.15.93:0/4017302643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf08101be0 0x7fdf081084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.312+0000 7fdf0f59d700 1 --2- 172.21.15.93:0/4017302643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf08101250 0x7fdf081016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.312+0000 7fdf0f59d700 1 -- 172.21.15.93:0/4017302643 >> 172.21.15.93:0/4017302643 conn(0x7fdf080fb640 msgr2=0x7fdf080fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.312+0000 7fdf0f59d700 1 -- 172.21.15.93:0/4017302643 shutdown_connections 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.312+0000 7fdf0f59d700 1 -- 172.21.15.93:0/4017302643 wait complete. 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.313+0000 7fdf0f59d700 1 Processor -- start 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.314+0000 7fdf0f59d700 1 -- start start 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.314+0000 7fdf0f59d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf08100060 0x7fdf08198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.314+0000 7fdf0f59d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf08101250 0x7fdf081985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.314+0000 7fdf0f59d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf08101be0 0x7fdf0819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.314+0000 7fdf0f59d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0819fbe0 con 0x7fdf08101250 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.314+0000 7fdf0f59d700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0819fd20 con 0x7fdf08100060 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.314+0000 7fdf0f59d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0819feb0 con 0x7fdf08101be0 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.314+0000 7fdf0e59b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf08100060 0x7fdf08198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.314+0000 7fdf0dd9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf08101250 0x7fdf081985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.314+0000 7fdf0e59b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf08100060 0x7fdf08198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:38808/0 (socket says 172.21.15.93:38808) 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.314+0000 7fdf0e59b700 1 -- 172.21.15.93:0/1779505757 learned_addr learned my addr 172.21.15.93:0/1779505757 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.314+0000 7fdf0ed9c700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf08101be0 0x7fdf0819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.315+0000 7fdf0dd9a700 1 -- 172.21.15.93:0/1779505757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf08101be0 msgr2=0x7fdf0819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.315+0000 7fdf0dd9a700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf08101be0 0x7fdf0819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.315+0000 7fdf0dd9a700 1 -- 172.21.15.93:0/1779505757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf08100060 msgr2=0x7fdf08198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.315+0000 7fdf0dd9a700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf08100060 0x7fdf08198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.315+0000 7fdf0dd9a700 1 -- 172.21.15.93:0/1779505757 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef8009520 con 0x7fdf08101250 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.315+0000 7fdf0dd9a700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf08101250 0x7fdf081985a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdef800e9f0 tx=0x7fdef800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.315+0000 7fdeff7fe700 1 -- 172.21.15.93:0/1779505757 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef800cb70 con 0x7fdf08101250 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.315+0000 7fdeff7fe700 1 -- 172.21.15.93:0/1779505757 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdef800ee90 con 0x7fdf08101250 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.315+0000 7fdf0f59d700 1 -- 172.21.15.93:0/1779505757 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf040095f0 con 0x7fdf08101250 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.315+0000 7fdeff7fe700 1 -- 172.21.15.93:0/1779505757 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8003cf0 con 0x7fdf08101250 2023-11-16T16:49:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.316+0000 7fdf0f59d700 1 -- 172.21.15.93:0/1779505757 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0810ba10 con 0x7fdf08101250 2023-11-16T16:49:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.320+0000 7fdeff7fe700 1 -- 172.21.15.93:0/1779505757 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdef800ccd0 con 0x7fdf08101250 2023-11-16T16:49:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.321+0000 7fdf24144b80 1 Processor -- start 2023-11-16T16:49:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdf24144b80 1 -- start start 2023-11-16T16:49:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdf24144b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979aada6b0 0x55979ac7ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdf24144b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7efb0 0x55979ac7f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdf24144b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7fd60 0x55979ac80200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdeff7fe700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdef40a1150 0x7fdef40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdf24144b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979ab84000 con 0x55979ac7fd60 2023-11-16T16:49:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdf24144b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979abd6840 con 0x55979ac7efb0 2023-11-16T16:49:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdf24144b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979abcfcd0 con 0x55979aada6b0 2023-11-16T16:49:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdeff7fe700 1 -- 172.21.15.93:0/1779505757 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fdef8014070 con 0x7fdf08101250 2023-11-16T16:49:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdf0ed9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979aada6b0 0x55979ac7ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdf0ed9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979aada6b0 0x55979ac7ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:56442/0 (socket says 172.21.15.93:56442) 2023-11-16T16:49:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdf0ed9c700 1 -- 172.21.15.93:0/307221156 learned_addr learned my addr 172.21.15.93:0/307221156 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.322+0000 7fdf0ed9c700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdef40a1150 0x7fdef40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.323+0000 7fdf0e59b700 1 --2- 172.21.15.93:0/307221156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7efb0 0x55979ac7f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.323+0000 7fdf0dd9a700 1 --2- 172.21.15.93:0/307221156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7fd60 0x55979ac80200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.323+0000 7fdf0e59b700 1 -- 172.21.15.93:0/307221156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979aada6b0 msgr2=0x55979ac7ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.323+0000 7fdf0e59b700 1 --2- 172.21.15.93:0/307221156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979aada6b0 0x55979ac7ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.323+0000 7fdf0e59b700 1 -- 172.21.15.93:0/307221156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7fd60 msgr2=0x55979ac80200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.323+0000 7fdf0e59b700 1 --2- 172.21.15.93:0/307221156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7fd60 0x55979ac80200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.323+0000 7fdf0e59b700 1 -- 172.21.15.93:0/307221156 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55979aa515d0 con 0x55979ac7efb0 2023-11-16T16:49:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.323+0000 7fdf0ed9c700 1 --2- 172.21.15.93:0/307221156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979aada6b0 0x55979ac7ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:49:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.323+0000 7fdf0e59b700 1 --2- 172.21.15.93:0/307221156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7efb0 0x55979ac7f820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdf04000c00 tx=0x7fdf04005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c3581b4e8d87217 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.323+0000 7fdf0ed9c700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdef40a1150 0x7fdef40a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55979ac79ee0 tx=0x7fdf0000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.324+0000 7fdefcff9700 1 -- 172.21.15.93:0/307221156 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0401d070 con 0x55979ac7efb0 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.324+0000 7fdefcff9700 1 -- 172.21.15.93:0/307221156 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0401e040 con 0x55979ac7efb0 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.324+0000 7fdefcff9700 1 -- 172.21.15.93:0/307221156 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04004920 con 0x55979ac7efb0 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.325+0000 7fdf24144b80 1 -- 172.21.15.93:0/307221156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7efb0 msgr2=0x55979ac7f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.325+0000 7fdf24144b80 1 --2- 172.21.15.93:0/307221156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7efb0 0x55979ac7f820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdf04000c00 tx=0x7fdf04005ea0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.325+0000 7fdf24144b80 1 -- 172.21.15.93:0/307221156 shutdown_connections 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.325+0000 7fdf24144b80 1 --2- 172.21.15.93:0/307221156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7fd60 0x55979ac80200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.325+0000 7fdf24144b80 1 --2- 172.21.15.93:0/307221156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979aada6b0 0x55979ac7ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.325+0000 7fdf24144b80 1 --2- 172.21.15.93:0/307221156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7efb0 0x55979ac7f820 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.325+0000 7fdf24144b80 1 -- 172.21.15.93:0/307221156 >> 172.21.15.93:0/307221156 conn(0x55979ac741a0 msgr2=0x55979ac792e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.325+0000 7fdf24144b80 1 -- 172.21.15.93:0/307221156 shutdown_connections 2023-11-16T16:49:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.325+0000 7fdf24144b80 1 -- 172.21.15.93:0/307221156 wait complete. 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.326+0000 7fdf24144b80 1 -- start start 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.326+0000 7fdf24144b80 1 Processor -- start 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf24144b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979aada6b0 0x55979ac90630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf24144b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 0x55979ac90b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf24144b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 0x55979ac97bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf24144b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979ab71880 con 0x55979ac7efb0 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf24144b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979ab71520 con 0x55979aada6b0 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf24144b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979aa538c0 con 0x55979ac7fd60 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf24144b80 10 client.0 _ll_register_callbacks cb 0x55979ab8f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf0e59b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979aada6b0 0x55979ac90630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf0dd9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 0x55979ac90b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf0ed9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 0x55979ac97bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf0ed9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 0x55979ac97bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:56454/0 (socket says 172.21.15.93:56454) 2023-11-16T16:49:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf0dd9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 0x55979ac90b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56830/0 (socket says 172.21.15.93:56830) 2023-11-16T16:49:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.327+0000 7fdf0ed9c700 1 -- 172.21.15.93:0/1688961437 learned_addr learned my addr 172.21.15.93:0/1688961437 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.328+0000 7fdf0ed9c700 1 -- 172.21.15.93:0/1688961437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979aada6b0 msgr2=0x55979ac90630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.328+0000 7fdf0ed9c700 1 --2- 172.21.15.93:0/1688961437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979aada6b0 0x55979ac90630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.328+0000 7fdf0ed9c700 1 -- 172.21.15.93:0/1688961437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 msgr2=0x55979ac90b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.328+0000 7fdf0ed9c700 1 --2- 172.21.15.93:0/1688961437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 0x55979ac90b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.328+0000 7fdf0ed9c700 1 -- 172.21.15.93:0/1688961437 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf08108dd0 con 0x55979ac7fd60 2023-11-16T16:49:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.328+0000 7fdf0ed9c700 1 --2- 172.21.15.93:0/1688961437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 0x55979ac97bb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009fa0 tx=0x7fdf00004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb898d341932913 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.328+0000 7fdef1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-11-16T16:49:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.328+0000 7fdef1ffb700 1 -- 172.21.15.93:0/1688961437 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000e070 con 0x55979ac7fd60 2023-11-16T16:49:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.329+0000 7fdef1ffb700 1 -- 172.21.15.93:0/1688961437 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fdf0001bd50 con 0x55979ac7fd60 2023-11-16T16:49:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.329+0000 7fdf24144b80 1 -- 172.21.15.93:0/1688961437 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55979ac71960 con 0x55979ac7fd60 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.329+0000 7fdef1ffb700 1 -- 172.21.15.93:0/1688961437 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf00004bb0 con 0x55979ac7fd60 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.329+0000 7fdf24144b80 10 client.25396 fetch_fsmap learned FSMap version 23 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.329+0000 7fdf24144b80 1 -- 172.21.15.93:0/1688961437 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55979ab96130 con 0x55979ac7fd60 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.330+0000 7fdef1ffb700 1 -- 172.21.15.93:0/1688961437 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55979ac71960 con 0x55979ac7fd60 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.330+0000 7fdef1ffb700 1 -- 172.21.15.93:0/1688961437 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdf00024070 con 0x55979ac7fd60 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.330+0000 7fdf24144b80 10 client.25396 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.330+0000 7fdf24144b80 10 client.25396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.330+0000 7fdf24144b80 20 client.25396 populate_metadata read hostname 'smithi093' 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.330+0000 7fdf24144b80 1 --2- 172.21.15.93:0/1688961437 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55979ac8be10 0x55979ac8e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.331+0000 7fdf24144b80 4 client.25396 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.331+0000 7fdf24144b80 1 -- 172.21.15.93:0/1688961437 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55979ab92040 con 0x55979ac8be10 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.331+0000 7fdf0e59b700 1 --2- 172.21.15.93:0/1688961437 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55979ac8be10 0x55979ac8e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.332+0000 7fdef1ffb700 10 client.25396 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.332+0000 7fdf0e59b700 1 --2- 172.21.15.93:0/1688961437 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55979ac8be10 0x55979ac8e220 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=393cf61abd8fb2d1 server_cookie=f84cc98391545b57 in_seq=0 out_seq=0 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.332+0000 7fdef1ffb700 1 -- 172.21.15.93:0/1688961437 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55979ab92040 con 0x55979ac8be10 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.332+0000 7fdef1ffb700 10 client.25396 handle_command_reply: tid=0 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.332+0000 7fdf24144b80 1 client.25396 shutdown 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.333+0000 7fdf24144b80 1 -- 172.21.15.93:0/1688961437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 msgr2=0x55979ac97bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.333+0000 7fdf24144b80 1 --2- 172.21.15.93:0/1688961437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 0x55979ac97bb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009fa0 tx=0x7fdf00004b40 comp rx=0 tx=0).stop 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.333+0000 7fdf24144b80 1 -- 172.21.15.93:0/1688961437 shutdown_connections 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.333+0000 7fdf24144b80 1 --2- 172.21.15.93:0/1688961437 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55979ac8be10 0x55979ac8e220 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.333+0000 7fdf24144b80 1 --2- 172.21.15.93:0/1688961437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 0x55979ac97bb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.333+0000 7fdf24144b80 1 --2- 172.21.15.93:0/1688961437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979aada6b0 0x55979ac90630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.333+0000 7fdf24144b80 1 --2- 172.21.15.93:0/1688961437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 0x55979ac90b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.333+0000 7fdf24144b80 1 -- 172.21.15.93:0/1688961437 >> 172.21.15.93:0/1688961437 conn(0x55979ac741a0 msgr2=0x55979ac78a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.333+0000 7fdef1ffb700 0 client.25396 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.333+0000 7fdf24144b80 1 -- 172.21.15.93:0/1688961437 shutdown_connections 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.333+0000 7fdf24144b80 1 -- 172.21.15.93:0/1688961437 wait complete. 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.334+0000 7fdf24144b80 20 client.25396 trim_cache size 0 max 16384 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.351+0000 7fdf24144b80 1 Processor -- start 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.351+0000 7fdf24144b80 1 -- start start 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.351+0000 7fdf24144b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979aada6b0 0x55979ac86060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.351+0000 7fdf24144b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 0x55979ac97b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.351+0000 7fdf24144b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 0x55979ac82b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.351+0000 7fdf24144b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979ac872d0 con 0x55979ac7efb0 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.351+0000 7fdf24144b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979ac73d70 con 0x55979aada6b0 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.351+0000 7fdf24144b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979ac7a160 con 0x55979ac7fd60 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.352+0000 7fdf0e59b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979aada6b0 0x55979ac86060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.352+0000 7fdf0e59b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979aada6b0 0x55979ac86060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:38840/0 (socket says 172.21.15.93:38840) 2023-11-16T16:49:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.352+0000 7fdf0e59b700 1 -- 172.21.15.93:0/527721314 learned_addr learned my addr 172.21.15.93:0/527721314 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.352+0000 7fdf0dd9a700 1 --2- 172.21.15.93:0/527721314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 0x55979ac97b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.352+0000 7fdf0ed9c700 1 --2- 172.21.15.93:0/527721314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 0x55979ac82b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.352+0000 7fdf0e59b700 1 -- 172.21.15.93:0/527721314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 msgr2=0x55979ac82b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.352+0000 7fdf0e59b700 1 --2- 172.21.15.93:0/527721314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 0x55979ac82b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.352+0000 7fdf0e59b700 1 -- 172.21.15.93:0/527721314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 msgr2=0x55979ac97b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.352+0000 7fdf0e59b700 1 --2- 172.21.15.93:0/527721314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 0x55979ac97b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.352+0000 7fdf0e59b700 1 -- 172.21.15.93:0/527721314 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55979ab95e80 con 0x55979aada6b0 2023-11-16T16:49:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.352+0000 7fdf0ed9c700 1 --2- 172.21.15.93:0/527721314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 0x55979ac82b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:49:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.353+0000 7fdf0e59b700 1 --2- 172.21.15.93:0/527721314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979aada6b0 0x55979ac86060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55979ac7e7d0 tx=0x7fdf0401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e9e7b3f52e27619 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.353+0000 7fdefd7fa700 1 -- 172.21.15.93:0/527721314 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0401d040 con 0x55979aada6b0 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.353+0000 7fdefd7fa700 1 -- 172.21.15.93:0/527721314 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf04034d20 con 0x55979aada6b0 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.353+0000 7fdefd7fa700 1 -- 172.21.15.93:0/527721314 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0400bcb0 con 0x55979aada6b0 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.354+0000 7fdf24144b80 1 -- 172.21.15.93:0/527721314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979aada6b0 msgr2=0x55979ac86060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.354+0000 7fdf24144b80 1 --2- 172.21.15.93:0/527721314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979aada6b0 0x55979ac86060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55979ac7e7d0 tx=0x7fdf0401a040 comp rx=0 tx=0).stop 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.354+0000 7fdf24144b80 1 -- 172.21.15.93:0/527721314 shutdown_connections 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.354+0000 7fdf24144b80 1 --2- 172.21.15.93:0/527721314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979ac7fd60 0x55979ac82b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.354+0000 7fdf24144b80 1 --2- 172.21.15.93:0/527721314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979aada6b0 0x55979ac86060 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.354+0000 7fdf24144b80 1 --2- 172.21.15.93:0/527721314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 0x55979ac97b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.354+0000 7fdf24144b80 1 -- 172.21.15.93:0/527721314 >> 172.21.15.93:0/527721314 conn(0x55979ac741a0 msgr2=0x55979acad120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.354+0000 7fdf24144b80 1 -- 172.21.15.93:0/527721314 shutdown_connections 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.354+0000 7fdf24144b80 1 -- 172.21.15.93:0/527721314 wait complete. 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.354+0000 7fdf24144b80 1 -- start start 2023-11-16T16:49:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.355+0000 7fdf24144b80 1 Processor -- start 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.355+0000 7fdf24144b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979aada6b0 0x55979ac91cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.355+0000 7fdf24144b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 0x55979ac922d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.355+0000 7fdf24144b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7fd60 0x55979ac97030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.355+0000 7fdf24144b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979ac97570 con 0x55979ac7efb0 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.355+0000 7fdf24144b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979ac976e0 con 0x55979ac7fd60 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.355+0000 7fdf24144b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979ac978a0 con 0x55979aada6b0 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.355+0000 7fdf24144b80 10 client.0 _ll_register_callbacks cb 0x7fdf00009350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.356+0000 7fdf0ed9c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7fd60 0x55979ac97030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.356+0000 7fdf0e59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979aada6b0 0x55979ac91cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.356+0000 7fdf0ed9c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7fd60 0x55979ac97030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:38846/0 (socket says 172.21.15.93:38846) 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.356+0000 7fdf0ed9c700 1 -- 172.21.15.93:0/1441578105 learned_addr learned my addr 172.21.15.93:0/1441578105 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.356+0000 7fdf0ed9c700 1 -- 172.21.15.93:0/1441578105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979aada6b0 msgr2=0x55979ac91cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.356+0000 7fdf0ed9c700 1 --2- 172.21.15.93:0/1441578105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979aada6b0 0x55979ac91cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.356+0000 7fdf0ed9c700 1 -- 172.21.15.93:0/1441578105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 msgr2=0x55979ac922d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.356+0000 7fdf0ed9c700 1 --2- 172.21.15.93:0/1441578105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 0x55979ac922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.356+0000 7fdf0ed9c700 1 -- 172.21.15.93:0/1441578105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55979aa515d0 con 0x55979ac7fd60 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.357+0000 7fdf0ed9c700 1 --2- 172.21.15.93:0/1441578105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7fd60 0x55979ac97030 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdf0001b010 tx=0x7fdf000043d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47e5b5c9546db772 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.357+0000 7fdef1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.357+0000 7fdef1ffb700 1 -- 172.21.15.93:0/1441578105 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf000045a0 con 0x55979ac7fd60 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.357+0000 7fdef1ffb700 1 -- 172.21.15.93:0/1441578105 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fdf00004bb0 con 0x55979ac7fd60 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.357+0000 7fdf24144b80 1 -- 172.21.15.93:0/1441578105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf0401e040 con 0x55979ac7fd60 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.357+0000 7fdef1ffb700 1 -- 172.21.15.93:0/1441578105 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf00004960 con 0x55979ac7fd60 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.357+0000 7fdf24144b80 10 client.25403 fetch_fsmap learned FSMap version 23 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.357+0000 7fdf24144b80 1 -- 172.21.15.93:0/1441578105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55979a9e6c00 con 0x55979ac7fd60 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.358+0000 7fdef1ffb700 1 -- 172.21.15.93:0/1441578105 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0401e040 con 0x55979ac7fd60 2023-11-16T16:49:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.358+0000 7fdef1ffb700 1 -- 172.21.15.93:0/1441578105 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdf0001b450 con 0x55979ac7fd60 2023-11-16T16:49:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.358+0000 7fdf24144b80 10 client.25403 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:49:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.358+0000 7fdf24144b80 10 client.25403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.358+0000 7fdf24144b80 20 client.25403 populate_metadata read hostname 'smithi093' 2023-11-16T16:49:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.358+0000 7fdf24144b80 1 --2- 172.21.15.93:0/1441578105 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55979acf93e0 0x55979acfb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.358+0000 7fdf24144b80 4 client.25403 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:49:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.358+0000 7fdf24144b80 1 -- 172.21.15.93:0/1441578105 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55979ab9e600 con 0x55979acf93e0 2023-11-16T16:49:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.359+0000 7fdf0e59b700 1 --2- 172.21.15.93:0/1441578105 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55979acf93e0 0x55979acfb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.359+0000 7fdf0e59b700 1 --2- 172.21.15.93:0/1441578105 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55979acf93e0 0x55979acfb7f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf85f2eb68098c38 server_cookie=7d0bc559ab65e287 in_seq=0 out_seq=0 2023-11-16T16:49:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.359+0000 7fdef1ffb700 10 client.25403 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.367+0000 7fdef1ffb700 1 -- 172.21.15.93:0/1441578105 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55979ab9e600 con 0x55979acf93e0 2023-11-16T16:49:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.367+0000 7fdef1ffb700 10 client.25403 handle_command_reply: tid=0 2023-11-16T16:49:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.367+0000 7fdf24144b80 1 client.25403 shutdown 2023-11-16T16:49:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.367+0000 7fdf24144b80 1 -- 172.21.15.93:0/1441578105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7fd60 msgr2=0x55979ac97030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.367+0000 7fdf24144b80 1 --2- 172.21.15.93:0/1441578105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7fd60 0x55979ac97030 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdf0001b010 tx=0x7fdf000043d0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.368+0000 7fdf24144b80 1 -- 172.21.15.93:0/1441578105 shutdown_connections 2023-11-16T16:49:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.368+0000 7fdf24144b80 1 --2- 172.21.15.93:0/1441578105 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55979acf93e0 0x55979acfb7f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.368+0000 7fdf24144b80 1 --2- 172.21.15.93:0/1441578105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55979ac7fd60 0x55979ac97030 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.368+0000 7fdf24144b80 1 --2- 172.21.15.93:0/1441578105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55979aada6b0 0x55979ac91cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.368+0000 7fdf24144b80 1 --2- 172.21.15.93:0/1441578105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55979ac7efb0 0x55979ac922d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.368+0000 7fdef1ffb700 0 client.25403 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.368+0000 7fdf24144b80 1 -- 172.21.15.93:0/1441578105 >> 172.21.15.93:0/1441578105 conn(0x55979ac741a0 msgr2=0x55979ac86060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.368+0000 7fdf24144b80 1 -- 172.21.15.93:0/1441578105 shutdown_connections 2023-11-16T16:49:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.368+0000 7fdf24144b80 1 -- 172.21.15.93:0/1441578105 wait complete. 2023-11-16T16:49:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.368+0000 7fdf24144b80 20 client.25403 trim_cache size 0 max 16384 2023-11-16T16:49:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:49:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:49:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:49:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:49:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.369+0000 7fdf0f59d700 1 -- 172.21.15.93:0/1779505757 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdef40a1150 msgr2=0x7fdef40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.369+0000 7fdf0f59d700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdef40a1150 0x7fdef40a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55979ac79ee0 tx=0x7fdf0000b040 comp rx=0 tx=0).stop 2023-11-16T16:49:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.369+0000 7fdf0f59d700 1 -- 172.21.15.93:0/1779505757 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf08101250 msgr2=0x7fdf081985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.369+0000 7fdf0f59d700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf08101250 0x7fdf081985a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdef800e9f0 tx=0x7fdef800ed00 comp rx=0 tx=0).stop 2023-11-16T16:49:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.369+0000 7fdf0f59d700 1 -- 172.21.15.93:0/1779505757 shutdown_connections 2023-11-16T16:49:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.369+0000 7fdf0f59d700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fdef40a1150 0x7fdef40a3600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.369+0000 7fdf0f59d700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf08101be0 0x7fdf0819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.369+0000 7fdf0f59d700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf08100060 0x7fdf08198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.369+0000 7fdf0f59d700 1 --2- 172.21.15.93:0/1779505757 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf08101250 0x7fdf081985a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.369+0000 7fdf0f59d700 1 -- 172.21.15.93:0/1779505757 >> 172.21.15.93:0/1779505757 conn(0x7fdf080fb640 msgr2=0x7fdf080fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.369+0000 7fdf0f59d700 1 -- 172.21.15.93:0/1779505757 shutdown_connections 2023-11-16T16:49:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:28.369+0000 7fdf0f59d700 1 -- 172.21.15.93:0/1779505757 wait complete. 2023-11-16T16:49:33.020 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:29 smithi093 ceph-mon[97863]: pgmap v371: 129 pgs: 129 active+clean; 664 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-16T16:49:33.020 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:31 smithi093 ceph-mon[97863]: pgmap v372: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-11-16T16:49:33.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:33 smithi093 ceph-mon[97863]: pgmap v373: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-11-16T16:49:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:33 smithi159 ceph-mon[131528]: pgmap v373: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-11-16T16:49:34.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:33 smithi125 ceph-mon[131076]: pgmap v373: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-11-16T16:49:35.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:35 smithi093 ceph-mon[97863]: pgmap v374: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-11-16T16:49:35.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:35 smithi159 ceph-mon[131528]: pgmap v374: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-11-16T16:49:36.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:35 smithi125 ceph-mon[131076]: pgmap v374: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-11-16T16:49:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:37 smithi093 ceph-mon[97863]: pgmap v375: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 7 op/s 2023-11-16T16:49:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:37 smithi159 ceph-mon[131528]: pgmap v375: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 7 op/s 2023-11-16T16:49:38.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:37 smithi125 ceph-mon[131076]: pgmap v375: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 7 op/s 2023-11-16T16:49:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:39 smithi093 ceph-mon[97863]: pgmap v376: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 7 op/s 2023-11-16T16:49:39.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:39 smithi159 ceph-mon[131528]: pgmap v376: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 7 op/s 2023-11-16T16:49:40.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:39 smithi125 ceph-mon[131076]: pgmap v376: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 7 op/s 2023-11-16T16:49:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.602+0000 7f574d1ff700 1 -- 172.21.15.93:0/1014801012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57481035e0 msgr2=0x7f57481059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.602+0000 7f574d1ff700 1 --2- 172.21.15.93:0/1014801012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57481035e0 0x7f57481059c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f573c009990 tx=0x7f573c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.602+0000 7f574d1ff700 1 -- 172.21.15.93:0/1014801012 shutdown_connections 2023-11-16T16:49:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.602+0000 7f574d1ff700 1 --2- 172.21.15.93:0/1014801012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5748105f00 0x7f5748108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.602+0000 7f574d1ff700 1 --2- 172.21.15.93:0/1014801012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5748100cc0 0x7f57481030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.602+0000 7f574d1ff700 1 --2- 172.21.15.93:0/1014801012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57481035e0 0x7f57481059c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.602+0000 7f574d1ff700 1 -- 172.21.15.93:0/1014801012 >> 172.21.15.93:0/1014801012 conn(0x7f57480fa720 msgr2=0x7f57480fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.602+0000 7f574d1ff700 1 -- 172.21.15.93:0/1014801012 shutdown_connections 2023-11-16T16:49:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.602+0000 7f574d1ff700 1 -- 172.21.15.93:0/1014801012 wait complete. 2023-11-16T16:49:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.604+0000 7f574d1ff700 1 Processor -- start 2023-11-16T16:49:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.604+0000 7f574d1ff700 1 -- start start 2023-11-16T16:49:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.604+0000 7f574d1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5748100cc0 0x7f5748100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.604+0000 7f574d1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57481035e0 0x7f57480fec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.605+0000 7f574d1ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5748105f00 0x7f57480ff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.605+0000 7f574d1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57480ff950 con 0x7f57481035e0 2023-11-16T16:49:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.605+0000 7f574d1ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57480ffac0 con 0x7f5748105f00 2023-11-16T16:49:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.605+0000 7f574d1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574819ffe0 con 0x7f5748100cc0 2023-11-16T16:49:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.605+0000 7f5746d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5748100cc0 0x7f5748100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.605+0000 7f5746d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5748100cc0 0x7f5748100630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:35658/0 (socket says 172.21.15.93:35658) 2023-11-16T16:49:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.605+0000 7f5746d9d700 1 -- 172.21.15.93:0/268210271 learned_addr learned my addr 172.21.15.93:0/268210271 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.605+0000 7f574659c700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57481035e0 0x7f57480fec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.605+0000 7f574759e700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5748105f00 0x7f57480ff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.606+0000 7f5746d9d700 1 -- 172.21.15.93:0/268210271 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5748105f00 msgr2=0x7f57480ff1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.606+0000 7f5746d9d700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5748105f00 0x7f57480ff1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.606+0000 7f5746d9d700 1 -- 172.21.15.93:0/268210271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57481035e0 msgr2=0x7f57480fec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.606+0000 7f5746d9d700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57481035e0 0x7f57480fec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.606+0000 7f5746d9d700 1 -- 172.21.15.93:0/268210271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c0095f0 con 0x7f5748100cc0 2023-11-16T16:49:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.606+0000 7f574659c700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57481035e0 0x7f57480fec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:49:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.606+0000 7f5746d9d700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5748100cc0 0x7f5748100630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f573800d950 tx=0x7f573800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.606+0000 7f572ffff700 1 -- 172.21.15.93:0/268210271 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800f890 con 0x7f5748100cc0 2023-11-16T16:49:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.606+0000 7f574d1ff700 1 -- 172.21.15.93:0/268210271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574810bc50 con 0x7f5748100cc0 2023-11-16T16:49:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.606+0000 7f572ffff700 1 -- 172.21.15.93:0/268210271 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5738010460 con 0x7f5748100cc0 2023-11-16T16:49:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.606+0000 7f572ffff700 1 -- 172.21.15.93:0/268210271 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800e610 con 0x7f5748100cc0 2023-11-16T16:49:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.607+0000 7f574d1ff700 1 -- 172.21.15.93:0/268210271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574810bac0 con 0x7f5748100cc0 2023-11-16T16:49:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.611+0000 7f572ffff700 1 -- 172.21.15.93:0/268210271 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f57380108d0 con 0x7f5748100cc0 2023-11-16T16:49:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.613+0000 7f5760ac6b80 1 Processor -- start 2023-11-16T16:49:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.613+0000 7f5760ac6b80 1 -- start start 2023-11-16T16:49:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.614+0000 7f5760ac6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 0x561de89c3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.614+0000 7f572ffff700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f57300a1150 0x7f57300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.614+0000 7f572ffff700 1 -- 172.21.15.93:0/268210271 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f57380ccb20 con 0x7f5748100cc0 2023-11-16T16:49:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.614+0000 7f5760ac6b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c3db0 0x561de89c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.614+0000 7f5760ac6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c4b10 0x561de89c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.614+0000 7f5760ac6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561de88c9000 con 0x561de881f6b0 2023-11-16T16:49:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.614+0000 7f5760ac6b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561de891b840 con 0x561de89c3db0 2023-11-16T16:49:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.614+0000 7f5760ac6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561de8914cd0 con 0x561de89c4b10 2023-11-16T16:49:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.614+0000 7f574759e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 0x561de89c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.614+0000 7f574759e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 0x561de89c3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59630/0 (socket says 172.21.15.93:59630) 2023-11-16T16:49:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.614+0000 7f574759e700 1 -- 172.21.15.93:0/1715967185 learned_addr learned my addr 172.21.15.93:0/1715967185 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.615+0000 7f574659c700 1 --2- 172.21.15.93:0/1715967185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c3db0 0x561de89c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.615+0000 7f574759e700 1 -- 172.21.15.93:0/1715967185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c4b10 msgr2=0x561de89c4f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:49:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.615+0000 7f574759e700 1 --2- 172.21.15.93:0/1715967185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c4b10 0x561de89c4f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.615+0000 7f574759e700 1 -- 172.21.15.93:0/1715967185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c3db0 msgr2=0x561de89c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.615+0000 7f574759e700 1 --2- 172.21.15.93:0/1715967185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c3db0 0x561de89c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.615+0000 7f574759e700 1 -- 172.21.15.93:0/1715967185 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561de87965d0 con 0x561de881f6b0 2023-11-16T16:49:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.615+0000 7f574659c700 1 --2- 172.21.15.93:0/1715967185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c3db0 0x561de89c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:49:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.615+0000 7f5746d9d700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f57300a1150 0x7f57300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.616+0000 7f574759e700 1 --2- 172.21.15.93:0/1715967185 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 0x561de89c3870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5740011630 tx=0x7f5740009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d61cd0e0a6b33f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.616+0000 7f572d7fa700 1 -- 172.21.15.93:0/1715967185 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574000e040 con 0x561de881f6b0 2023-11-16T16:49:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.616+0000 7f572d7fa700 1 -- 172.21.15.93:0/1715967185 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f574000f040 con 0x561de881f6b0 2023-11-16T16:49:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.616+0000 7f572d7fa700 1 -- 172.21.15.93:0/1715967185 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740004790 con 0x561de881f6b0 2023-11-16T16:49:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.617+0000 7f5760ac6b80 1 -- 172.21.15.93:0/1715967185 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 msgr2=0x561de89c3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.617+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/1715967185 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 0x561de89c3870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5740011630 tx=0x7f5740009920 comp rx=0 tx=0).stop 2023-11-16T16:49:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.617+0000 7f5746d9d700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f57300a1150 0x7f57300a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5738096000 tx=0x7f5738010cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.617+0000 7f5760ac6b80 1 -- 172.21.15.93:0/1715967185 shutdown_connections 2023-11-16T16:49:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.617+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/1715967185 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 0x561de89c3870 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.617+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/1715967185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c4b10 0x561de89c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.617+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/1715967185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c3db0 0x561de89c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.617+0000 7f5760ac6b80 1 -- 172.21.15.93:0/1715967185 >> 172.21.15.93:0/1715967185 conn(0x561de89b8f50 msgr2=0x561de89be100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.617+0000 7f5760ac6b80 1 -- 172.21.15.93:0/1715967185 shutdown_connections 2023-11-16T16:49:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.618+0000 7f5760ac6b80 1 -- 172.21.15.93:0/1715967185 wait complete. 2023-11-16T16:49:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.618+0000 7f5760ac6b80 1 -- start start 2023-11-16T16:49:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.619+0000 7f5760ac6b80 1 Processor -- start 2023-11-16T16:49:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.619+0000 7f5760ac6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 0x561de89cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.619+0000 7f5760ac6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c3db0 0x561de89cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.619+0000 7f5760ac6b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c4b10 0x561de89d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.619+0000 7f5760ac6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561de88b6880 con 0x561de881f6b0 2023-11-16T16:49:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.619+0000 7f5760ac6b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561de88b6520 con 0x561de89c4b10 2023-11-16T16:49:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.619+0000 7f5760ac6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561de87988c0 con 0x561de89c3db0 2023-11-16T16:49:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.619+0000 7f5760ac6b80 10 client.0 _ll_register_callbacks cb 0x561de88d4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.620+0000 7f574759e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 0x561de89cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.620+0000 7f574759e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 0x561de89cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59644/0 (socket says 172.21.15.93:59644) 2023-11-16T16:49:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.620+0000 7f574759e700 1 -- 172.21.15.93:0/2500524078 learned_addr learned my addr 172.21.15.93:0/2500524078 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.620+0000 7f574759e700 1 --2- 172.21.15.93:0/2500524078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c4b10 0x561de89d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.620+0000 7f574759e700 1 -- 172.21.15.93:0/2500524078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c3db0 msgr2=0x561de89cd200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:49:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.620+0000 7f574759e700 1 --2- 172.21.15.93:0/2500524078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c3db0 0x561de89cd200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.620+0000 7f574759e700 1 -- 172.21.15.93:0/2500524078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c4b10 msgr2=0x561de89d4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.620+0000 7f574759e700 1 --2- 172.21.15.93:0/2500524078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c4b10 0x561de89d4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.620+0000 7f574759e700 1 -- 172.21.15.93:0/2500524078 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5740011310 con 0x561de881f6b0 2023-11-16T16:49:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.621+0000 7f574759e700 1 --2- 172.21.15.93:0/2500524078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 0x561de89cccc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5740009ec0 tx=0x7f5740006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720c9f726d3f6bd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.621+0000 7f572aff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:49:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.621+0000 7f572aff5700 1 -- 172.21.15.93:0/2500524078 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574000e040 con 0x561de881f6b0 2023-11-16T16:49:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.621+0000 7f572aff5700 1 -- 172.21.15.93:0/2500524078 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f5740007500 con 0x561de881f6b0 2023-11-16T16:49:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.621+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2500524078 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561de89b6710 con 0x561de881f6b0 2023-11-16T16:49:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.621+0000 7f572aff5700 1 -- 172.21.15.93:0/2500524078 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57400076c0 con 0x561de881f6b0 2023-11-16T16:49:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.621+0000 7f5760ac6b80 10 client.15378 fetch_fsmap learned FSMap version 23 2023-11-16T16:49:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.621+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2500524078 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561de88db130 con 0x561de881f6b0 2023-11-16T16:49:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.622+0000 7f572aff5700 1 -- 172.21.15.93:0/2500524078 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561de89b6710 con 0x561de881f6b0 2023-11-16T16:49:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.622+0000 7f572aff5700 1 -- 172.21.15.93:0/2500524078 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f57400157f0 con 0x561de881f6b0 2023-11-16T16:49:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.622+0000 7f5760ac6b80 10 client.15378 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:49:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.622+0000 7f5760ac6b80 10 client.15378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.622+0000 7f5760ac6b80 20 client.15378 populate_metadata read hostname 'smithi093' 2023-11-16T16:49:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.622+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2500524078 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561de89d8930 0x561de89dad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.622+0000 7f5760ac6b80 4 client.15378 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:49:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.622+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2500524078 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561de88d7040 con 0x561de89d8930 2023-11-16T16:49:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.623+0000 7f574659c700 1 --2- 172.21.15.93:0/2500524078 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561de89d8930 0x561de89dad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.623+0000 7f574659c700 1 --2- 172.21.15.93:0/2500524078 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561de89d8930 0x561de89dad40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aa402a820c55b30 server_cookie=562ab198c13c90c in_seq=0 out_seq=0 2023-11-16T16:49:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.623+0000 7f572aff5700 10 client.15378 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.624+0000 7f572aff5700 1 -- 172.21.15.93:0/2500524078 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561de88d7040 con 0x561de89d8930 2023-11-16T16:49:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.624+0000 7f572aff5700 10 client.15378 handle_command_reply: tid=0 2023-11-16T16:49:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.624+0000 7f5760ac6b80 1 client.15378 shutdown 2023-11-16T16:49:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.624+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2500524078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 msgr2=0x561de89cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.624+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2500524078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 0x561de89cccc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5740009ec0 tx=0x7f5740006000 comp rx=0 tx=0).stop 2023-11-16T16:49:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.625+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2500524078 shutdown_connections 2023-11-16T16:49:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.625+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2500524078 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561de89d8930 0x561de89dad40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.625+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2500524078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de881f6b0 0x561de89cccc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.625+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2500524078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c4b10 0x561de89d4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.625+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2500524078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c3db0 0x561de89cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.625+0000 7f572aff5700 0 client.15378 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.625+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2500524078 >> 172.21.15.93:0/2500524078 conn(0x561de89b8f50 msgr2=0x561de89bd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.625+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2500524078 shutdown_connections 2023-11-16T16:49:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.625+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2500524078 wait complete. 2023-11-16T16:49:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.625+0000 7f5760ac6b80 20 client.15378 trim_cache size 0 max 16384 2023-11-16T16:49:41.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:41 smithi093 ceph-mon[97863]: pgmap v377: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 8 op/s 2023-11-16T16:49:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.654+0000 7f5760ac6b80 1 Processor -- start 2023-11-16T16:49:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.654+0000 7f5760ac6b80 1 -- start start 2023-11-16T16:49:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.655+0000 7f5760ac6b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de881f6b0 0x561de89c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.655+0000 7f5760ac6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de89c3db0 0x561de89d4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.655+0000 7f5760ac6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c4b10 0x561de89cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.655+0000 7f5760ac6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561de89befa0 con 0x561de89c3db0 2023-11-16T16:49:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.655+0000 7f5760ac6b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561de89caca0 con 0x561de881f6b0 2023-11-16T16:49:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.655+0000 7f5760ac6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561de89c8590 con 0x561de89c4b10 2023-11-16T16:49:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.655+0000 7f574759e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de881f6b0 0x561de89c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.655+0000 7f574659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de89c3db0 0x561de89d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.655+0000 7f574759e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de881f6b0 0x561de89c4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:54398/0 (socket says 172.21.15.93:54398) 2023-11-16T16:49:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.655+0000 7f574759e700 1 -- 172.21.15.93:0/2747809622 learned_addr learned my addr 172.21.15.93:0/2747809622 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.656+0000 7f574759e700 1 --2- 172.21.15.93:0/2747809622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c4b10 0x561de89cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.656+0000 7f574759e700 1 -- 172.21.15.93:0/2747809622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c4b10 msgr2=0x561de89cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.656+0000 7f574759e700 1 --2- 172.21.15.93:0/2747809622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c4b10 0x561de89cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.656+0000 7f574759e700 1 -- 172.21.15.93:0/2747809622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de89c3db0 msgr2=0x561de89d4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.656+0000 7f574759e700 1 --2- 172.21.15.93:0/2747809622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de89c3db0 0x561de89d4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.656+0000 7f574759e700 1 -- 172.21.15.93:0/2747809622 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561de88dae80 con 0x561de881f6b0 2023-11-16T16:49:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.656+0000 7f574759e700 1 --2- 172.21.15.93:0/2747809622 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de881f6b0 0x561de89c4ee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5740022040 tx=0x7f5740017040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b5d13cf4a0ba8ac server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.656+0000 7f572dffb700 1 -- 172.21.15.93:0/2747809622 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57400041f0 con 0x561de881f6b0 2023-11-16T16:49:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.656+0000 7f572dffb700 1 -- 172.21.15.93:0/2747809622 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5740015450 con 0x561de881f6b0 2023-11-16T16:49:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.656+0000 7f572dffb700 1 -- 172.21.15.93:0/2747809622 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740007880 con 0x561de881f6b0 2023-11-16T16:49:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.657+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2747809622 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de881f6b0 msgr2=0x561de89c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.657+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2747809622 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de881f6b0 0x561de89c4ee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5740022040 tx=0x7f5740017040 comp rx=0 tx=0).stop 2023-11-16T16:49:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.657+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2747809622 shutdown_connections 2023-11-16T16:49:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.657+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2747809622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de89c4b10 0x561de89cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.657+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2747809622 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de881f6b0 0x561de89c4ee0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.657+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2747809622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de89c3db0 0x561de89d4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.657+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2747809622 >> 172.21.15.93:0/2747809622 conn(0x561de89b8f50 msgr2=0x561de89df6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.658+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2747809622 shutdown_connections 2023-11-16T16:49:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.658+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2747809622 wait complete. 2023-11-16T16:49:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.658+0000 7f5760ac6b80 1 -- start start 2023-11-16T16:49:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.659+0000 7f5760ac6b80 1 Processor -- start 2023-11-16T16:49:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.659+0000 7f5760ac6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de881f6b0 0x561de89cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.659+0000 7f5760ac6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de89c3db0 0x561de89cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.659+0000 7f5760ac6b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c4b10 0x561de89d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.659+0000 7f5760ac6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561de89d32d0 con 0x561de89c3db0 2023-11-16T16:49:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.659+0000 7f5760ac6b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561de89d3440 con 0x561de89c4b10 2023-11-16T16:49:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.659+0000 7f5760ac6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561de89d3600 con 0x561de881f6b0 2023-11-16T16:49:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.659+0000 7f5760ac6b80 10 client.0 _ll_register_callbacks cb 0x7f57400097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.660+0000 7f574759e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c4b10 0x561de89d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.660+0000 7f574759e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c4b10 0x561de89d2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:54402/0 (socket says 172.21.15.93:54402) 2023-11-16T16:49:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.660+0000 7f574759e700 1 -- 172.21.15.93:0/2637733835 learned_addr learned my addr 172.21.15.93:0/2637733835 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.660+0000 7f574759e700 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de881f6b0 0x561de89cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.660+0000 7f574659c700 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de89c3db0 0x561de89cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.660+0000 7f574759e700 1 -- 172.21.15.93:0/2637733835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de881f6b0 msgr2=0x561de89cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.660+0000 7f574759e700 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de881f6b0 0x561de89cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.660+0000 7f574759e700 1 -- 172.21.15.93:0/2637733835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de89c3db0 msgr2=0x561de89cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.660+0000 7f574759e700 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de89c3db0 0x561de89cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.660+0000 7f574759e700 1 -- 172.21.15.93:0/2637733835 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561de88db130 con 0x561de89c4b10 2023-11-16T16:49:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.660+0000 7f574759e700 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c4b10 0x561de89d2d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5740011f50 tx=0x7f5740006130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b636ac2125e56ca2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.660+0000 7f572aff5700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:49:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.661+0000 7f572aff5700 1 -- 172.21.15.93:0/2637733835 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574003e070 con 0x561de89c4b10 2023-11-16T16:49:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.661+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2637733835 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f574000f040 con 0x561de89c4b10 2023-11-16T16:49:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.661+0000 7f572aff5700 1 -- 172.21.15.93:0/2637733835 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f57400041f0 con 0x561de89c4b10 2023-11-16T16:49:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.661+0000 7f572aff5700 1 -- 172.21.15.93:0/2637733835 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57400099e0 con 0x561de89c4b10 2023-11-16T16:49:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.661+0000 7f5760ac6b80 10 client.25421 fetch_fsmap learned FSMap version 23 2023-11-16T16:49:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.661+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2637733835 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561de872bc00 con 0x561de89c4b10 2023-11-16T16:49:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.662+0000 7f572aff5700 1 -- 172.21.15.93:0/2637733835 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f574000f040 con 0x561de89c4b10 2023-11-16T16:49:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.662+0000 7f572aff5700 1 -- 172.21.15.93:0/2637733835 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5740044020 con 0x561de89c4b10 2023-11-16T16:49:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.662+0000 7f5760ac6b80 10 client.25421 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:49:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.662+0000 7f5760ac6b80 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.662+0000 7f5760ac6b80 20 client.25421 populate_metadata read hostname 'smithi093' 2023-11-16T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.662+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561de8a2cb90 0x561de8a2ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.662+0000 7f5760ac6b80 4 client.25421 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.662+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2637733835 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561de88e3600 con 0x561de8a2cb90 2023-11-16T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.663+0000 7f574659c700 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561de8a2cb90 0x561de8a2ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.663+0000 7f574659c700 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561de8a2cb90 0x561de8a2ef70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663d6c08fc0b86b8 server_cookie=6f403bb0871e4ce1 in_seq=0 out_seq=0 2023-11-16T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.663+0000 7f572aff5700 10 client.25421 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.667+0000 7f572aff5700 1 -- 172.21.15.93:0/2637733835 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561de88e3600 con 0x561de8a2cb90 2023-11-16T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.667+0000 7f572aff5700 10 client.25421 handle_command_reply: tid=0 2023-11-16T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.667+0000 7f5760ac6b80 1 client.25421 shutdown 2023-11-16T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.668+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2637733835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c4b10 msgr2=0x561de89d2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.668+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c4b10 0x561de89d2d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5740011f50 tx=0x7f5740006130 comp rx=0 tx=0).stop 2023-11-16T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.668+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2637733835 shutdown_connections 2023-11-16T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.668+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x561de8a2cb90 0x561de8a2ef70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.668+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561de89c4b10 0x561de89d2d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.668+0000 7f572aff5700 0 client.25421 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.668+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561de881f6b0 0x561de89cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.668+0000 7f5760ac6b80 1 --2- 172.21.15.93:0/2637733835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561de89c3db0 0x561de89cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.668+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2637733835 >> 172.21.15.93:0/2637733835 conn(0x561de89b8f50 msgr2=0x561de89cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.668+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2637733835 shutdown_connections 2023-11-16T16:49:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.668+0000 7f5760ac6b80 1 -- 172.21.15.93:0/2637733835 wait complete. 2023-11-16T16:49:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.668+0000 7f5760ac6b80 20 client.25421 trim_cache size 0 max 16384 2023-11-16T16:49:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:49:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:49:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:49:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:49:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.669+0000 7f574d1ff700 1 -- 172.21.15.93:0/268210271 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f57300a1150 msgr2=0x7f57300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.669+0000 7f574d1ff700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f57300a1150 0x7f57300a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5738096000 tx=0x7f5738010cd0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.669+0000 7f574d1ff700 1 -- 172.21.15.93:0/268210271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5748100cc0 msgr2=0x7f5748100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.669+0000 7f574d1ff700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5748100cc0 0x7f5748100630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f573800d950 tx=0x7f573800dc60 comp rx=0 tx=0).stop 2023-11-16T16:49:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.669+0000 7f574d1ff700 1 -- 172.21.15.93:0/268210271 shutdown_connections 2023-11-16T16:49:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.669+0000 7f574d1ff700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f57300a1150 0x7f57300a3600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.669+0000 7f574d1ff700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5748105f00 0x7f57480ff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.669+0000 7f574d1ff700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5748100cc0 0x7f5748100630 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.670+0000 7f574d1ff700 1 --2- 172.21.15.93:0/268210271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57481035e0 0x7f57480fec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.670+0000 7f574d1ff700 1 -- 172.21.15.93:0/268210271 >> 172.21.15.93:0/268210271 conn(0x7f57480fa720 msgr2=0x7f5748101920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.670+0000 7f574d1ff700 1 -- 172.21.15.93:0/268210271 shutdown_connections 2023-11-16T16:49:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:41.670+0000 7f574d1ff700 1 -- 172.21.15.93:0/268210271 wait complete. 2023-11-16T16:49:41.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:41 smithi159 ceph-mon[131528]: pgmap v377: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 8 op/s 2023-11-16T16:49:41.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:41 smithi125 ceph-mon[131076]: pgmap v377: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 8 op/s 2023-11-16T16:49:43.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:43 smithi093 ceph-mon[97863]: pgmap v378: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-16T16:49:43.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:43 smithi159 ceph-mon[131528]: pgmap v378: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-16T16:49:44.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:43 smithi125 ceph-mon[131076]: pgmap v378: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-16T16:49:45.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:45 smithi093 ceph-mon[97863]: pgmap v379: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-16T16:49:45.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:45 smithi159 ceph-mon[131528]: pgmap v379: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-16T16:49:46.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:45 smithi125 ceph-mon[131076]: pgmap v379: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-16T16:49:47.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:47 smithi093 ceph-mon[97863]: pgmap v380: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:49:47.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:47 smithi159 ceph-mon[131528]: pgmap v380: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:49:48.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:47 smithi125 ceph-mon[131076]: pgmap v380: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:49:49.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:49 smithi093 ceph-mon[97863]: pgmap v381: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-16T16:49:49.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:49 smithi159 ceph-mon[131528]: pgmap v381: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-16T16:49:50.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:49 smithi125 ceph-mon[131076]: pgmap v381: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-16T16:49:51.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:51 smithi093 ceph-mon[97863]: pgmap v382: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-16T16:49:51.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:51 smithi159 ceph-mon[131528]: pgmap v382: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-16T16:49:51.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:51 smithi125 ceph-mon[131076]: pgmap v382: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-16T16:49:52.943 INFO:tasks.workunit.client.0.smithi093.stderr:Updating files: 65% (7852/12074) Updating files: 66% (7969/12074) Updating files: 67% (8090/12074) Updating files: 68% (8211/12074) Updating files: 69% (8332/12074) Updating files: 70% (8452/12074) Updating files: 71% (8573/12074) Updating files: 72% (8694/12074) Updating files: 73% (8815/12074) Updating files: 74% (8935/12074) Updating files: 75% (9056/12074) Updating files: 76% (9177/12074) Updating files: 77% (9297/12074) Updating files: 78% (9418/12074) Updating files: 79% (9539/12074) Updating files: 80% (9660/12074) Updating files: 81% (9780/12074) Updating files: 82% (9901/12074) Updating files: 83% (10022/12074) Updating files: 84% (10143/12074) Updating files: 85% (10263/12074) Updating files: 86% (10384/12074) Updating files: 87% (10505/12074) Updating files: 88% (10626/12074) Updating files: 89% (10746/12074) Updating files: 90% (10867/12074) Updating files: 91% (10988/12074) Updating files: 92% (11109/12074) Updating files: 93% (11229/12074) Updating files: 94% (11350/12074) Updating files: 95% (11471/12074) Updating files: 96% (11592/12074) Updating files: 97% (11712/12074) Updating files: 98% (11833/12074) Updating files: 99% (11954/12074) Updating files: 100% (12074/12074) Updating files: 100% (12074/12074), done. 2023-11-16T16:49:53.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:53 smithi093 ceph-mon[97863]: pgmap v383: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:49:53.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:53 smithi159 ceph-mon[131528]: pgmap v383: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:49:54.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:53 smithi125 ceph-mon[131076]: pgmap v383: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:49:54.108 INFO:tasks.workunit.client.0.smithi093.stderr:Note: switching to '5e4a743ad7fd1422ebd6c4dc950a21a303279207'. 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr:changes and commit them, and you can discard any commits you make in this 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr:state without impacting any branches by switching back to a branch. 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr:If you want to create a new branch to retain commits you create, you may 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr:do so (now or later) by using -c with the switch command. Example: 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr: git switch -c 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr:Or undo this operation with: 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-11-16T16:49:54.109 INFO:tasks.workunit.client.0.smithi093.stderr: git switch - 2023-11-16T16:49:54.110 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-11-16T16:49:54.110 INFO:tasks.workunit.client.0.smithi093.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-11-16T16:49:54.110 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-11-16T16:49:54.110 INFO:tasks.workunit.client.0.smithi093.stderr:HEAD is now at 5e4a743ad7f doc/cephfs: Update docs to match remove functionality and respective vxattrs 2023-11-16T16:49:54.127 DEBUG:teuthology.orchestra.run.smithi093:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-11-16T16:49:54.221 INFO:tasks.workunit.client.0.smithi093.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-11-16T16:49:54.226 INFO:tasks.workunit.client.0.smithi093.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-16T16:49:54.226 INFO:tasks.workunit.client.0.smithi093.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-11-16T16:49:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.827+0000 7f4bad3ff700 1 -- 172.21.15.93:0/1584090655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ba8100240 msgr2=0x7f4ba8100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.827+0000 7f4bad3ff700 1 --2- 172.21.15.93:0/1584090655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ba8100240 0x7f4ba8100650 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009990 tx=0x7f4b98009ca0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.827+0000 7f4bad3ff700 1 -- 172.21.15.93:0/1584090655 shutdown_connections 2023-11-16T16:49:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.827+0000 7f4bad3ff700 1 --2- 172.21.15.93:0/1584090655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ba8100240 0x7f4ba8100650 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.827+0000 7f4bad3ff700 1 --2- 172.21.15.93:0/1584090655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ba8101dc0 0x7f4ba8108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.827+0000 7f4bad3ff700 1 --2- 172.21.15.93:0/1584090655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ba8101430 0x7f4ba8101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.827+0000 7f4bad3ff700 1 -- 172.21.15.93:0/1584090655 >> 172.21.15.93:0/1584090655 conn(0x7f4ba80fb840 msgr2=0x7f4ba80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.827+0000 7f4bad3ff700 1 -- 172.21.15.93:0/1584090655 shutdown_connections 2023-11-16T16:49:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.827+0000 7f4bad3ff700 1 -- 172.21.15.93:0/1584090655 wait complete. 2023-11-16T16:49:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.828+0000 7f4bad3ff700 1 Processor -- start 2023-11-16T16:49:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.828+0000 7f4bad3ff700 1 -- start start 2023-11-16T16:49:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.828+0000 7f4bad3ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ba8100240 0x7f4ba80748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.828+0000 7f4bad3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ba8101430 0x7f4ba8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.828+0000 7f4bad3ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ba8101dc0 0x7f4ba8073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.828+0000 7f4bad3ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8073bc0 con 0x7f4ba8101dc0 2023-11-16T16:49:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.828+0000 7f4bad3ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8073d30 con 0x7f4ba8100240 2023-11-16T16:49:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.828+0000 7f4bad3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba81a00b0 con 0x7f4ba8101430 2023-11-16T16:49:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.829+0000 7f4ba77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ba8101dc0 0x7f4ba8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.829+0000 7f4ba6ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ba8100240 0x7f4ba80748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.829+0000 7f4ba6ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ba8100240 0x7f4ba80748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:50224/0 (socket says 172.21.15.93:50224) 2023-11-16T16:49:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.829+0000 7f4ba6ffd700 1 -- 172.21.15.93:0/3470424465 learned_addr learned my addr 172.21.15.93:0/3470424465 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.829+0000 7f4ba77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ba8101dc0 0x7f4ba8073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59822/0 (socket says 172.21.15.93:59822) 2023-11-16T16:49:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.829+0000 7f4ba67fc700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ba8101430 0x7f4ba8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.829+0000 7f4ba6ffd700 1 -- 172.21.15.93:0/3470424465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ba8101430 msgr2=0x7f4ba8072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.829+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ba8101430 0x7f4ba8072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.829+0000 7f4ba6ffd700 1 -- 172.21.15.93:0/3470424465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ba8101dc0 msgr2=0x7f4ba8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.829+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ba8101dc0 0x7f4ba8073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.829+0000 7f4ba6ffd700 1 -- 172.21.15.93:0/3470424465 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b980095f0 con 0x7f4ba8100240 2023-11-16T16:49:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.829+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ba8100240 0x7f4ba80748a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4b98005740 tx=0x7f4b98005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.830+0000 7f4b8ffff700 1 -- 172.21.15.93:0/3470424465 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9801d070 con 0x7f4ba8100240 2023-11-16T16:49:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.830+0000 7f4b8ffff700 1 -- 172.21.15.93:0/3470424465 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b98021410 con 0x7f4ba8100240 2023-11-16T16:49:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.830+0000 7f4bad3ff700 1 -- 172.21.15.93:0/3470424465 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba810bbf0 con 0x7f4ba8100240 2023-11-16T16:49:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.830+0000 7f4b8ffff700 1 -- 172.21.15.93:0/3470424465 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98017650 con 0x7f4ba8100240 2023-11-16T16:49:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.830+0000 7f4bad3ff700 1 -- 172.21.15.93:0/3470424465 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba810ba60 con 0x7f4ba8100240 2023-11-16T16:49:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.833+0000 7f4b8ffff700 1 -- 172.21.15.93:0/3470424465 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4b98021580 con 0x7f4ba8100240 2023-11-16T16:49:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.834+0000 7f4bc0d05b80 1 Processor -- start 2023-11-16T16:49:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.834+0000 7f4b8ffff700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4b900a1150 0x7f4b900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.834+0000 7f4bc0d05b80 1 -- start start 2023-11-16T16:49:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.834+0000 7f4b8ffff700 1 -- 172.21.15.93:0/3470424465 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f4b980cde60 con 0x7f4ba8100240 2023-11-16T16:49:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.834+0000 7f4ba77fe700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4b900a1150 0x7f4b900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.834+0000 7f4bc0d05b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601314aa6b0 0x56013164e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.834+0000 7f4bc0d05b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 0x56013164f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.834+0000 7f4bc0d05b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164fb10 0x56013164ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.834+0000 7f4bc0d05b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560131554000 con 0x5601314aa6b0 2023-11-16T16:49:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.834+0000 7f4bc0d05b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601315a6840 con 0x56013164edb0 2023-11-16T16:49:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.834+0000 7f4bc0d05b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013159fcd0 con 0x56013164fb10 2023-11-16T16:49:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601314aa6b0 0x56013164e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba67fc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 0x56013164f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba67fc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 0x56013164f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:50226/0 (socket says 172.21.15.93:50226) 2023-11-16T16:49:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba67fc700 1 -- 172.21.15.93:0/11426908 learned_addr learned my addr 172.21.15.93:0/11426908 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba77fe700 1 --2- 172.21.15.93:0/11426908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164fb10 0x56013164ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba6ffd700 1 -- 172.21.15.93:0/11426908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164fb10 msgr2=0x56013164ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/11426908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164fb10 0x56013164ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba6ffd700 1 -- 172.21.15.93:0/11426908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 msgr2=0x56013164f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba77fe700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4b900a1150 0x7f4b900a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8074f00 tx=0x7f4ba000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/11426908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 0x56013164f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba6ffd700 1 -- 172.21.15.93:0/11426908 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601314215d0 con 0x5601314aa6b0 2023-11-16T16:49:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba67fc700 1 --2- 172.21.15.93:0/11426908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 0x56013164f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:49:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.835+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/11426908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601314aa6b0 0x56013164e870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4b9809acf0 tx=0x7f4b9809adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7325a3bb8cdfc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.836+0000 7f4b8d7fa700 1 -- 172.21.15.93:0/11426908 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9809a950 con 0x5601314aa6b0 2023-11-16T16:49:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.836+0000 7f4b8d7fa700 1 -- 172.21.15.93:0/11426908 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b9809aff0 con 0x5601314aa6b0 2023-11-16T16:49:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.836+0000 7f4b8d7fa700 1 -- 172.21.15.93:0/11426908 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9809a740 con 0x5601314aa6b0 2023-11-16T16:49:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.836+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/11426908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601314aa6b0 msgr2=0x56013164e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.836+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/11426908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601314aa6b0 0x56013164e870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4b9809acf0 tx=0x7f4b9809adc0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.836+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/11426908 shutdown_connections 2023-11-16T16:49:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.836+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/11426908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601314aa6b0 0x56013164e870 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.836+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/11426908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164fb10 0x56013164ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.836+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/11426908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 0x56013164f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.836+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/11426908 >> 172.21.15.93:0/11426908 conn(0x560131643f50 msgr2=0x560131649100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.837+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/11426908 shutdown_connections 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.837+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/11426908 wait complete. 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.837+0000 7f4bc0d05b80 1 -- start start 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.837+0000 7f4bc0d05b80 1 Processor -- start 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4bc0d05b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601314aa6b0 0x560131657950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4bc0d05b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164edb0 0x560131657e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4bc0d05b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013165eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4bc0d05b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560131541880 con 0x56013164fb10 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4bc0d05b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560131541520 con 0x5601314aa6b0 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4bc0d05b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601314238c0 con 0x56013164edb0 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4bc0d05b80 10 client.0 _ll_register_callbacks cb 0x56013155f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4ba77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013165eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4ba77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013165eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59844/0 (socket says 172.21.15.93:59844) 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4ba77fe700 1 -- 172.21.15.93:0/2451972235 learned_addr learned my addr 172.21.15.93:0/2451972235 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4ba67fc700 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164edb0 0x560131657e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601314aa6b0 0x560131657950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4ba77fe700 1 -- 172.21.15.93:0/2451972235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164edb0 msgr2=0x560131657e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4ba77fe700 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164edb0 0x560131657e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4ba77fe700 1 -- 172.21.15.93:0/2451972235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601314aa6b0 msgr2=0x560131657950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.838+0000 7f4ba77fe700 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601314aa6b0 0x560131657950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.839+0000 7f4ba77fe700 1 -- 172.21.15.93:0/2451972235 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba810ba60 con 0x56013164fb10 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.839+0000 7f4ba77fe700 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013165eed0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000f400 tx=0x7f4ba0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d6c8ebeee93c6f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.839+0000 7f4b8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.839+0000 7f4b8aff5700 1 -- 172.21.15.93:0/2451972235 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000e040 con 0x56013164fb10 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.839+0000 7f4b8aff5700 1 -- 172.21.15.93:0/2451972235 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f4ba001c040 con 0x56013164fb10 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.839+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2451972235 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560131641710 con 0x56013164fb10 2023-11-16T16:49:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.839+0000 7f4b8aff5700 1 -- 172.21.15.93:0/2451972235 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ba0017aa0 con 0x56013164fb10 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.839+0000 7f4bc0d05b80 10 client.15396 fetch_fsmap learned FSMap version 23 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.839+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2451972235 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560131566130 con 0x56013164fb10 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.840+0000 7f4b8aff5700 1 -- 172.21.15.93:0/2451972235 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560131641710 con 0x56013164fb10 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.840+0000 7f4b8aff5700 1 -- 172.21.15.93:0/2451972235 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4ba0015410 con 0x56013164fb10 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.840+0000 7f4bc0d05b80 10 client.15396 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.840+0000 7f4bc0d05b80 10 client.15396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.840+0000 7f4bc0d05b80 20 client.15396 populate_metadata read hostname 'smithi093' 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.840+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5601316637e0 0x560131665bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.840+0000 7f4bc0d05b80 4 client.15396 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.840+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2451972235 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560131562040 con 0x5601316637e0 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.840+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5601316637e0 0x560131665bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.841+0000 7f4b8aff5700 10 client.15396 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.841+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5601316637e0 0x560131665bf0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d60adecf72b99a server_cookie=d693af8b99888d6a in_seq=0 out_seq=0 2023-11-16T16:49:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.841+0000 7f4b8aff5700 1 -- 172.21.15.93:0/2451972235 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560131562040 con 0x5601316637e0 2023-11-16T16:49:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.841+0000 7f4b8aff5700 10 client.15396 handle_command_reply: tid=0 2023-11-16T16:49:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4bc0d05b80 1 client.15396 shutdown 2023-11-16T16:49:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2451972235 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 msgr2=0x56013165eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013165eed0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000f400 tx=0x7f4ba0004ad0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2451972235 shutdown_connections 2023-11-16T16:49:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5601316637e0 0x560131665bf0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013165eed0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4b8aff5700 0 client.15396 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601314aa6b0 0x560131657950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/2451972235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164edb0 0x560131657e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2451972235 >> 172.21.15.93:0/2451972235 conn(0x560131643f50 msgr2=0x5601316487e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2451972235 shutdown_connections 2023-11-16T16:49:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2451972235 wait complete. 2023-11-16T16:49:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.842+0000 7f4bc0d05b80 20 client.15396 trim_cache size 0 max 16384 2023-11-16T16:49:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.857+0000 7f4bc0d05b80 1 Processor -- start 2023-11-16T16:49:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.857+0000 7f4bc0d05b80 1 -- start start 2023-11-16T16:49:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.857+0000 7f4bc0d05b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601314aa6b0 0x56013164d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.857+0000 7f4bc0d05b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164edb0 0x56013165edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.857+0000 7f4bc0d05b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013164da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.857+0000 7f4bc0d05b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560131562040 con 0x56013164fb10 2023-11-16T16:49:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.857+0000 7f4bc0d05b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560131660bc0 con 0x5601314aa6b0 2023-11-16T16:49:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.857+0000 7f4bc0d05b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013164f420 con 0x56013164edb0 2023-11-16T16:49:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.858+0000 7f4ba77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013164da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.858+0000 7f4ba67fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164edb0 0x56013165edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.858+0000 7f4ba77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013164da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59860/0 (socket says 172.21.15.93:59860) 2023-11-16T16:49:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.858+0000 7f4ba77fe700 1 -- 172.21.15.93:0/657926492 learned_addr learned my addr 172.21.15.93:0/657926492 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.858+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/657926492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601314aa6b0 0x56013164d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.858+0000 7f4ba77fe700 1 -- 172.21.15.93:0/657926492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164edb0 msgr2=0x56013165edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.858+0000 7f4ba77fe700 1 --2- 172.21.15.93:0/657926492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164edb0 0x56013165edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.858+0000 7f4ba77fe700 1 -- 172.21.15.93:0/657926492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601314aa6b0 msgr2=0x56013164d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.858+0000 7f4ba77fe700 1 --2- 172.21.15.93:0/657926492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601314aa6b0 0x56013164d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.858+0000 7f4ba77fe700 1 -- 172.21.15.93:0/657926492 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560131565e80 con 0x56013164fb10 2023-11-16T16:49:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.858+0000 7f4ba77fe700 1 --2- 172.21.15.93:0/657926492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013164da90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0015990 tx=0x7f4ba0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371ce286b6454bdb server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.859+0000 7f4b8dffb700 1 -- 172.21.15.93:0/657926492 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000e040 con 0x56013164fb10 2023-11-16T16:49:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.859+0000 7f4b8dffb700 1 -- 172.21.15.93:0/657926492 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ba000f040 con 0x56013164fb10 2023-11-16T16:49:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.859+0000 7f4b8dffb700 1 -- 172.21.15.93:0/657926492 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0017ac0 con 0x56013164fb10 2023-11-16T16:49:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.859+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/657926492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 msgr2=0x56013164da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.859+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/657926492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013164da90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0015990 tx=0x7f4ba0009920 comp rx=0 tx=0).stop 2023-11-16T16:49:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.859+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/657926492 shutdown_connections 2023-11-16T16:49:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.859+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/657926492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013164da90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.859+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/657926492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601314aa6b0 0x56013164d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.859+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/657926492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56013164edb0 0x56013165edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.859+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/657926492 >> 172.21.15.93:0/657926492 conn(0x560131643f50 msgr2=0x560131657950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.859+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/657926492 shutdown_connections 2023-11-16T16:49:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.859+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/657926492 wait complete. 2023-11-16T16:49:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.860+0000 7f4bc0d05b80 1 -- start start 2023-11-16T16:49:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.860+0000 7f4bc0d05b80 1 Processor -- start 2023-11-16T16:49:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.860+0000 7f4bc0d05b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601314aa6b0 0x56013165abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.860+0000 7f4bc0d05b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 0x56013165b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.860+0000 7f4bc0d05b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013165e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.860+0000 7f4bc0d05b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013165e5d0 con 0x56013164fb10 2023-11-16T16:49:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.860+0000 7f4bc0d05b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013165e740 con 0x56013164edb0 2023-11-16T16:49:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.860+0000 7f4bc0d05b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013165e900 con 0x5601314aa6b0 2023-11-16T16:49:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4bc0d05b80 10 client.0 _ll_register_callbacks cb 0x7f4ba00094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4ba77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013165e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4ba67fc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 0x56013165b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4ba67fc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 0x56013165b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:50252/0 (socket says 172.21.15.93:50252) 2023-11-16T16:49:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4ba67fc700 1 -- 172.21.15.93:0/2556109715 learned_addr learned my addr 172.21.15.93:0/2556109715 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:49:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601314aa6b0 0x56013165abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4ba67fc700 1 -- 172.21.15.93:0/2556109715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601314aa6b0 msgr2=0x56013165abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4ba67fc700 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601314aa6b0 0x56013165abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4ba67fc700 1 -- 172.21.15.93:0/2556109715 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 msgr2=0x56013165e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4ba67fc700 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013165e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4ba67fc700 1 -- 172.21.15.93:0/2556109715 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560131566130 con 0x56013164edb0 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4ba77fe700 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013165e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4ba67fc700 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 0x56013165b110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c005c20 tx=0x7f4b9c012c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e74a08373d9e4447 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.861+0000 7f4b8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4b8aff5700 1 -- 172.21.15.93:0/2556109715 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c022070 con 0x56013164edb0 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2556109715 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b9809aff0 con 0x56013164edb0 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4b8aff5700 1 -- 172.21.15.93:0/2556109715 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f4b9c004020 con 0x56013164edb0 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4b8aff5700 1 -- 172.21.15.93:0/2556109715 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b9c017040 con 0x56013164edb0 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4bc0d05b80 10 client.25439 fetch_fsmap learned FSMap version 23 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2556109715 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5601313b6c00 con 0x56013164edb0 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4b8aff5700 1 -- 172.21.15.93:0/2556109715 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b9809aff0 con 0x56013164edb0 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4b8aff5700 1 -- 172.21.15.93:0/2556109715 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4b9c025030 con 0x56013164edb0 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4bc0d05b80 10 client.25439 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:49:54.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4bc0d05b80 10 client.25439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4bc0d05b80 20 client.25439 populate_metadata read hostname 'smithi093' 2023-11-16T16:49:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5601316b86d0 0x5601316baae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4bc0d05b80 4 client.25439 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:49:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.862+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2556109715 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56013156e600 con 0x5601316b86d0 2023-11-16T16:49:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.863+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5601316b86d0 0x5601316baae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.863+0000 7f4b8aff5700 10 client.25439 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.863+0000 7f4ba6ffd700 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5601316b86d0 0x5601316baae0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed994b862001a30 server_cookie=af93140c03c8b0c1 in_seq=0 out_seq=0 2023-11-16T16:49:54.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.867+0000 7f4b8aff5700 1 -- 172.21.15.93:0/2556109715 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56013156e600 con 0x5601316b86d0 2023-11-16T16:49:54.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.867+0000 7f4b8aff5700 10 client.25439 handle_command_reply: tid=0 2023-11-16T16:49:54.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.867+0000 7f4bc0d05b80 1 client.25439 shutdown 2023-11-16T16:49:54.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.867+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2556109715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 msgr2=0x56013165b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.867+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 0x56013165b110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c005c20 tx=0x7f4b9c012c30 comp rx=0 tx=0).stop 2023-11-16T16:49:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.867+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2556109715 shutdown_connections 2023-11-16T16:49:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.867+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5601316b86d0 0x5601316baae0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.867+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56013164fb10 0x56013165e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.867+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601314aa6b0 0x56013165abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.867+0000 7f4b8aff5700 0 client.25439 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:49:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.867+0000 7f4bc0d05b80 1 --2- 172.21.15.93:0/2556109715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56013164edb0 0x56013165b110 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.867+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2556109715 >> 172.21.15.93:0/2556109715 conn(0x560131643f50 msgr2=0x560131655d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2556109715 shutdown_connections 2023-11-16T16:49:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bc0d05b80 1 -- 172.21.15.93:0/2556109715 wait complete. 2023-11-16T16:49:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bc0d05b80 20 client.25439 trim_cache size 0 max 16384 2023-11-16T16:49:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:49:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:49:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:49:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:49:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bad3ff700 1 -- 172.21.15.93:0/3470424465 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4b900a1150 msgr2=0x7f4b900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bad3ff700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4b900a1150 0x7f4b900a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8074f00 tx=0x7f4ba000b040 comp rx=0 tx=0).stop 2023-11-16T16:49:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bad3ff700 1 -- 172.21.15.93:0/3470424465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ba8100240 msgr2=0x7f4ba80748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bad3ff700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ba8100240 0x7f4ba80748a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4b98005740 tx=0x7f4b98005770 comp rx=0 tx=0).stop 2023-11-16T16:49:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bad3ff700 1 -- 172.21.15.93:0/3470424465 shutdown_connections 2023-11-16T16:49:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bad3ff700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4b900a1150 0x7f4b900a3600 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bad3ff700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ba8101dc0 0x7f4ba8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bad3ff700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ba8100240 0x7f4ba80748a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bad3ff700 1 --2- 172.21.15.93:0/3470424465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ba8101430 0x7f4ba8072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.868+0000 7f4bad3ff700 1 -- 172.21.15.93:0/3470424465 >> 172.21.15.93:0/3470424465 conn(0x7f4ba80fb840 msgr2=0x7f4ba80fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.869+0000 7f4bad3ff700 1 -- 172.21.15.93:0/3470424465 shutdown_connections 2023-11-16T16:49:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:49:54.869+0000 7f4bad3ff700 1 -- 172.21.15.93:0/3470424465 wait complete. 2023-11-16T16:49:54.955 INFO:tasks.workunit.client.0.smithi093.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-11-16T16:49:55.044 INFO:tasks.workunit.client.0.smithi093.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-11-16T16:49:55.109 INFO:tasks.workunit.client.0.smithi093.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-16T16:49:55.113 INFO:tasks.workunit.client.0.smithi093.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-16T16:49:55.113 INFO:tasks.workunit.client.0.smithi093.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-11-16T16:49:55.177 INFO:tasks.workunit.client.0.smithi093.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-16T16:49:55.184 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:49:55.184 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-11-16T16:49:55.267 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2023-11-16T16:49:55.269 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2023-11-16T16:49:55.269 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2023-11-16T16:49:55.364 INFO:tasks.workunit.client.0.smithi093.stderr:+ set -e 2023-11-16T16:49:55.365 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir -p testdir 2023-11-16T16:49:55.368 INFO:tasks.workunit.client.0.smithi093.stderr:+ cd testdir 2023-11-16T16:49:55.369 INFO:tasks.workunit.client.0.smithi093.stderr:+ set +e 2023-11-16T16:49:55.369 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfacl -d -m u:nobody:rw . 2023-11-16T16:49:55.381 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-11-16T16:49:55.381 INFO:tasks.workunit.client.0.smithi093.stderr:+ set -e 2023-11-16T16:49:55.381 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=0 2023-11-16T16:49:55.381 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 0 -lt 100 ']' 2023-11-16T16:49:55.381 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 0 + 1 2023-11-16T16:49:55.383 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=1 2023-11-16T16:49:55.383 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:49:55.388 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:49:55.389 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:49:55.397 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:49:55.397 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:49:55.397 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:49:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:55 smithi093 ceph-mon[97863]: pgmap v384: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:49:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:55 smithi159 ceph-mon[131528]: pgmap v384: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:49:56.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:55 smithi125 ceph-mon[131076]: pgmap v384: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:49:57.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:57 smithi093 ceph-mon[97863]: pgmap v385: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:49:57.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:57 smithi159 ceph-mon[131528]: pgmap v385: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:49:58.058 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:49:58.058 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:49:58.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:57 smithi125 ceph-mon[131076]: pgmap v385: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:49:58.235 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:49:58.235 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:49:58.256 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:49:58.257 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:49:58.257 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 1 -lt 100 ']' 2023-11-16T16:49:58.257 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 1 + 1 2023-11-16T16:49:58.277 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=2 2023-11-16T16:49:58.277 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:49:58.311 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:49:58.311 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:49:58.314 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:49:58.314 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:49:58.315 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:49:58.816 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:49:58.816 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:49:58.950 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:49:58.950 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:49:58.964 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:49:58.964 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:49:58.964 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -lt 100 ']' 2023-11-16T16:49:58.965 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 2 + 1 2023-11-16T16:49:58.986 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=3 2023-11-16T16:49:58.986 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:49:59.016 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:49:59.016 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:49:59.019 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:49:59.019 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:49:59.019 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:49:59.441 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:49:59.441 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:49:59.568 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:49:59.568 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:49:59.581 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:49:59.581 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:49:59.582 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 3 -lt 100 ']' 2023-11-16T16:49:59.582 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 3 + 1 2023-11-16T16:49:59.602 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=4 2023-11-16T16:49:59.602 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:49:59.632 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:49:59.633 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:49:59.635 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:49:59.636 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:49:59.636 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:49:59.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:49:59 smithi093 ceph-mon[97863]: pgmap v386: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:49:59.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:49:59 smithi159 ceph-mon[131528]: pgmap v386: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:50:00.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:49:59 smithi125 ceph-mon[131076]: pgmap v386: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:50:00.107 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:00.108 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:00.311 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:00.311 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:00.322 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:00.323 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:00.323 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 4 -lt 100 ']' 2023-11-16T16:50:00.323 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 4 + 1 2023-11-16T16:50:00.365 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=5 2023-11-16T16:50:00.366 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:00.390 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:00.390 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:00.393 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:00.393 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:00.393 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:00.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:00 smithi125 ceph-mon[131076]: overall HEALTH_OK 2023-11-16T16:50:00.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:00 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:50:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:00 smithi093 ceph-mon[97863]: overall HEALTH_OK 2023-11-16T16:50:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:00 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:50:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:00 smithi159 ceph-mon[131528]: overall HEALTH_OK 2023-11-16T16:50:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:00 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:50:01.299 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:01.299 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:01.468 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:01.468 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:01.498 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:01.498 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:01.498 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 5 -lt 100 ']' 2023-11-16T16:50:01.499 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 5 + 1 2023-11-16T16:50:01.557 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=6 2023-11-16T16:50:01.557 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:01.634 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:01.635 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:01.637 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:01.638 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:01.638 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:01.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:01 smithi093 ceph-mon[97863]: pgmap v387: 129 pgs: 129 active+clean; 680 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-11-16T16:50:01.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:01 smithi159 ceph-mon[131528]: pgmap v387: 129 pgs: 129 active+clean; 680 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-11-16T16:50:02.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:01 smithi125 ceph-mon[131076]: pgmap v387: 129 pgs: 129 active+clean; 680 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-11-16T16:50:02.091 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:02.091 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:02.233 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:02.233 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:02.247 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:02.247 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:02.247 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 6 -lt 100 ']' 2023-11-16T16:50:02.248 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 6 + 1 2023-11-16T16:50:02.274 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=7 2023-11-16T16:50:02.274 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:02.326 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:02.327 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:02.329 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:02.330 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:02.330 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:03.257 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:03.258 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:03.417 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:03.417 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:03.448 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:03.448 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:03.448 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 7 -lt 100 ']' 2023-11-16T16:50:03.449 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 7 + 1 2023-11-16T16:50:03.482 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=8 2023-11-16T16:50:03.482 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:03.567 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:03.567 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:03.571 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:03.572 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:03.572 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:03.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:03 smithi093 ceph-mon[97863]: pgmap v388: 129 pgs: 129 active+clean; 680 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-11-16T16:50:03.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:03 smithi159 ceph-mon[131528]: pgmap v388: 129 pgs: 129 active+clean; 680 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-11-16T16:50:04.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:03 smithi125 ceph-mon[131076]: pgmap v388: 129 pgs: 129 active+clean; 680 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-11-16T16:50:04.165 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:04.165 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:04.384 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:04.384 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:04.422 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:04.422 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:04.422 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 8 -lt 100 ']' 2023-11-16T16:50:04.423 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 8 + 1 2023-11-16T16:50:04.460 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=9 2023-11-16T16:50:04.460 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:04.509 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:04.509 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:04.512 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:04.512 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:04.513 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:05.107 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:05.107 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:05.231 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:05.231 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:05.255 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:05.255 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:05.255 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 9 -lt 100 ']' 2023-11-16T16:50:05.255 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 9 + 1 2023-11-16T16:50:05.323 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=10 2023-11-16T16:50:05.323 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:05.334 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:05.335 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:05.337 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:05.338 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:05.338 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:06.040 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:06.040 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:06.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:05 smithi093 ceph-mon[97863]: pgmap v389: 129 pgs: 129 active+clean; 698 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2023-11-16T16:50:06.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:05 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:50:06.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:05 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:50:06.192 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:06.192 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:06.214 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:06.214 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:06.214 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 10 -lt 100 ']' 2023-11-16T16:50:06.214 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 10 + 1 2023-11-16T16:50:06.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:05 smithi159 ceph-mon[131528]: pgmap v389: 129 pgs: 129 active+clean; 698 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2023-11-16T16:50:06.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:05 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:50:06.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:05 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:50:06.240 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=11 2023-11-16T16:50:06.240 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:06.293 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:06.294 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:06.296 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:06.297 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:06.297 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:06.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:05 smithi125 ceph-mon[131076]: pgmap v389: 129 pgs: 129 active+clean; 698 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2023-11-16T16:50:06.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:05 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:50:06.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:05 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:50:06.881 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:06.882 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:07.057 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:07.057 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:07.072 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:07.072 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:07.072 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 11 -lt 100 ']' 2023-11-16T16:50:07.072 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 11 + 1 2023-11-16T16:50:07.114 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=12 2023-11-16T16:50:07.115 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:07.158 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:07.159 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:07.161 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:07.162 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:07.162 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:08.110 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:08.111 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:08.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:07 smithi093 ceph-mon[97863]: pgmap v390: 129 pgs: 129 active+clean; 698 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2023-11-16T16:50:08.216 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:08.216 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:08.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:07 smithi159 ceph-mon[131528]: pgmap v390: 129 pgs: 129 active+clean; 698 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2023-11-16T16:50:08.240 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:08.240 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:08.240 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 12 -lt 100 ']' 2023-11-16T16:50:08.240 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 12 + 1 2023-11-16T16:50:08.261 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=13 2023-11-16T16:50:08.261 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:08.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:07 smithi125 ceph-mon[131076]: pgmap v390: 129 pgs: 129 active+clean; 698 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2023-11-16T16:50:08.341 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:08.342 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:08.344 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:08.345 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:08.345 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:09.223 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:09.223 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:09.338 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:09.338 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:09.374 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:09.374 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:09.374 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 13 -lt 100 ']' 2023-11-16T16:50:09.374 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 13 + 1 2023-11-16T16:50:09.440 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=14 2023-11-16T16:50:09.440 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:09.515 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:09.516 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:09.518 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:09.519 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:09.519 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:10.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:09 smithi093 ceph-mon[97863]: pgmap v391: 129 pgs: 129 active+clean; 698 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2023-11-16T16:50:10.173 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:10.173 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:10.234 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:09 smithi159 ceph-mon[131528]: pgmap v391: 129 pgs: 129 active+clean; 698 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2023-11-16T16:50:10.289 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:10.289 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:10.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:09 smithi125 ceph-mon[131076]: pgmap v391: 129 pgs: 129 active+clean; 698 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2023-11-16T16:50:10.338 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:10.338 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:10.339 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 14 -lt 100 ']' 2023-11-16T16:50:10.339 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 14 + 1 2023-11-16T16:50:10.392 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=15 2023-11-16T16:50:10.392 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:10.426 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:10.426 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:10.429 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:10.429 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:10.430 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.733+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/985183472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a54073550 msgr2=0x7f4a540739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.733+0000 7f4a5b7ff700 1 --2- 172.21.15.93:0/985183472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a54073550 0x7f4a540739c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009990 tx=0x7f4a44009ca0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.733+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/985183472 shutdown_connections 2023-11-16T16:50:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.733+0000 7f4a5b7ff700 1 --2- 172.21.15.93:0/985183472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a54108100 0x7f4a5410a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.733+0000 7f4a5b7ff700 1 --2- 172.21.15.93:0/985183472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a54074b30 0x7f4a54072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.733+0000 7f4a5b7ff700 1 --2- 172.21.15.93:0/985183472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a54073550 0x7f4a540739c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.733+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/985183472 >> 172.21.15.93:0/985183472 conn(0x7f4a540fb840 msgr2=0x7f4a540fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.733+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/985183472 shutdown_connections 2023-11-16T16:50:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.733+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/985183472 wait complete. 2023-11-16T16:50:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.734+0000 7f4a5b7ff700 1 Processor -- start 2023-11-16T16:50:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.734+0000 7f4a5b7ff700 1 -- start start 2023-11-16T16:50:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a5b7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a54073550 0x7f4a54198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a5b7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a54074b30 0x7f4a54198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a5b7ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a54108100 0x7f4a5419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a5b7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5419fdb0 con 0x7f4a54074b30 2023-11-16T16:50:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a5b7ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5419fef0 con 0x7f4a54108100 2023-11-16T16:50:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a5b7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a541a0080 con 0x7f4a54073550 2023-11-16T16:50:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a5959b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a54073550 0x7f4a54198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a58d9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a54074b30 0x7f4a54198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a5959b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a54073550 0x7f4a54198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:46350/0 (socket says 172.21.15.93:46350) 2023-11-16T16:50:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a5959b700 1 -- 172.21.15.93:0/8063555 learned_addr learned my addr 172.21.15.93:0/8063555 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a59d9c700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a54108100 0x7f4a5419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a58d9a700 1 -- 172.21.15.93:0/8063555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a54073550 msgr2=0x7f4a54198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a54073550 0x7f4a54198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a58d9a700 1 -- 172.21.15.93:0/8063555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a54108100 msgr2=0x7f4a5419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a54108100 0x7f4a5419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a58d9a700 1 -- 172.21.15.93:0/8063555 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a440095f0 con 0x7f4a54074b30 2023-11-16T16:50:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.735+0000 7f4a5959b700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a54073550 0x7f4a54198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:50:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.736+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a54074b30 0x7f4a54198770 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4a44000c00 tx=0x7f4a4400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.736+0000 7f4a4a7fc700 1 -- 172.21.15.93:0/8063555 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4401d070 con 0x7f4a54074b30 2023-11-16T16:50:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.736+0000 7f4a4a7fc700 1 -- 172.21.15.93:0/8063555 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a44005050 con 0x7f4a54074b30 2023-11-16T16:50:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.736+0000 7f4a4a7fc700 1 -- 172.21.15.93:0/8063555 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44004a70 con 0x7f4a54074b30 2023-11-16T16:50:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.736+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/8063555 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5410de20 con 0x7f4a54074b30 2023-11-16T16:50:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.736+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/8063555 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5410dc90 con 0x7f4a54074b30 2023-11-16T16:50:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.739+0000 7f4a4a7fc700 1 -- 172.21.15.93:0/8063555 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4a440051c0 con 0x7f4a54074b30 2023-11-16T16:50:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.740+0000 7f4a4a7fc700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4a400a1150 0x7f4a400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.740+0000 7f4a4a7fc700 1 -- 172.21.15.93:0/8063555 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f4a440cdc80 con 0x7f4a54074b30 2023-11-16T16:50:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.741+0000 7f4a5959b700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4a400a1150 0x7f4a400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.741+0000 7f4a5959b700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4a400a1150 0x7f4a400a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4a500104e0 tx=0x7f4a5000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.808+0000 7f4a6f061b80 1 Processor -- start 2023-11-16T16:50:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.809+0000 7f4a6f061b80 1 -- start start 2023-11-16T16:50:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.809+0000 7f4a6f061b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b349166b0 0x560b34aba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.809+0000 7f4a6f061b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 0x560b34abb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.809+0000 7f4a6f061b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abbb10 0x560b34abbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.809+0000 7f4a6f061b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b349c0000 con 0x560b34abadb0 2023-11-16T16:50:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.809+0000 7f4a6f061b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b34a12840 con 0x560b34abbb10 2023-11-16T16:50:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.809+0000 7f4a6f061b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b34a0bcd0 con 0x560b349166b0 2023-11-16T16:50:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.809+0000 7f4a58d9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 0x560b34abb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.809+0000 7f4a58d9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 0x560b34abb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35898/0 (socket says 172.21.15.93:35898) 2023-11-16T16:50:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.809+0000 7f4a58d9a700 1 -- 172.21.15.93:0/1450182428 learned_addr learned my addr 172.21.15.93:0/1450182428 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.809+0000 7f4a5959b700 1 --2- 172.21.15.93:0/1450182428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b349166b0 0x560b34aba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.809+0000 7f4a59d9c700 1 --2- 172.21.15.93:0/1450182428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abbb10 0x560b34abbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.810+0000 7f4a58d9a700 1 -- 172.21.15.93:0/1450182428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b349166b0 msgr2=0x560b34aba870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.810+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/1450182428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b349166b0 0x560b34aba870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.810+0000 7f4a58d9a700 1 -- 172.21.15.93:0/1450182428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abbb10 msgr2=0x560b34abbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.810+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/1450182428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abbb10 0x560b34abbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.810+0000 7f4a58d9a700 1 -- 172.21.15.93:0/1450182428 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b3488d5d0 con 0x560b34abadb0 2023-11-16T16:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.810+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/1450182428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 0x560b34abb5d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4a440a2b80 tx=0x7f4a4409ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6a5ceb044d7881 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.810+0000 7f4a3f7fe700 1 -- 172.21.15.93:0/1450182428 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44099090 con 0x560b34abadb0 2023-11-16T16:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.810+0000 7f4a3f7fe700 1 -- 172.21.15.93:0/1450182428 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a44004c30 con 0x560b34abadb0 2023-11-16T16:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.810+0000 7f4a3f7fe700 1 -- 172.21.15.93:0/1450182428 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4409a440 con 0x560b34abadb0 2023-11-16T16:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.811+0000 7f4a6f061b80 1 -- 172.21.15.93:0/1450182428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 msgr2=0x560b34abb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.811+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/1450182428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 0x560b34abb5d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4a440a2b80 tx=0x7f4a4409ad70 comp rx=0 tx=0).stop 2023-11-16T16:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.811+0000 7f4a6f061b80 1 -- 172.21.15.93:0/1450182428 shutdown_connections 2023-11-16T16:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.811+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/1450182428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abbb10 0x560b34abbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.811+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/1450182428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b349166b0 0x560b34aba870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.811+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/1450182428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 0x560b34abb5d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.811+0000 7f4a6f061b80 1 -- 172.21.15.93:0/1450182428 >> 172.21.15.93:0/1450182428 conn(0x560b34aaff50 msgr2=0x560b34ab5100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.811+0000 7f4a6f061b80 1 -- 172.21.15.93:0/1450182428 shutdown_connections 2023-11-16T16:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.811+0000 7f4a6f061b80 1 -- 172.21.15.93:0/1450182428 wait complete. 2023-11-16T16:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.812+0000 7f4a6f061b80 1 -- start start 2023-11-16T16:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.812+0000 7f4a6f061b80 1 Processor -- start 2023-11-16T16:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.812+0000 7f4a6f061b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b349166b0 0x560b34ac38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.812+0000 7f4a6f061b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abadb0 0x560b34ac3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.812+0000 7f4a6f061b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abbb10 0x560b34acae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.812+0000 7f4a6f061b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b349ad880 con 0x560b34abbb10 2023-11-16T16:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.812+0000 7f4a6f061b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b349ad520 con 0x560b34abadb0 2023-11-16T16:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.812+0000 7f4a6f061b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3488f8c0 con 0x560b349166b0 2023-11-16T16:50:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.812+0000 7f4a6f061b80 10 client.0 _ll_register_callbacks cb 0x560b349cb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:50:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a59d9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abbb10 0x560b34acae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a59d9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abbb10 0x560b34acae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35904/0 (socket says 172.21.15.93:35904) 2023-11-16T16:50:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a59d9c700 1 -- 172.21.15.93:0/4235887797 learned_addr learned my addr 172.21.15.93:0/4235887797 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abadb0 0x560b34ac3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a5959b700 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b349166b0 0x560b34ac38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a59d9c700 1 -- 172.21.15.93:0/4235887797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b349166b0 msgr2=0x560b34ac38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a59d9c700 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b349166b0 0x560b34ac38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a59d9c700 1 -- 172.21.15.93:0/4235887797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abadb0 msgr2=0x560b34ac3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a59d9c700 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abadb0 0x560b34ac3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a59d9c700 1 -- 172.21.15.93:0/4235887797 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5410dc90 con 0x560b34abbb10 2023-11-16T16:50:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a59d9c700 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abbb10 0x560b34acae40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0042a0 tx=0x7f4a4c0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b11bb6106380e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:50:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.813+0000 7f4a3cff9700 1 -- 172.21.15.93:0/4235887797 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c007910 con 0x560b34abbb10 2023-11-16T16:50:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a6f061b80 1 -- 172.21.15.93:0/4235887797 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b34aad710 con 0x560b34abbb10 2023-11-16T16:50:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a3cff9700 1 -- 172.21.15.93:0/4235887797 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f4a4c007a70 con 0x560b34abbb10 2023-11-16T16:50:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a3cff9700 1 -- 172.21.15.93:0/4235887797 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a4c0048e0 con 0x560b34abbb10 2023-11-16T16:50:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a6f061b80 10 client.15414 fetch_fsmap learned FSMap version 23 2023-11-16T16:50:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a6f061b80 1 -- 172.21.15.93:0/4235887797 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560b349d2130 con 0x560b34abbb10 2023-11-16T16:50:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a3cff9700 1 -- 172.21.15.93:0/4235887797 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560b34aad710 con 0x560b34abbb10 2023-11-16T16:50:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a3cff9700 1 -- 172.21.15.93:0/4235887797 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a4c01e3b0 con 0x560b34abbb10 2023-11-16T16:50:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a6f061b80 10 client.15414 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:50:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a6f061b80 10 client.15414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:50:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a6f061b80 20 client.15414 populate_metadata read hostname 'smithi093' 2023-11-16T16:50:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x560b34acf770 0x560b34ad1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a6f061b80 4 client.15414 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:50:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.814+0000 7f4a6f061b80 1 -- 172.21.15.93:0/4235887797 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b349ce040 con 0x560b34acf770 2023-11-16T16:50:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.815+0000 7f4a5959b700 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x560b34acf770 0x560b34ad1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.815+0000 7f4a3cff9700 10 client.15414 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.815+0000 7f4a5959b700 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x560b34acf770 0x560b34ad1b80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bf946b3853ea582 server_cookie=40a1f4cb0f15ec93 in_seq=0 out_seq=0 2023-11-16T16:50:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.816+0000 7f4a3cff9700 1 -- 172.21.15.93:0/4235887797 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560b349ce040 con 0x560b34acf770 2023-11-16T16:50:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.816+0000 7f4a3cff9700 10 client.15414 handle_command_reply: tid=0 2023-11-16T16:50:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.816+0000 7f4a6f061b80 1 client.15414 shutdown 2023-11-16T16:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.816+0000 7f4a6f061b80 1 -- 172.21.15.93:0/4235887797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abbb10 msgr2=0x560b34acae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.816+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abbb10 0x560b34acae40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0042a0 tx=0x7f4a4c0042d0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.816+0000 7f4a6f061b80 1 -- 172.21.15.93:0/4235887797 shutdown_connections 2023-11-16T16:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.816+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x560b34acf770 0x560b34ad1b80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.816+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abbb10 0x560b34acae40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.816+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b349166b0 0x560b34ac38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.816+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/4235887797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abadb0 0x560b34ac3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.816+0000 7f4a3cff9700 0 client.15414 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.817+0000 7f4a6f061b80 1 -- 172.21.15.93:0/4235887797 >> 172.21.15.93:0/4235887797 conn(0x560b34aaff50 msgr2=0x560b34ab47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.817+0000 7f4a6f061b80 1 -- 172.21.15.93:0/4235887797 shutdown_connections 2023-11-16T16:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.817+0000 7f4a6f061b80 1 -- 172.21.15.93:0/4235887797 wait complete. 2023-11-16T16:50:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.817+0000 7f4a6f061b80 20 client.15414 trim_cache size 0 max 16384 2023-11-16T16:50:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.832+0000 7f4a6f061b80 1 Processor -- start 2023-11-16T16:50:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.832+0000 7f4a6f061b80 1 -- start start 2023-11-16T16:50:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.832+0000 7f4a6f061b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b349166b0 0x560b34ac1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.832+0000 7f4a6f061b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 0x560b34acad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.832+0000 7f4a6f061b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b34abbb10 0x560b34ac21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.832+0000 7f4a6f061b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b34ac2d70 con 0x560b34abadb0 2023-11-16T16:50:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.832+0000 7f4a6f061b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b34ac3430 con 0x560b349166b0 2023-11-16T16:50:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.832+0000 7f4a6f061b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b34aafb70 con 0x560b34abbb10 2023-11-16T16:50:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a58d9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 0x560b34acad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a58d9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 0x560b34acad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35914/0 (socket says 172.21.15.93:35914) 2023-11-16T16:50:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a58d9a700 1 -- 172.21.15.93:0/788939402 learned_addr learned my addr 172.21.15.93:0/788939402 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a59d9c700 1 --2- 172.21.15.93:0/788939402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b34abbb10 0x560b34ac21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a5959b700 1 --2- 172.21.15.93:0/788939402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b349166b0 0x560b34ac1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a58d9a700 1 -- 172.21.15.93:0/788939402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b34abbb10 msgr2=0x560b34ac21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/788939402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b34abbb10 0x560b34ac21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a58d9a700 1 -- 172.21.15.93:0/788939402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b349166b0 msgr2=0x560b34ac1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/788939402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b349166b0 0x560b34ac1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a58d9a700 1 -- 172.21.15.93:0/788939402 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b349d1e80 con 0x560b34abadb0 2023-11-16T16:50:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a5959b700 1 --2- 172.21.15.93:0/788939402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b349166b0 0x560b34ac1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:50:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/788939402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 0x560b34acad60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x560b34abc5b0 tx=0x7f4a4409abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbabd819af4599ce server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a3ffff700 1 -- 172.21.15.93:0/788939402 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44097030 con 0x560b34abadb0 2023-11-16T16:50:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a3ffff700 1 -- 172.21.15.93:0/788939402 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a44098e30 con 0x560b34abadb0 2023-11-16T16:50:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.833+0000 7f4a3ffff700 1 -- 172.21.15.93:0/788939402 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4409a1e0 con 0x560b34abadb0 2023-11-16T16:50:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.834+0000 7f4a6f061b80 1 -- 172.21.15.93:0/788939402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 msgr2=0x560b34acad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.834+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/788939402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 0x560b34acad60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x560b34abc5b0 tx=0x7f4a4409abf0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.834+0000 7f4a6f061b80 1 -- 172.21.15.93:0/788939402 shutdown_connections 2023-11-16T16:50:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.834+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/788939402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b34abbb10 0x560b34ac21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.834+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/788939402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b349166b0 0x560b34ac1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.834+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/788939402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b34abadb0 0x560b34acad60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.834+0000 7f4a6f061b80 1 -- 172.21.15.93:0/788939402 >> 172.21.15.93:0/788939402 conn(0x560b34aaff50 msgr2=0x560b34ac38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.834+0000 7f4a6f061b80 1 -- 172.21.15.93:0/788939402 shutdown_connections 2023-11-16T16:50:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.834+0000 7f4a6f061b80 1 -- 172.21.15.93:0/788939402 wait complete. 2023-11-16T16:50:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.834+0000 7f4a6f061b80 1 -- start start 2023-11-16T16:50:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.834+0000 7f4a6f061b80 1 Processor -- start 2023-11-16T16:50:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.835+0000 7f4a6f061b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b349166b0 0x560b34ac68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.835+0000 7f4a6f061b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b34abadb0 0x560b34ac6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.835+0000 7f4a6f061b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abbb10 0x560b34ac9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.835+0000 7f4a6f061b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b34aca360 con 0x560b349166b0 2023-11-16T16:50:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.835+0000 7f4a6f061b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b34aca4d0 con 0x560b34abbb10 2023-11-16T16:50:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.835+0000 7f4a6f061b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b34aca690 con 0x560b34abadb0 2023-11-16T16:50:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.835+0000 7f4a6f061b80 10 client.0 _ll_register_callbacks cb 0x7f4a4c009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:50:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.835+0000 7f4a5959b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b349166b0 0x560b34ac68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.835+0000 7f4a5959b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b349166b0 0x560b34ac68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35930/0 (socket says 172.21.15.93:35930) 2023-11-16T16:50:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.835+0000 7f4a5959b700 1 -- 172.21.15.93:0/3760951662 learned_addr learned my addr 172.21.15.93:0/3760951662 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.835+0000 7f4a59d9c700 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abbb10 0x560b34ac9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.835+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b34abadb0 0x560b34ac6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a5959b700 1 -- 172.21.15.93:0/3760951662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b34abadb0 msgr2=0x560b34ac6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a5959b700 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b34abadb0 0x560b34ac6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a5959b700 1 -- 172.21.15.93:0/3760951662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abbb10 msgr2=0x560b34ac9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a5959b700 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abbb10 0x560b34ac9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a5959b700 1 -- 172.21.15.93:0/3760951662 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b3488d5d0 con 0x560b349166b0 2023-11-16T16:50:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a59d9c700 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abbb10 0x560b34ac9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:50:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a5959b700 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b349166b0 0x560b34ac68c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x560b34ad2460 tx=0x7f4a500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f898af7bfe0dec1 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:50:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a3cff9700 1 -- 172.21.15.93:0/3760951662 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a500283b0 con 0x560b349166b0 2023-11-16T16:50:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a6f061b80 1 -- 172.21.15.93:0/3760951662 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a44004c30 con 0x560b349166b0 2023-11-16T16:50:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a3cff9700 1 -- 172.21.15.93:0/3760951662 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f4a50017040 con 0x560b349166b0 2023-11-16T16:50:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a3cff9700 1 -- 172.21.15.93:0/3760951662 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a50028510 con 0x560b349166b0 2023-11-16T16:50:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a6f061b80 10 client.15420 fetch_fsmap learned FSMap version 23 2023-11-16T16:50:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.836+0000 7f4a6f061b80 1 -- 172.21.15.93:0/3760951662 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560b34822c00 con 0x560b349166b0 2023-11-16T16:50:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.837+0000 7f4a3cff9700 1 -- 172.21.15.93:0/3760951662 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a44004c30 con 0x560b349166b0 2023-11-16T16:50:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.837+0000 7f4a3cff9700 1 -- 172.21.15.93:0/3760951662 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a50011850 con 0x560b349166b0 2023-11-16T16:50:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.837+0000 7f4a6f061b80 10 client.15420 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:50:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.837+0000 7f4a6f061b80 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:50:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.837+0000 7f4a6f061b80 20 client.15420 populate_metadata read hostname 'smithi093' 2023-11-16T16:50:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.837+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x560b34b24430 0x560b34b26840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.837+0000 7f4a6f061b80 4 client.15420 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:50:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.837+0000 7f4a6f061b80 1 -- 172.21.15.93:0/3760951662 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560b349da600 con 0x560b34b24430 2023-11-16T16:50:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.837+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x560b34b24430 0x560b34b26840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.838+0000 7f4a58d9a700 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x560b34b24430 0x560b34b26840 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee23bd142019aa30 server_cookie=2923743108b556a9 in_seq=0 out_seq=0 2023-11-16T16:50:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.838+0000 7f4a3cff9700 10 client.15420 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.857+0000 7f4a3cff9700 1 -- 172.21.15.93:0/3760951662 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560b349da600 con 0x560b34b24430 2023-11-16T16:50:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.857+0000 7f4a3cff9700 10 client.15420 handle_command_reply: tid=0 2023-11-16T16:50:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.857+0000 7f4a6f061b80 1 client.15420 shutdown 2023-11-16T16:50:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.857+0000 7f4a6f061b80 1 -- 172.21.15.93:0/3760951662 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b349166b0 msgr2=0x560b34ac68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.857+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b349166b0 0x560b34ac68c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x560b34ad2460 tx=0x7f4a500047c0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.857+0000 7f4a6f061b80 1 -- 172.21.15.93:0/3760951662 shutdown_connections 2023-11-16T16:50:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.857+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x560b34b24430 0x560b34b26840 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.857+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560b349166b0 0x560b34ac68c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.857+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560b34abbb10 0x560b34ac9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.857+0000 7f4a6f061b80 1 --2- 172.21.15.93:0/3760951662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b34abadb0 0x560b34ac6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.857+0000 7f4a6f061b80 1 -- 172.21.15.93:0/3760951662 >> 172.21.15.93:0/3760951662 conn(0x560b34aaff50 msgr2=0x560b34ac1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.857+0000 7f4a3cff9700 0 client.15420 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a6f061b80 1 -- 172.21.15.93:0/3760951662 shutdown_connections 2023-11-16T16:50:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a6f061b80 1 -- 172.21.15.93:0/3760951662 wait complete. 2023-11-16T16:50:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a6f061b80 20 client.15420 trim_cache size 0 max 16384 2023-11-16T16:50:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:50:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:50:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:50:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:50:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/8063555 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4a400a1150 msgr2=0x7f4a400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a5b7ff700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4a400a1150 0x7f4a400a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4a500104e0 tx=0x7f4a5000b480 comp rx=0 tx=0).stop 2023-11-16T16:50:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/8063555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a54074b30 msgr2=0x7f4a54198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a5b7ff700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a54074b30 0x7f4a54198770 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4a44000c00 tx=0x7f4a4400f670 comp rx=0 tx=0).stop 2023-11-16T16:50:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/8063555 shutdown_connections 2023-11-16T16:50:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a5b7ff700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4a400a1150 0x7f4a400a3600 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a5b7ff700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a54108100 0x7f4a5419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a5b7ff700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a54073550 0x7f4a54198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a5b7ff700 1 --2- 172.21.15.93:0/8063555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a54074b30 0x7f4a54198770 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.858+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/8063555 >> 172.21.15.93:0/8063555 conn(0x7f4a540fb840 msgr2=0x7f4a540fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.859+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/8063555 shutdown_connections 2023-11-16T16:50:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:10.859+0000 7f4a5b7ff700 1 -- 172.21.15.93:0/8063555 wait complete. 2023-11-16T16:50:11.129 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:10 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:50:11.129 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:10 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:50:11.129 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:10 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:50:11.129 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:10 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:50:11.129 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:10 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:50:11.129 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:10 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:50:11.130 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:10 smithi093 ceph-mon[97863]: Service mds.all has no related daemon already placed. 2023-11-16T16:50:11.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:10 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:50:11.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:10 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:50:11.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:10 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:50:11.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:10 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:50:11.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:10 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:50:11.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:10 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:50:11.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:10 smithi159 ceph-mon[131528]: Service mds.all has no related daemon already placed. 2023-11-16T16:50:11.314 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:11.314 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:11.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:10 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:50:11.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:10 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:50:11.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:10 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:50:11.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:10 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:50:11.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:10 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:50:11.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:10 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:50:11.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:10 smithi125 ceph-mon[131076]: Service mds.all has no related daemon already placed. 2023-11-16T16:50:11.508 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:11.508 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:11.521 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:11.521 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:11.521 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 15 -lt 100 ']' 2023-11-16T16:50:11.522 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 15 + 1 2023-11-16T16:50:11.542 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=16 2023-11-16T16:50:11.542 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:11.574 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:11.575 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:11.577 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:11.578 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:11.578 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:12.047 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:12.048 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:12.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:11 smithi093 ceph-mon[97863]: pgmap v392: 129 pgs: 129 active+clean; 752 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 8 op/s 2023-11-16T16:50:12.207 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:12.207 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:12.224 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:12.224 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:12.224 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 16 -lt 100 ']' 2023-11-16T16:50:12.225 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 16 + 1 2023-11-16T16:50:12.234 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:11 smithi159 ceph-mon[131528]: pgmap v392: 129 pgs: 129 active+clean; 752 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 8 op/s 2023-11-16T16:50:12.251 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=17 2023-11-16T16:50:12.251 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:12.283 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:12.283 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:12.286 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:12.286 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:12.287 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:12.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:11 smithi125 ceph-mon[131076]: pgmap v392: 129 pgs: 129 active+clean; 752 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 8 op/s 2023-11-16T16:50:12.814 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:12.814 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:12.958 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:12.958 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:12.972 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:12.972 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:12.972 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 17 -lt 100 ']' 2023-11-16T16:50:12.972 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 17 + 1 2023-11-16T16:50:12.992 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=18 2023-11-16T16:50:12.992 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:13.024 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:13.025 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:13.027 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:13.027 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:13.028 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:13.447 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:13.447 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:13.624 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:13.624 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:13.637 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:13.638 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:13.638 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 18 -lt 100 ']' 2023-11-16T16:50:13.638 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 18 + 1 2023-11-16T16:50:13.664 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=19 2023-11-16T16:50:13.664 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:13.698 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:13.699 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:13.701 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:13.702 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:13.702 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:14.148 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:14.148 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:14.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:13 smithi093 ceph-mon[97863]: pgmap v393: 129 pgs: 129 active+clean; 752 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 8 op/s 2023-11-16T16:50:14.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:13 smithi159 ceph-mon[131528]: pgmap v393: 129 pgs: 129 active+clean; 752 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 8 op/s 2023-11-16T16:50:14.299 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:14.299 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:14.313 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:14.313 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:14.313 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 19 -lt 100 ']' 2023-11-16T16:50:14.314 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 19 + 1 2023-11-16T16:50:14.328 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:13 smithi125 ceph-mon[131076]: pgmap v393: 129 pgs: 129 active+clean; 752 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 8 op/s 2023-11-16T16:50:14.334 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=20 2023-11-16T16:50:14.334 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:14.363 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:14.364 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:14.366 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:14.366 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:14.367 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:14.797 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:14.797 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:14.949 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:14.949 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:14.962 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:14.962 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:14.962 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 20 -lt 100 ']' 2023-11-16T16:50:14.962 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 20 + 1 2023-11-16T16:50:15.005 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=21 2023-11-16T16:50:15.005 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:15.028 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:15.029 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:15.031 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:15.032 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:15.032 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:15.463 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:15.464 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:15.622 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:15.622 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:15.638 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:15.638 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:15.638 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 21 -lt 100 ']' 2023-11-16T16:50:15.638 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 21 + 1 2023-11-16T16:50:15.658 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=22 2023-11-16T16:50:15.658 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:15.687 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:15.688 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:15.690 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:15.691 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:15.691 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:16.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:15 smithi093 ceph-mon[97863]: pgmap v394: 129 pgs: 129 active+clean; 840 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-11-16T16:50:16.163 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:16.163 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:16.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:15 smithi159 ceph-mon[131528]: pgmap v394: 129 pgs: 129 active+clean; 840 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-11-16T16:50:16.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:15 smithi125 ceph-mon[131076]: pgmap v394: 129 pgs: 129 active+clean; 840 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-11-16T16:50:16.340 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:16.340 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:16.353 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:16.353 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:16.354 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 22 -lt 100 ']' 2023-11-16T16:50:16.354 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 22 + 1 2023-11-16T16:50:16.375 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=23 2023-11-16T16:50:16.375 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:16.404 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:16.404 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:16.407 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:16.407 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:16.407 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:16.813 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:16.814 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:16.965 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:16.965 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:16.980 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:16.980 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:16.980 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 23 -lt 100 ']' 2023-11-16T16:50:16.980 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 23 + 1 2023-11-16T16:50:17.000 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=24 2023-11-16T16:50:17.000 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:17.032 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:17.032 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:17.035 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:17.035 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:17.036 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:17.447 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:17.447 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:17.582 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:17.582 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:17.603 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:17.603 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:17.603 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 24 -lt 100 ']' 2023-11-16T16:50:17.604 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 24 + 1 2023-11-16T16:50:17.683 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=25 2023-11-16T16:50:17.683 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:17.711 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:17.712 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:17.714 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:17.715 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:17.715 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:18.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:17 smithi093 ceph-mon[97863]: pgmap v395: 129 pgs: 129 active+clean; 840 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-11-16T16:50:18.180 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:18.180 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:18.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:17 smithi159 ceph-mon[131528]: pgmap v395: 129 pgs: 129 active+clean; 840 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-11-16T16:50:18.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:17 smithi125 ceph-mon[131076]: pgmap v395: 129 pgs: 129 active+clean; 840 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-11-16T16:50:18.349 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:18.349 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:18.362 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:18.362 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:18.362 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 25 -lt 100 ']' 2023-11-16T16:50:18.363 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 25 + 1 2023-11-16T16:50:18.383 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=26 2023-11-16T16:50:18.383 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:18.412 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:18.412 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:18.414 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:18.415 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:18.415 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:18.830 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:18.830 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:18.982 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:18.982 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:19.003 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:19.003 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:19.003 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 26 -lt 100 ']' 2023-11-16T16:50:19.004 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 26 + 1 2023-11-16T16:50:19.025 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=27 2023-11-16T16:50:19.025 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:19.059 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:19.060 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:19.062 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:19.063 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:19.063 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:19.479 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:19.480 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:19.615 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:19.615 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:19.628 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:19.628 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:19.628 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 27 -lt 100 ']' 2023-11-16T16:50:19.629 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 27 + 1 2023-11-16T16:50:19.649 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=28 2023-11-16T16:50:19.649 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:19.679 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:19.679 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:19.682 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:19.682 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:19.683 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:20.104 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:20.105 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:20.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:19 smithi093 ceph-mon[97863]: pgmap v396: 129 pgs: 129 active+clean; 840 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-11-16T16:50:20.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:19 smithi159 ceph-mon[131528]: pgmap v396: 129 pgs: 129 active+clean; 840 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-11-16T16:50:20.263 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:20.263 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:20.281 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:20.281 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:20.281 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 28 -lt 100 ']' 2023-11-16T16:50:20.281 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 28 + 1 2023-11-16T16:50:20.327 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:19 smithi125 ceph-mon[131076]: pgmap v396: 129 pgs: 129 active+clean; 840 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-11-16T16:50:20.329 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=29 2023-11-16T16:50:20.329 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:20.423 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:20.424 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:20.426 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:20.427 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:20.427 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:20.854 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:20.855 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:21.015 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:21.015 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:21.037 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:21.037 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:21.037 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 29 -lt 100 ']' 2023-11-16T16:50:21.038 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 29 + 1 2023-11-16T16:50:21.057 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=30 2023-11-16T16:50:21.058 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:21.089 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:21.090 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:21.092 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:21.093 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:21.093 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:21.579 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:21.579 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:21.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:21 smithi093 ceph-mon[97863]: pgmap v397: 129 pgs: 129 active+clean; 886 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-11-16T16:50:21.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:21 smithi159 ceph-mon[131528]: pgmap v397: 129 pgs: 129 active+clean; 886 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-11-16T16:50:21.754 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:21.755 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:21.770 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:21.770 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:21.770 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 30 -lt 100 ']' 2023-11-16T16:50:21.770 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 30 + 1 2023-11-16T16:50:21.791 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=31 2023-11-16T16:50:21.791 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:21.823 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:21.824 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:21.826 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:21.827 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:21.827 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:21.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:21 smithi125 ceph-mon[131076]: pgmap v397: 129 pgs: 129 active+clean; 886 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-11-16T16:50:22.246 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:22.246 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:22.396 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:22.396 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:22.412 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:22.412 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:22.412 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 31 -lt 100 ']' 2023-11-16T16:50:22.413 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 31 + 1 2023-11-16T16:50:22.471 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=32 2023-11-16T16:50:22.471 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:22.557 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:22.557 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:22.559 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:22.560 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:22.560 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:22.979 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:22.979 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:23.148 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:23.148 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:23.203 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:23.203 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:23.203 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 32 -lt 100 ']' 2023-11-16T16:50:23.203 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 32 + 1 2023-11-16T16:50:23.224 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=33 2023-11-16T16:50:23.224 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:23.256 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:23.257 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:23.259 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:23.260 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:23.260 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:23.671 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:23.671 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:23.814 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:23.814 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:23.829 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:23.829 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:23.829 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 33 -lt 100 ']' 2023-11-16T16:50:23.829 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 33 + 1 2023-11-16T16:50:23.849 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=34 2023-11-16T16:50:23.849 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:23.881 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:23.881 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:23.884 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:23.884 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:23.885 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:23.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:23 smithi093 ceph-mon[97863]: pgmap v398: 129 pgs: 129 active+clean; 887 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-11-16T16:50:23.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:23 smithi159 ceph-mon[131528]: pgmap v398: 129 pgs: 129 active+clean; 887 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-11-16T16:50:24.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:23 smithi125 ceph-mon[131076]: pgmap v398: 129 pgs: 129 active+clean; 887 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-11-16T16:50:24.379 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:24.379 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:24.547 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:24.547 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:24.602 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:24.602 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:24.602 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 34 -lt 100 ']' 2023-11-16T16:50:24.602 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 34 + 1 2023-11-16T16:50:24.662 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=35 2023-11-16T16:50:24.662 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:24.714 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:24.714 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:24.716 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:24.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:24.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:25.470 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:25.471 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:25.731 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:25.731 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:25.787 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:25.787 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:25.787 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 35 -lt 100 ']' 2023-11-16T16:50:25.787 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 35 + 1 2023-11-16T16:50:25.829 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=36 2023-11-16T16:50:25.829 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:25.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:25 smithi093 ceph-mon[97863]: pgmap v399: 129 pgs: 129 active+clean; 966 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-11-16T16:50:25.940 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:25.940 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:25.943 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:25.943 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:25.944 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:25.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:25 smithi159 ceph-mon[131528]: pgmap v399: 129 pgs: 129 active+clean; 966 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-11-16T16:50:26.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:25 smithi125 ceph-mon[131076]: pgmap v399: 129 pgs: 129 active+clean; 966 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-11-16T16:50:27.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.606+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3131869544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aac1002c0 msgr2=0x7f8aac1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.606+0000 7f8ab15ff700 1 --2- 172.21.15.93:0/3131869544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aac1002c0 0x7f8aac1006d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009990 tx=0x7f8a9c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.606+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3131869544 shutdown_connections 2023-11-16T16:50:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.606+0000 7f8ab15ff700 1 --2- 172.21.15.93:0/3131869544 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8aac101e40 0x7f8aac108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.606+0000 7f8ab15ff700 1 --2- 172.21.15.93:0/3131869544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aac1002c0 0x7f8aac1006d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.606+0000 7f8ab15ff700 1 --2- 172.21.15.93:0/3131869544 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8aac1014b0 0x7f8aac101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.606+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3131869544 >> 172.21.15.93:0/3131869544 conn(0x7f8aac0fb8e0 msgr2=0x7f8aac0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.606+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3131869544 shutdown_connections 2023-11-16T16:50:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.606+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3131869544 wait complete. 2023-11-16T16:50:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.607+0000 7f8ab15ff700 1 Processor -- start 2023-11-16T16:50:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.608+0000 7f8ab15ff700 1 -- start start 2023-11-16T16:50:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.608+0000 7f8ab15ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8aac1002c0 0x7f8aac1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.608+0000 7f8ab15ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aac1014b0 0x7f8aac198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.608+0000 7f8ab15ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8aac101e40 0x7f8aac19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.608+0000 7f8ab15ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac19fd40 con 0x7f8aac1002c0 2023-11-16T16:50:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.608+0000 7f8ab15ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac19fe80 con 0x7f8aac101e40 2023-11-16T16:50:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.608+0000 7f8ab15ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac1a0010 con 0x7f8aac1014b0 2023-11-16T16:50:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.608+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8aac1002c0 0x7f8aac1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.608+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8aac1002c0 0x7f8aac1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39834/0 (socket says 172.21.15.93:39834) 2023-11-16T16:50:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.608+0000 7f8aaaffd700 1 -- 172.21.15.93:0/3759084872 learned_addr learned my addr 172.21.15.93:0/3759084872 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:27.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.608+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aac1014b0 0x7f8aac198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.608+0000 7f8aab7fe700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8aac101e40 0x7f8aac19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.609+0000 7f8aaaffd700 1 -- 172.21.15.93:0/3759084872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aac1014b0 msgr2=0x7f8aac198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.609+0000 7f8aaaffd700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aac1014b0 0x7f8aac198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.609+0000 7f8aaaffd700 1 -- 172.21.15.93:0/3759084872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8aac101e40 msgr2=0x7f8aac19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.609+0000 7f8aaaffd700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8aac101e40 0x7f8aac19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.609+0000 7f8aaaffd700 1 -- 172.21.15.93:0/3759084872 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c0095f0 con 0x7f8aac1002c0 2023-11-16T16:50:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.609+0000 7f8aaaffd700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8aac1002c0 0x7f8aac1981c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0095c0 tx=0x7f8a9c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.610+0000 7f8aa8ff9700 1 -- 172.21.15.93:0/3759084872 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c01d070 con 0x7f8aac1002c0 2023-11-16T16:50:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.610+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3759084872 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac10bbb0 con 0x7f8aac1002c0 2023-11-16T16:50:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.610+0000 7f8aa8ff9700 1 -- 172.21.15.93:0/3759084872 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a9c022470 con 0x7f8aac1002c0 2023-11-16T16:50:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.610+0000 7f8aa8ff9700 1 -- 172.21.15.93:0/3759084872 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c017650 con 0x7f8aac1002c0 2023-11-16T16:50:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.610+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3759084872 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac10ba20 con 0x7f8aac1002c0 2023-11-16T16:50:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.613+0000 7f8aa8ff9700 1 -- 172.21.15.93:0/3759084872 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8a9c0225e0 con 0x7f8aac1002c0 2023-11-16T16:50:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.615+0000 7f8aa8ff9700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8a980a1150 0x7f8a980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.615+0000 7f8aa8ff9700 1 -- 172.21.15.93:0/3759084872 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f8a9c0ce110 con 0x7f8aac1002c0 2023-11-16T16:50:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.615+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8a980a1150 0x7f8a980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.616+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8a980a1150 0x7f8a980a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009c60 tx=0x7f8a94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.690+0000 7f8ac5006b80 1 Processor -- start 2023-11-16T16:50:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.690+0000 7f8ac5006b80 1 -- start start 2023-11-16T16:50:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.691+0000 7f8ac5006b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558befe0e6b0 0x558beffb2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.691+0000 7f8ac5006b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558beffb2db0 0x558beffb35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.691+0000 7f8ac5006b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb3b10 0x558beffb3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.691+0000 7f8ac5006b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558befeb8000 con 0x558befe0e6b0 2023-11-16T16:50:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.691+0000 7f8ac5006b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beff0a840 con 0x558beffb2db0 2023-11-16T16:50:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.691+0000 7f8ac5006b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beff03cd0 con 0x558beffb3b10 2023-11-16T16:50:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.691+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558befe0e6b0 0x558beffb2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.691+0000 7f8aa3fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558beffb2db0 0x558beffb35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.691+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558befe0e6b0 0x558beffb2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39836/0 (socket says 172.21.15.93:39836) 2023-11-16T16:50:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.691+0000 7f8aaaffd700 1 -- 172.21.15.93:0/31628293 learned_addr learned my addr 172.21.15.93:0/31628293 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.691+0000 7f8aab7fe700 1 --2- 172.21.15.93:0/31628293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb3b10 0x558beffb3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.692+0000 7f8aa3fff700 1 -- 172.21.15.93:0/31628293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb3b10 msgr2=0x558beffb3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.692+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/31628293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb3b10 0x558beffb3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.692+0000 7f8aa3fff700 1 -- 172.21.15.93:0/31628293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558befe0e6b0 msgr2=0x558beffb2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.692+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/31628293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558befe0e6b0 0x558beffb2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.692+0000 7f8aa3fff700 1 -- 172.21.15.93:0/31628293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558befd855d0 con 0x558beffb2db0 2023-11-16T16:50:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.692+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/31628293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558beffb2db0 0x558beffb35d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400e3b0 tx=0x7f8a94014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac75b2df14908f8d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.692+0000 7f8aa17fa700 1 -- 172.21.15.93:0/31628293 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400ed50 con 0x558beffb2db0 2023-11-16T16:50:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.692+0000 7f8aa17fa700 1 -- 172.21.15.93:0/31628293 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a94021070 con 0x558beffb2db0 2023-11-16T16:50:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.692+0000 7f8aa17fa700 1 -- 172.21.15.93:0/31628293 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94026470 con 0x558beffb2db0 2023-11-16T16:50:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.693+0000 7f8ac5006b80 1 -- 172.21.15.93:0/31628293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558beffb2db0 msgr2=0x558beffb35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.693+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/31628293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558beffb2db0 0x558beffb35d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400e3b0 tx=0x7f8a94014000 comp rx=0 tx=0).stop 2023-11-16T16:50:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.693+0000 7f8ac5006b80 1 -- 172.21.15.93:0/31628293 shutdown_connections 2023-11-16T16:50:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.693+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/31628293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558befe0e6b0 0x558beffb2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.693+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/31628293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb3b10 0x558beffb3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.693+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/31628293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558beffb2db0 0x558beffb35d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.693+0000 7f8ac5006b80 1 -- 172.21.15.93:0/31628293 >> 172.21.15.93:0/31628293 conn(0x558beffa7f50 msgr2=0x558beffad100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.693+0000 7f8ac5006b80 1 -- 172.21.15.93:0/31628293 shutdown_connections 2023-11-16T16:50:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.693+0000 7f8ac5006b80 1 -- 172.21.15.93:0/31628293 wait complete. 2023-11-16T16:50:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.694+0000 7f8ac5006b80 1 -- start start 2023-11-16T16:50:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.694+0000 7f8ac5006b80 1 Processor -- start 2023-11-16T16:50:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.695+0000 7f8ac5006b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 0x558beffbbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.695+0000 7f8ac5006b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb2db0 0x558beffbc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.695+0000 7f8ac5006b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb3b10 0x558beffc3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.695+0000 7f8ac5006b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558befea5880 con 0x558beffb2db0 2023-11-16T16:50:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.695+0000 7f8ac5006b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558befea5520 con 0x558befe0e6b0 2023-11-16T16:50:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.695+0000 7f8ac5006b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558befd878c0 con 0x558beffb3b10 2023-11-16T16:50:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.695+0000 7f8ac5006b80 10 client.0 _ll_register_callbacks cb 0x558befec3720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.695+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 0x558beffbbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.695+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 0x558beffbbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:60376/0 (socket says 172.21.15.93:60376) 2023-11-16T16:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.695+0000 7f8aaaffd700 1 -- 172.21.15.93:0/472743144 learned_addr learned my addr 172.21.15.93:0/472743144 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.695+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb2db0 0x558beffbc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.695+0000 7f8aab7fe700 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb3b10 0x558beffc3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.696+0000 7f8aaaffd700 1 -- 172.21.15.93:0/472743144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb3b10 msgr2=0x558beffc3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.696+0000 7f8aaaffd700 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb3b10 0x558beffc3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.696+0000 7f8aaaffd700 1 -- 172.21.15.93:0/472743144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb2db0 msgr2=0x558beffbc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.696+0000 7f8aaaffd700 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb2db0 0x558beffbc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.696+0000 7f8aaaffd700 1 -- 172.21.15.93:0/472743144 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a94017070 con 0x558befe0e6b0 2023-11-16T16:50:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.696+0000 7f8aab7fe700 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb3b10 0x558beffc3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.696+0000 7f8aaaffd700 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 0x558beffbbca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00b680 tx=0x7f8a9c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20cce00c521773d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.696+0000 7f8a927fc700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.696+0000 7f8a927fc700 1 -- 172.21.15.93:0/472743144 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c09a870 con 0x558befe0e6b0 2023-11-16T16:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.696+0000 7f8a927fc700 1 -- 172.21.15.93:0/472743144 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f8a9c09f1b0 con 0x558befe0e6b0 2023-11-16T16:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.696+0000 7f8ac5006b80 1 -- 172.21.15.93:0/472743144 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558beffa5710 con 0x558befe0e6b0 2023-11-16T16:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.697+0000 7f8a927fc700 1 -- 172.21.15.93:0/472743144 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a9c0a3540 con 0x558befe0e6b0 2023-11-16T16:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.697+0000 7f8ac5006b80 10 client.25469 fetch_fsmap learned FSMap version 23 2023-11-16T16:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.697+0000 7f8ac5006b80 1 -- 172.21.15.93:0/472743144 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558befeca130 con 0x558befe0e6b0 2023-11-16T16:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.697+0000 7f8a927fc700 1 -- 172.21.15.93:0/472743144 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a9c0c9080 con 0x558befe0e6b0 2023-11-16T16:50:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.697+0000 7f8a927fc700 1 -- 172.21.15.93:0/472743144 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a9c096fe0 con 0x558befe0e6b0 2023-11-16T16:50:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.697+0000 7f8ac5006b80 10 client.25469 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:50:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.697+0000 7f8ac5006b80 10 client.25469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:50:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.697+0000 7f8ac5006b80 20 client.25469 populate_metadata read hostname 'smithi093' 2023-11-16T16:50:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.697+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x558beffc76c0 0x558beffc9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.697+0000 7f8ac5006b80 4 client.25469 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:50:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.697+0000 7f8ac5006b80 1 -- 172.21.15.93:0/472743144 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558befec6040 con 0x558beffc76c0 2023-11-16T16:50:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.698+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x558beffc76c0 0x558beffc9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.698+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x558beffc76c0 0x558beffc9ad0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a15e0abf44b18aec server_cookie=1ff2b5c46f2c7ab9 in_seq=0 out_seq=0 2023-11-16T16:50:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.698+0000 7f8a927fc700 10 client.25469 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.699+0000 7f8a927fc700 1 -- 172.21.15.93:0/472743144 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558befec6040 con 0x558beffc76c0 2023-11-16T16:50:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.699+0000 7f8a927fc700 10 client.25469 handle_command_reply: tid=0 2023-11-16T16:50:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.699+0000 7f8ac5006b80 1 client.25469 shutdown 2023-11-16T16:50:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.699+0000 7f8ac5006b80 1 -- 172.21.15.93:0/472743144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 msgr2=0x558beffbbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.699+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 0x558beffbbca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00b680 tx=0x7f8a9c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.699+0000 7f8ac5006b80 1 -- 172.21.15.93:0/472743144 shutdown_connections 2023-11-16T16:50:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.699+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x558beffc76c0 0x558beffc9ad0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.699+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb3b10 0x558beffc3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.699+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 0x558beffbbca0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.699+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/472743144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb2db0 0x558beffbc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.699+0000 7f8a927fc700 0 client.25469 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.699+0000 7f8ac5006b80 1 -- 172.21.15.93:0/472743144 >> 172.21.15.93:0/472743144 conn(0x558beffa7f50 msgr2=0x558beffac7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.700+0000 7f8ac5006b80 1 -- 172.21.15.93:0/472743144 shutdown_connections 2023-11-16T16:50:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.700+0000 7f8ac5006b80 1 -- 172.21.15.93:0/472743144 wait complete. 2023-11-16T16:50:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.700+0000 7f8ac5006b80 20 client.25469 trim_cache size 0 max 16384 2023-11-16T16:50:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8ac5006b80 1 Processor -- start 2023-11-16T16:50:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8ac5006b80 1 -- start start 2023-11-16T16:50:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8ac5006b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558befe0e6b0 0x558beffba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8ac5006b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb2db0 0x558beffc31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8ac5006b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558beffb3b10 0x558beffce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8ac5006b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beffbb150 con 0x558beffb2db0 2023-11-16T16:50:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8ac5006b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beffbccd0 con 0x558beffb3b10 2023-11-16T16:50:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8ac5006b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beffaebb0 con 0x558befe0e6b0 2023-11-16T16:50:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8aa3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb2db0 0x558beffc31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558befe0e6b0 0x558beffba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8aa3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb2db0 0x558beffc31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39860/0 (socket says 172.21.15.93:39860) 2023-11-16T16:50:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8aa3fff700 1 -- 172.21.15.93:0/3897132639 learned_addr learned my addr 172.21.15.93:0/3897132639 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.718+0000 7f8aab7fe700 1 --2- 172.21.15.93:0/3897132639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558beffb3b10 0x558beffce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.719+0000 7f8aa3fff700 1 -- 172.21.15.93:0/3897132639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558befe0e6b0 msgr2=0x558beffba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.719+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/3897132639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558befe0e6b0 0x558beffba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.719+0000 7f8aa3fff700 1 -- 172.21.15.93:0/3897132639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558beffb3b10 msgr2=0x558beffce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.719+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/3897132639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558beffb3b10 0x558beffce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.719+0000 7f8aa3fff700 1 -- 172.21.15.93:0/3897132639 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558befec9e80 con 0x558beffb2db0 2023-11-16T16:50:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.719+0000 7f8aaaffd700 1 --2- 172.21.15.93:0/3897132639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558befe0e6b0 0x558beffba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:50:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.719+0000 7f8aab7fe700 1 --2- 172.21.15.93:0/3897132639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558beffb3b10 0x558beffce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:50:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.719+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/3897132639 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb2db0 0x558beffc31e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400e9a0 tx=0x7f8a94005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2347c8e3eeb5e9e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.719+0000 7f8aa1ffb700 1 -- 172.21.15.93:0/3897132639 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94035030 con 0x558beffb2db0 2023-11-16T16:50:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.719+0000 7f8aa1ffb700 1 -- 172.21.15.93:0/3897132639 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a94003680 con 0x558beffb2db0 2023-11-16T16:50:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.719+0000 7f8aa1ffb700 1 -- 172.21.15.93:0/3897132639 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9402a040 con 0x558beffb2db0 2023-11-16T16:50:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.720+0000 7f8ac5006b80 1 -- 172.21.15.93:0/3897132639 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb2db0 msgr2=0x558beffc31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.720+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/3897132639 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb2db0 0x558beffc31e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400e9a0 tx=0x7f8a94005040 comp rx=0 tx=0).stop 2023-11-16T16:50:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.720+0000 7f8ac5006b80 1 -- 172.21.15.93:0/3897132639 shutdown_connections 2023-11-16T16:50:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.720+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/3897132639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558beffb3b10 0x558beffce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.720+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/3897132639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558befe0e6b0 0x558beffba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.720+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/3897132639 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb2db0 0x558beffc31e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.720+0000 7f8ac5006b80 1 -- 172.21.15.93:0/3897132639 >> 172.21.15.93:0/3897132639 conn(0x558beffa7f50 msgr2=0x558beffad0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.720+0000 7f8ac5006b80 1 -- 172.21.15.93:0/3897132639 shutdown_connections 2023-11-16T16:50:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.720+0000 7f8ac5006b80 1 -- 172.21.15.93:0/3897132639 wait complete. 2023-11-16T16:50:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.721+0000 7f8ac5006b80 1 -- start start 2023-11-16T16:50:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.721+0000 7f8ac5006b80 1 Processor -- start 2023-11-16T16:50:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.721+0000 7f8ac5006b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 0x558beffbef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.721+0000 7f8ac5006b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb2db0 0x558beffbf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.721+0000 7f8ac5006b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb3b10 0x558beffc24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.721+0000 7f8ac5006b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beffc2a20 con 0x558beffb3b10 2023-11-16T16:50:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.721+0000 7f8ac5006b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beffc2b90 con 0x558befe0e6b0 2023-11-16T16:50:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.721+0000 7f8ac5006b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beffc2d50 con 0x558beffb2db0 2023-11-16T16:50:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.722+0000 7f8ac5006b80 10 client.0 _ll_register_callbacks cb 0x7f8a9c0a10b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:50:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.722+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 0x558beffbef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.722+0000 7f8aab7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb3b10 0x558beffc24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.722+0000 7f8aa3fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb2db0 0x558beffbf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.722+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 0x558beffbef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:60384/0 (socket says 172.21.15.93:60384) 2023-11-16T16:50:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.722+0000 7f8aaaffd700 1 -- 172.21.15.93:0/657030841 learned_addr learned my addr 172.21.15.93:0/657030841 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.722+0000 7f8aaaffd700 1 -- 172.21.15.93:0/657030841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb2db0 msgr2=0x558beffbf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.722+0000 7f8aaaffd700 1 --2- 172.21.15.93:0/657030841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb2db0 0x558beffbf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.722+0000 7f8aaaffd700 1 -- 172.21.15.93:0/657030841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb3b10 msgr2=0x558beffc24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.722+0000 7f8aaaffd700 1 --2- 172.21.15.93:0/657030841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb3b10 0x558beffc24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.722+0000 7f8aaaffd700 1 -- 172.21.15.93:0/657030841 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558befeca130 con 0x558befe0e6b0 2023-11-16T16:50:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.722+0000 7f8aab7fe700 1 --2- 172.21.15.93:0/657030841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb3b10 0x558beffc24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:50:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.723+0000 7f8aaaffd700 1 --2- 172.21.15.93:0/657030841 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 0x558beffbef80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c097680 tx=0x7f8a9c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcffdf031119172a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.723+0000 7f8a927fc700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:50:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.723+0000 7f8a927fc700 1 -- 172.21.15.93:0/657030841 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0a32b0 con 0x558befe0e6b0 2023-11-16T16:50:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.723+0000 7f8a927fc700 1 -- 172.21.15.93:0/657030841 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f8a9c09f1b0 con 0x558befe0e6b0 2023-11-16T16:50:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.723+0000 7f8ac5006b80 1 -- 172.21.15.93:0/657030841 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a94021070 con 0x558befe0e6b0 2023-11-16T16:50:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.723+0000 7f8a927fc700 1 -- 172.21.15.93:0/657030841 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a9c0a3540 con 0x558befe0e6b0 2023-11-16T16:50:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.723+0000 7f8ac5006b80 10 client.25475 fetch_fsmap learned FSMap version 23 2023-11-16T16:50:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.723+0000 7f8ac5006b80 1 -- 172.21.15.93:0/657030841 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f8aa4009100 con 0x558befe0e6b0 2023-11-16T16:50:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.724+0000 7f8a927fc700 1 -- 172.21.15.93:0/657030841 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a94021070 con 0x558befe0e6b0 2023-11-16T16:50:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.724+0000 7f8a927fc700 1 -- 172.21.15.93:0/657030841 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a9c0b92a0 con 0x558befe0e6b0 2023-11-16T16:50:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.724+0000 7f8ac5006b80 10 client.25475 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:50:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.724+0000 7f8ac5006b80 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:50:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.724+0000 7f8ac5006b80 20 client.25475 populate_metadata read hostname 'smithi093' 2023-11-16T16:50:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.724+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/657030841 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x558bf001c1f0 0x558bf001e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.724+0000 7f8ac5006b80 4 client.25475 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:50:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.724+0000 7f8ac5006b80 1 -- 172.21.15.93:0/657030841 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558befed2600 con 0x558bf001c1f0 2023-11-16T16:50:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.724+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/657030841 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x558bf001c1f0 0x558bf001e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.725+0000 7f8aa3fff700 1 --2- 172.21.15.93:0/657030841 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x558bf001c1f0 0x558bf001e600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b205d0ff6fa3ec27 server_cookie=4ed350726b41300d in_seq=0 out_seq=0 2023-11-16T16:50:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.725+0000 7f8a927fc700 10 client.25475 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.743+0000 7f8a927fc700 1 -- 172.21.15.93:0/657030841 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558befed2600 con 0x558bf001c1f0 2023-11-16T16:50:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.743+0000 7f8a927fc700 10 client.25475 handle_command_reply: tid=0 2023-11-16T16:50:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.743+0000 7f8ac5006b80 1 client.25475 shutdown 2023-11-16T16:50:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.744+0000 7f8ac5006b80 1 -- 172.21.15.93:0/657030841 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 msgr2=0x558beffbef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.744+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/657030841 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 0x558beffbef80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c097680 tx=0x7f8a9c005c00 comp rx=0 tx=0).stop 2023-11-16T16:50:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.744+0000 7f8ac5006b80 1 -- 172.21.15.93:0/657030841 shutdown_connections 2023-11-16T16:50:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.744+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/657030841 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x558bf001c1f0 0x558bf001e600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.744+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/657030841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558beffb3b10 0x558beffc24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.744+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/657030841 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558befe0e6b0 0x558beffbef80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.744+0000 7f8ac5006b80 1 --2- 172.21.15.93:0/657030841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558beffb2db0 0x558beffbf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.744+0000 7f8a927fc700 0 client.25475 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.744+0000 7f8ac5006b80 1 -- 172.21.15.93:0/657030841 >> 172.21.15.93:0/657030841 conn(0x558beffa7f50 msgr2=0x558beffb9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.744+0000 7f8ac5006b80 1 -- 172.21.15.93:0/657030841 shutdown_connections 2023-11-16T16:50:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.744+0000 7f8ac5006b80 1 -- 172.21.15.93:0/657030841 wait complete. 2023-11-16T16:50:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.744+0000 7f8ac5006b80 20 client.25475 trim_cache size 0 max 16384 2023-11-16T16:50:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:50:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:50:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:50:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:50:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.745+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3759084872 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8a980a1150 msgr2=0x7f8a980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.745+0000 7f8ab15ff700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8a980a1150 0x7f8a980a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009c60 tx=0x7f8a94009260 comp rx=0 tx=0).stop 2023-11-16T16:50:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.745+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3759084872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8aac1002c0 msgr2=0x7f8aac1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.745+0000 7f8ab15ff700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8aac1002c0 0x7f8aac1981c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0095c0 tx=0x7f8a9c005720 comp rx=0 tx=0).stop 2023-11-16T16:50:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.745+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3759084872 shutdown_connections 2023-11-16T16:50:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.745+0000 7f8ab15ff700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f8a980a1150 0x7f8a980a3600 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.745+0000 7f8ab15ff700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8aac1002c0 0x7f8aac1981c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.745+0000 7f8ab15ff700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8aac101e40 0x7f8aac19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.745+0000 7f8ab15ff700 1 --2- 172.21.15.93:0/3759084872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aac1014b0 0x7f8aac198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.745+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3759084872 >> 172.21.15.93:0/3759084872 conn(0x7f8aac0fb8e0 msgr2=0x7f8aac106de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.745+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3759084872 shutdown_connections 2023-11-16T16:50:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:27.745+0000 7f8ab15ff700 1 -- 172.21.15.93:0/3759084872 wait complete. 2023-11-16T16:50:27.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:27 smithi093 ceph-mon[97863]: pgmap v400: 129 pgs: 129 active+clean; 967 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-11-16T16:50:27.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:27 smithi159 ceph-mon[131528]: pgmap v400: 129 pgs: 129 active+clean; 967 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-11-16T16:50:28.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:27 smithi125 ceph-mon[131076]: pgmap v400: 129 pgs: 129 active+clean; 967 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-11-16T16:50:28.262 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:28.262 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:28.438 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:28.439 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:28.452 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:28.452 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:28.452 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 36 -lt 100 ']' 2023-11-16T16:50:28.453 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 36 + 1 2023-11-16T16:50:28.473 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=37 2023-11-16T16:50:28.473 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:28.505 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:28.506 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:28.508 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:28.509 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:28.509 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:28.970 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:28.970 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:29.156 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:29.156 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:29.477 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:29.477 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:29.477 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 37 -lt 100 ']' 2023-11-16T16:50:29.478 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 37 + 1 2023-11-16T16:50:29.536 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=38 2023-11-16T16:50:29.536 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:29.605 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:29.606 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:29.608 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:29.609 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:29.609 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:29.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:29 smithi093 ceph-mon[97863]: pgmap v401: 129 pgs: 129 active+clean; 967 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 8 op/s 2023-11-16T16:50:29.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:29 smithi159 ceph-mon[131528]: pgmap v401: 129 pgs: 129 active+clean; 967 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 8 op/s 2023-11-16T16:50:30.061 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:30.062 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:30.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:29 smithi125 ceph-mon[131076]: pgmap v401: 129 pgs: 129 active+clean; 967 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 8 op/s 2023-11-16T16:50:30.197 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:30.197 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:30.210 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:30.210 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:30.210 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 38 -lt 100 ']' 2023-11-16T16:50:30.210 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 38 + 1 2023-11-16T16:50:30.231 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=39 2023-11-16T16:50:30.231 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:30.261 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:30.262 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:30.264 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:30.265 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:30.265 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:30.636 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:30.637 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:30.782 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:30.783 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:30.794 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:30.794 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:30.794 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 39 -lt 100 ']' 2023-11-16T16:50:30.795 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 39 + 1 2023-11-16T16:50:30.814 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=40 2023-11-16T16:50:30.814 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:30.843 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:30.844 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:30.846 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:30.847 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:30.847 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:31.253 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:31.253 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:31.388 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:31.388 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:31.410 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:31.410 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:31.410 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 40 -lt 100 ']' 2023-11-16T16:50:31.410 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 40 + 1 2023-11-16T16:50:31.431 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=41 2023-11-16T16:50:31.431 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:31.461 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:31.462 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:31.464 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:31.465 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:31.465 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:31.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:31 smithi093 ceph-mon[97863]: pgmap v402: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-11-16T16:50:31.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:31 smithi159 ceph-mon[131528]: pgmap v402: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-11-16T16:50:31.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:31 smithi125 ceph-mon[131076]: pgmap v402: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-11-16T16:50:31.903 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:31.903 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:32.036 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:32.036 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:32.052 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:32.052 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:32.053 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 41 -lt 100 ']' 2023-11-16T16:50:32.053 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 41 + 1 2023-11-16T16:50:32.072 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=42 2023-11-16T16:50:32.072 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:32.101 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:32.102 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:32.104 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:32.105 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:32.105 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:32.627 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:32.628 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:32.796 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:32.796 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:32.809 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:32.809 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:32.810 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 42 -lt 100 ']' 2023-11-16T16:50:32.810 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 42 + 1 2023-11-16T16:50:32.831 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=43 2023-11-16T16:50:32.831 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:32.863 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:32.864 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:32.866 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:32.867 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:32.867 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:33.336 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:33.336 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:33.521 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:33.521 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:33.552 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:33.552 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:33.552 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 43 -lt 100 ']' 2023-11-16T16:50:33.552 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 43 + 1 2023-11-16T16:50:33.572 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=44 2023-11-16T16:50:33.572 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:33.602 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:33.603 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:33.605 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:33.606 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:33.606 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:33.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:33 smithi093 ceph-mon[97863]: pgmap v403: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-11-16T16:50:33.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:33 smithi159 ceph-mon[131528]: pgmap v403: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-11-16T16:50:34.053 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:34.053 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:34.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:33 smithi125 ceph-mon[131076]: pgmap v403: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-11-16T16:50:34.203 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:34.203 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:34.218 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:34.218 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:34.218 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 44 -lt 100 ']' 2023-11-16T16:50:34.218 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 44 + 1 2023-11-16T16:50:34.239 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=45 2023-11-16T16:50:34.239 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:34.277 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:34.277 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:34.280 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:34.281 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:34.281 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:34.669 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:34.670 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:34.813 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:34.813 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:34.835 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:34.836 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:34.836 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 45 -lt 100 ']' 2023-11-16T16:50:34.836 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 45 + 1 2023-11-16T16:50:34.855 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=46 2023-11-16T16:50:34.855 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:34.885 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:34.885 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:34.888 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:34.888 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:34.889 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:35.361 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:35.361 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:35.504 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:35.504 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:35.526 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:35.526 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:35.526 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 46 -lt 100 ']' 2023-11-16T16:50:35.527 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 46 + 1 2023-11-16T16:50:35.547 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=47 2023-11-16T16:50:35.547 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:35.577 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:35.577 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:35.580 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:35.581 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:35.581 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:35.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:35 smithi093 ceph-mon[97863]: pgmap v404: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-11-16T16:50:35.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:35 smithi159 ceph-mon[131528]: pgmap v404: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-11-16T16:50:36.011 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:36.011 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:36.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:35 smithi125 ceph-mon[131076]: pgmap v404: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-11-16T16:50:36.171 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:36.171 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:36.185 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:36.185 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:36.185 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 47 -lt 100 ']' 2023-11-16T16:50:36.186 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 47 + 1 2023-11-16T16:50:36.205 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=48 2023-11-16T16:50:36.205 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:36.235 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:36.235 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:36.238 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:36.239 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:36.239 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:36.586 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:36.586 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:36.729 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:36.729 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:36.751 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:36.751 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:36.751 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 48 -lt 100 ']' 2023-11-16T16:50:36.752 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 48 + 1 2023-11-16T16:50:36.772 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=49 2023-11-16T16:50:36.772 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:36.803 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:36.803 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:36.806 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:36.807 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:36.807 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:37.210 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:37.211 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:37.345 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:37.346 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:37.377 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:37.377 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:37.377 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 49 -lt 100 ']' 2023-11-16T16:50:37.378 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 49 + 1 2023-11-16T16:50:37.435 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=50 2023-11-16T16:50:37.435 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:37.470 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:37.471 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:37.473 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:37.474 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:37.474 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:37.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:37 smithi093 ceph-mon[97863]: pgmap v405: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-11-16T16:50:37.944 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:37.944 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:37.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:37 smithi159 ceph-mon[131528]: pgmap v405: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-11-16T16:50:38.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:37 smithi125 ceph-mon[131076]: pgmap v405: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-11-16T16:50:38.170 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:38.170 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:38.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:38.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:38.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 50 -lt 100 ']' 2023-11-16T16:50:38.194 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 50 + 1 2023-11-16T16:50:38.213 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=51 2023-11-16T16:50:38.213 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:38.244 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:38.244 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:38.247 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:38.248 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:38.249 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:38.644 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:38.644 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:38.787 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:38.787 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:38.811 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:38.811 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:38.811 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 51 -lt 100 ']' 2023-11-16T16:50:38.812 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 51 + 1 2023-11-16T16:50:38.838 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=52 2023-11-16T16:50:38.838 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:38.868 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:38.869 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:38.872 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:38.872 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:38.873 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:39.260 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:39.261 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:39.404 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:39.404 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:39.417 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:39.417 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:39.417 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 52 -lt 100 ']' 2023-11-16T16:50:39.418 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 52 + 1 2023-11-16T16:50:39.438 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=53 2023-11-16T16:50:39.438 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:39.469 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:39.470 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:39.472 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:39.473 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:39.474 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:39.852 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:39.852 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:39 smithi093 ceph-mon[97863]: pgmap v406: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-11-16T16:50:39.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:39 smithi159 ceph-mon[131528]: pgmap v406: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-11-16T16:50:40.003 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:40.004 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:40.018 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:40.018 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:40.018 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 53 -lt 100 ']' 2023-11-16T16:50:40.018 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 53 + 1 2023-11-16T16:50:40.038 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=54 2023-11-16T16:50:40.038 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:40.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:39 smithi125 ceph-mon[131076]: pgmap v406: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-11-16T16:50:40.084 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:40.084 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:40.087 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:40.088 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:40.088 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:40.577 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:40.577 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:40.704 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:40.704 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:40.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:40.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:40.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 54 -lt 100 ']' 2023-11-16T16:50:40.717 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 54 + 1 2023-11-16T16:50:40.738 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=55 2023-11-16T16:50:40.738 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:40.767 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:40.767 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:40.770 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:40.770 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:40.771 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:41.294 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:41.294 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:41.513 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:41.514 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:41.535 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:41.535 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:41.535 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 55 -lt 100 ']' 2023-11-16T16:50:41.535 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 55 + 1 2023-11-16T16:50:41.610 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=56 2023-11-16T16:50:41.610 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:41.674 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:41.675 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:41.677 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:41.678 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:41.678 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:42.376 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:42.377 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:42.628 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:42.628 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:42.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:42 smithi093 ceph-mon[97863]: pgmap v407: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-11-16T16:50:42.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:42.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:42.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 56 -lt 100 ']' 2023-11-16T16:50:42.718 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 56 + 1 2023-11-16T16:50:42.734 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:42 smithi159 ceph-mon[131528]: pgmap v407: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-11-16T16:50:42.818 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=57 2023-11-16T16:50:42.818 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:42.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:42 smithi125 ceph-mon[131076]: pgmap v407: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-11-16T16:50:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.846+0000 7fe9fcdff700 1 -- 172.21.15.93:0/848596832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9f81014b0 msgr2=0x7fe9f8101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.846+0000 7fe9fcdff700 1 --2- 172.21.15.93:0/848596832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9f81014b0 0x7fe9f8101900 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009990 tx=0x7fe9e0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:50:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.846+0000 7fe9fcdff700 1 -- 172.21.15.93:0/848596832 shutdown_connections 2023-11-16T16:50:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.846+0000 7fe9fcdff700 1 --2- 172.21.15.93:0/848596832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f8101e40 0x7fe9f8108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.846+0000 7fe9fcdff700 1 --2- 172.21.15.93:0/848596832 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9f81002c0 0x7fe9f81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.846+0000 7fe9fcdff700 1 --2- 172.21.15.93:0/848596832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9f81014b0 0x7fe9f8101900 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.846+0000 7fe9fcdff700 1 -- 172.21.15.93:0/848596832 >> 172.21.15.93:0/848596832 conn(0x7fe9f80fb8e0 msgr2=0x7fe9f80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.846+0000 7fe9fcdff700 1 -- 172.21.15.93:0/848596832 shutdown_connections 2023-11-16T16:50:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.846+0000 7fe9fcdff700 1 -- 172.21.15.93:0/848596832 wait complete. 2023-11-16T16:50:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.847+0000 7fe9fcdff700 1 Processor -- start 2023-11-16T16:50:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.848+0000 7fe9fcdff700 1 -- start start 2023-11-16T16:50:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.848+0000 7fe9fcdff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9f81002c0 0x7fe9f8196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.848+0000 7fe9fcdff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9f81014b0 0x7fe9f8196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.848+0000 7fe9fcdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f8101e40 0x7fe9f819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.848+0000 7fe9fcdff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f819db90 con 0x7fe9f81002c0 2023-11-16T16:50:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.848+0000 7fe9fcdff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f819dcd0 con 0x7fe9f81014b0 2023-11-16T16:50:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.849+0000 7fe9fcdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f8072d20 con 0x7fe9f8101e40 2023-11-16T16:50:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.849+0000 7fe9f6d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f8101e40 0x7fe9f819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.849+0000 7fe9f6d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f8101e40 0x7fe9f819d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:57472/0 (socket says 172.21.15.93:57472) 2023-11-16T16:50:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.849+0000 7fe9f6d9d700 1 -- 172.21.15.93:0/1839527156 learned_addr learned my addr 172.21.15.93:0/1839527156 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.849+0000 7fe9effff700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9f81014b0 0x7fe9f8196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.849+0000 7fe9f659c700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9f81002c0 0x7fe9f8196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.850+0000 7fe9f6d9d700 1 -- 172.21.15.93:0/1839527156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9f81014b0 msgr2=0x7fe9f8196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.850+0000 7fe9f6d9d700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9f81014b0 0x7fe9f8196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.850+0000 7fe9f6d9d700 1 -- 172.21.15.93:0/1839527156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9f81002c0 msgr2=0x7fe9f8196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.850+0000 7fe9f6d9d700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9f81002c0 0x7fe9f8196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.850+0000 7fe9f6d9d700 1 -- 172.21.15.93:0/1839527156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e00095f0 con 0x7fe9f8101e40 2023-11-16T16:50:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.850+0000 7fe9f6d9d700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f8101e40 0x7fe9f819d5c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000ef30 tx=0x7fe9f000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.850+0000 7fe9ef7fe700 1 -- 172.21.15.93:0/1839527156 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0018070 con 0x7fe9f8101e40 2023-11-16T16:50:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.850+0000 7fe9fcdff700 1 -- 172.21.15.93:0/1839527156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f810bc10 con 0x7fe9f8101e40 2023-11-16T16:50:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.851+0000 7fe9ef7fe700 1 -- 172.21.15.93:0/1839527156 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f000f040 con 0x7fe9f8101e40 2023-11-16T16:50:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.851+0000 7fe9ef7fe700 1 -- 172.21.15.93:0/1839527156 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f000cdb0 con 0x7fe9f8101e40 2023-11-16T16:50:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.851+0000 7fe9fcdff700 1 -- 172.21.15.93:0/1839527156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f810ba80 con 0x7fe9f8101e40 2023-11-16T16:50:42.853 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:42.853 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.854+0000 7fe9ef7fe700 1 -- 172.21.15.93:0/1839527156 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe9f0003a00 con 0x7fe9f8101e40 2023-11-16T16:50:42.856 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:42.857 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:42.858 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.857+0000 7fe9ef7fe700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe9e40a1150 0x7fe9e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.857+0000 7fe9ef7fe700 1 -- 172.21.15.93:0/1839527156 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe9f00cc890 con 0x7fe9f8101e40 2023-11-16T16:50:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.857+0000 7fe9f659c700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe9e40a1150 0x7fe9e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:42.858+0000 7fe9f659c700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe9e40a1150 0x7fe9e40a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8101310 tx=0x7fe9e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.075+0000 7fea10694b80 1 Processor -- start 2023-11-16T16:50:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.075+0000 7fea10694b80 1 -- start start 2023-11-16T16:50:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.075+0000 7fea10694b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8ee706b0 0x557a8f014870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.075+0000 7fea10694b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8f014db0 0x557a8f0155d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.075+0000 7fea10694b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f015b10 0x557a8f015f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.075+0000 7fea10694b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8ef1a000 con 0x557a8ee706b0 2023-11-16T16:50:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.075+0000 7fea10694b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8ef6c840 con 0x557a8f015b10 2023-11-16T16:50:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.075+0000 7fea10694b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8ef65cd0 con 0x557a8f014db0 2023-11-16T16:50:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.076+0000 7fe9f659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8ee706b0 0x557a8f014870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.076+0000 7fe9f659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8ee706b0 0x557a8f014870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43878/0 (socket says 172.21.15.93:43878) 2023-11-16T16:50:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.076+0000 7fe9f659c700 1 -- 172.21.15.93:0/1564708209 learned_addr learned my addr 172.21.15.93:0/1564708209 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.076+0000 7fe9effff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8f014db0 0x557a8f0155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.076+0000 7fe9f6d9d700 1 --2- 172.21.15.93:0/1564708209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f015b10 0x557a8f015f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.076+0000 7fe9effff700 1 -- 172.21.15.93:0/1564708209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f015b10 msgr2=0x557a8f015f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.076+0000 7fe9effff700 1 --2- 172.21.15.93:0/1564708209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f015b10 0x557a8f015f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.076+0000 7fe9effff700 1 -- 172.21.15.93:0/1564708209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8ee706b0 msgr2=0x557a8f014870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.076+0000 7fe9effff700 1 --2- 172.21.15.93:0/1564708209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8ee706b0 0x557a8f014870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.076+0000 7fe9effff700 1 -- 172.21.15.93:0/1564708209 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a8ede75d0 con 0x557a8f014db0 2023-11-16T16:50:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.077+0000 7fe9effff700 1 --2- 172.21.15.93:0/1564708209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8f014db0 0x557a8f0155d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0000c00 tx=0x7fe9e0005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d13b8c21f4817b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.077+0000 7fe9ecff9700 1 -- 172.21.15.93:0/1564708209 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e001d070 con 0x557a8f014db0 2023-11-16T16:50:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.077+0000 7fe9ecff9700 1 -- 172.21.15.93:0/1564708209 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9e001e040 con 0x557a8f014db0 2023-11-16T16:50:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.077+0000 7fe9ecff9700 1 -- 172.21.15.93:0/1564708209 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0004860 con 0x557a8f014db0 2023-11-16T16:50:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.078+0000 7fea10694b80 1 -- 172.21.15.93:0/1564708209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8f014db0 msgr2=0x557a8f0155d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.078+0000 7fea10694b80 1 --2- 172.21.15.93:0/1564708209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8f014db0 0x557a8f0155d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0000c00 tx=0x7fe9e0005ea0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.078+0000 7fea10694b80 1 -- 172.21.15.93:0/1564708209 shutdown_connections 2023-11-16T16:50:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.078+0000 7fea10694b80 1 --2- 172.21.15.93:0/1564708209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8ee706b0 0x557a8f014870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.078+0000 7fea10694b80 1 --2- 172.21.15.93:0/1564708209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f015b10 0x557a8f015f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.078+0000 7fea10694b80 1 --2- 172.21.15.93:0/1564708209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8f014db0 0x557a8f0155d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.078+0000 7fea10694b80 1 -- 172.21.15.93:0/1564708209 >> 172.21.15.93:0/1564708209 conn(0x557a8f009f50 msgr2=0x557a8f00f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.078+0000 7fea10694b80 1 -- 172.21.15.93:0/1564708209 shutdown_connections 2023-11-16T16:50:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.078+0000 7fea10694b80 1 -- 172.21.15.93:0/1564708209 wait complete. 2023-11-16T16:50:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.079+0000 7fea10694b80 1 -- start start 2023-11-16T16:50:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.079+0000 7fea10694b80 1 Processor -- start 2023-11-16T16:50:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.080+0000 7fea10694b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 0x557a8f01dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.080+0000 7fea10694b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f014db0 0x557a8f01e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.080+0000 7fea10694b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f015b10 0x557a8f0251f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.080+0000 7fea10694b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8ef07880 con 0x557a8f014db0 2023-11-16T16:50:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.080+0000 7fea10694b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8ef07520 con 0x557a8f015b10 2023-11-16T16:50:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.080+0000 7fea10694b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8ede98c0 con 0x557a8ee706b0 2023-11-16T16:50:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.080+0000 7fea10694b80 10 client.0 _ll_register_callbacks cb 0x557a8ef25720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:50:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.080+0000 7fe9effff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f014db0 0x557a8f01e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.080+0000 7fe9f659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 0x557a8f01dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.080+0000 7fe9f659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 0x557a8f01dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:57496/0 (socket says 172.21.15.93:57496) 2023-11-16T16:50:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.080+0000 7fe9f659c700 1 -- 172.21.15.93:0/1341386545 learned_addr learned my addr 172.21.15.93:0/1341386545 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.080+0000 7fe9f6d9d700 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f015b10 0x557a8f0251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.081+0000 7fe9f659c700 1 -- 172.21.15.93:0/1341386545 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f015b10 msgr2=0x557a8f0251f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.081+0000 7fe9f659c700 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f015b10 0x557a8f0251f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.081+0000 7fe9f659c700 1 -- 172.21.15.93:0/1341386545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f014db0 msgr2=0x557a8f01e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.081+0000 7fe9f659c700 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f014db0 0x557a8f01e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.081+0000 7fe9f659c700 1 -- 172.21.15.93:0/1341386545 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f8109030 con 0x557a8ee706b0 2023-11-16T16:50:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.081+0000 7fe9f6d9d700 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f015b10 0x557a8f0251f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:50:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.081+0000 7fe9effff700 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f014db0 0x557a8f01e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:50:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.081+0000 7fe9f659c700 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 0x557a8f01dc70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8012c10 tx=0x7fe9e80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4733e59e23b7e6d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.081+0000 7fe9ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-11-16T16:50:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.081+0000 7fe9ddffb700 1 -- 172.21.15.93:0/1341386545 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8020840 con 0x557a8ee706b0 2023-11-16T16:50:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.082+0000 7fea10694b80 1 -- 172.21.15.93:0/1341386545 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a8f007710 con 0x557a8ee706b0 2023-11-16T16:50:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.082+0000 7fe9ddffb700 1 -- 172.21.15.93:0/1341386545 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe9e8018070 con 0x557a8ee706b0 2023-11-16T16:50:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.082+0000 7fe9ddffb700 1 -- 172.21.15.93:0/1341386545 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9e80209a0 con 0x557a8ee706b0 2023-11-16T16:50:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.082+0000 7fea10694b80 10 client.25480 fetch_fsmap learned FSMap version 23 2023-11-16T16:50:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.082+0000 7fea10694b80 1 -- 172.21.15.93:0/1341386545 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557a8ef2c130 con 0x557a8ee706b0 2023-11-16T16:50:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.082+0000 7fe9ddffb700 1 -- 172.21.15.93:0/1341386545 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a8f007710 con 0x557a8ee706b0 2023-11-16T16:50:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.082+0000 7fe9ddffb700 1 -- 172.21.15.93:0/1341386545 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe9e801fb80 con 0x557a8ee706b0 2023-11-16T16:50:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.082+0000 7fea10694b80 10 client.25480 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:50:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.083+0000 7fea10694b80 10 client.25480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:50:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.083+0000 7fea10694b80 20 client.25480 populate_metadata read hostname 'smithi093' 2023-11-16T16:50:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.083+0000 7fea10694b80 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557a8f029ad0 0x557a8f02bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.083+0000 7fea10694b80 4 client.25480 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:50:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.083+0000 7fea10694b80 1 -- 172.21.15.93:0/1341386545 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a8ef28040 con 0x557a8f029ad0 2023-11-16T16:50:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.083+0000 7fe9effff700 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557a8f029ad0 0x557a8f02bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.084+0000 7fe9effff700 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557a8f029ad0 0x557a8f02bee0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f91a3a78ed84a3a1 server_cookie=fa04919f1bda8933 in_seq=0 out_seq=0 2023-11-16T16:50:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.084+0000 7fe9ddffb700 10 client.25480 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.084+0000 7fe9ddffb700 1 -- 172.21.15.93:0/1341386545 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557a8ef28040 con 0x557a8f029ad0 2023-11-16T16:50:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.084+0000 7fe9ddffb700 10 client.25480 handle_command_reply: tid=0 2023-11-16T16:50:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fea10694b80 1 client.25480 shutdown 2023-11-16T16:50:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fea10694b80 1 -- 172.21.15.93:0/1341386545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 msgr2=0x557a8f01dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fea10694b80 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 0x557a8f01dc70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8012c10 tx=0x7fe9e80047c0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fea10694b80 1 -- 172.21.15.93:0/1341386545 shutdown_connections 2023-11-16T16:50:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fea10694b80 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557a8f029ad0 0x557a8f02bee0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fea10694b80 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f015b10 0x557a8f0251f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fea10694b80 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 0x557a8f01dc70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fea10694b80 1 --2- 172.21.15.93:0/1341386545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f014db0 0x557a8f01e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fe9ddffb700 0 client.25480 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fea10694b80 1 -- 172.21.15.93:0/1341386545 >> 172.21.15.93:0/1341386545 conn(0x557a8f009f50 msgr2=0x557a8f00e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fea10694b80 1 -- 172.21.15.93:0/1341386545 shutdown_connections 2023-11-16T16:50:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fea10694b80 1 -- 172.21.15.93:0/1341386545 wait complete. 2023-11-16T16:50:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.085+0000 7fea10694b80 20 client.25480 trim_cache size 0 max 16384 2023-11-16T16:50:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.101+0000 7fea10694b80 1 Processor -- start 2023-11-16T16:50:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.101+0000 7fea10694b80 1 -- start start 2023-11-16T16:50:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fea10694b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 0x557a8f018a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fea10694b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f014db0 0x557a8f0251b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fea10694b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 0x557a8f018f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fea10694b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8f0153c0 con 0x557a8f015b10 2023-11-16T16:50:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fea10694b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8f01bc60 con 0x557a8f014db0 2023-11-16T16:50:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fea10694b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8f019570 con 0x557a8ee706b0 2023-11-16T16:50:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fe9f6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 0x557a8f018f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fe9effff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f014db0 0x557a8f0251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fe9f6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 0x557a8f018f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43898/0 (socket says 172.21.15.93:43898) 2023-11-16T16:50:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fe9f6d9d700 1 -- 172.21.15.93:0/405289457 learned_addr learned my addr 172.21.15.93:0/405289457 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fe9f659c700 1 --2- 172.21.15.93:0/405289457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 0x557a8f018a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fe9f6d9d700 1 -- 172.21.15.93:0/405289457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 msgr2=0x557a8f018a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fe9f6d9d700 1 --2- 172.21.15.93:0/405289457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 0x557a8f018a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fe9f6d9d700 1 -- 172.21.15.93:0/405289457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f014db0 msgr2=0x557a8f0251b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.102+0000 7fe9f6d9d700 1 --2- 172.21.15.93:0/405289457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f014db0 0x557a8f0251b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.103+0000 7fe9f6d9d700 1 -- 172.21.15.93:0/405289457 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a8ef2be80 con 0x557a8f015b10 2023-11-16T16:50:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.103+0000 7fe9f659c700 1 --2- 172.21.15.93:0/405289457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 0x557a8f018a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:50:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.103+0000 7fe9f6d9d700 1 --2- 172.21.15.93:0/405289457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 0x557a8f018f40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000c7a0 tx=0x7fe9f001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfc956f0b7ec556 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.103+0000 7fe9ed7fa700 1 -- 172.21.15.93:0/405289457 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0099960 con 0x557a8f015b10 2023-11-16T16:50:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.103+0000 7fe9ed7fa700 1 -- 172.21.15.93:0/405289457 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f009d990 con 0x557a8f015b10 2023-11-16T16:50:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.103+0000 7fe9ed7fa700 1 -- 172.21.15.93:0/405289457 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00990b0 con 0x557a8f015b10 2023-11-16T16:50:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.104+0000 7fea10694b80 1 -- 172.21.15.93:0/405289457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 msgr2=0x557a8f018f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.104+0000 7fea10694b80 1 --2- 172.21.15.93:0/405289457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 0x557a8f018f40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000c7a0 tx=0x7fe9f001c000 comp rx=0 tx=0).stop 2023-11-16T16:50:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.104+0000 7fea10694b80 1 -- 172.21.15.93:0/405289457 shutdown_connections 2023-11-16T16:50:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.104+0000 7fea10694b80 1 --2- 172.21.15.93:0/405289457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 0x557a8f018f40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.104+0000 7fea10694b80 1 --2- 172.21.15.93:0/405289457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8ee706b0 0x557a8f018a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.104+0000 7fea10694b80 1 --2- 172.21.15.93:0/405289457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8f014db0 0x557a8f0251b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.104+0000 7fea10694b80 1 -- 172.21.15.93:0/405289457 >> 172.21.15.93:0/405289457 conn(0x557a8f009f50 msgr2=0x557a8f030840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.104+0000 7fea10694b80 1 -- 172.21.15.93:0/405289457 shutdown_connections 2023-11-16T16:50:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.104+0000 7fea10694b80 1 -- 172.21.15.93:0/405289457 wait complete. 2023-11-16T16:50:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.104+0000 7fea10694b80 1 -- start start 2023-11-16T16:50:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.105+0000 7fea10694b80 1 Processor -- start 2023-11-16T16:50:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.105+0000 7fea10694b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8ee706b0 0x557a8f0213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.105+0000 7fea10694b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8f014db0 0x557a8f021920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.105+0000 7fea10694b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 0x557a8f024720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.105+0000 7fea10694b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8f024c60 con 0x557a8f015b10 2023-11-16T16:50:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.105+0000 7fea10694b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8f024dd0 con 0x557a8ee706b0 2023-11-16T16:50:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.105+0000 7fea10694b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8f024f90 con 0x557a8f014db0 2023-11-16T16:50:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.105+0000 7fea10694b80 10 client.0 _ll_register_callbacks cb 0x7fe9e80127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:50:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.105+0000 7fe9f6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 0x557a8f024720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.106+0000 7fe9f6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 0x557a8f024720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43908/0 (socket says 172.21.15.93:43908) 2023-11-16T16:50:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.106+0000 7fe9f659c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8ee706b0 0x557a8f0213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.106+0000 7fe9f6d9d700 1 -- 172.21.15.93:0/4078988049 learned_addr learned my addr 172.21.15.93:0/4078988049 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.106+0000 7fe9effff700 1 --2- 172.21.15.93:0/4078988049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8f014db0 0x557a8f021920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.106+0000 7fe9f659c700 1 -- 172.21.15.93:0/4078988049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8f014db0 msgr2=0x557a8f021920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.106+0000 7fe9f659c700 1 --2- 172.21.15.93:0/4078988049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8f014db0 0x557a8f021920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.106+0000 7fe9f659c700 1 -- 172.21.15.93:0/4078988049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 msgr2=0x557a8f024720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.106+0000 7fe9f659c700 1 --2- 172.21.15.93:0/4078988049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 0x557a8f024720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.106+0000 7fe9f659c700 1 -- 172.21.15.93:0/4078988049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a8ef2c130 con 0x557a8ee706b0 2023-11-16T16:50:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.106+0000 7fe9f659c700 1 --2- 172.21.15.93:0/4078988049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8ee706b0 0x557a8f0213e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe9e801ff10 tx=0x7fe9e8005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11ad2fa6bb9eec4 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.106+0000 7fe9ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:50:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.107+0000 7fe9ddffb700 1 -- 172.21.15.93:0/4078988049 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8009920 con 0x557a8ee706b0 2023-11-16T16:50:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.107+0000 7fe9ddffb700 1 -- 172.21.15.93:0/4078988049 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe9e8018040 con 0x557a8ee706b0 2023-11-16T16:50:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.107+0000 7fea10694b80 1 -- 172.21.15.93:0/4078988049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9e001e040 con 0x557a8ee706b0 2023-11-16T16:50:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.107+0000 7fe9ddffb700 1 -- 172.21.15.93:0/4078988049 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9e801e520 con 0x557a8ee706b0 2023-11-16T16:50:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.107+0000 7fea10694b80 10 client.25493 fetch_fsmap learned FSMap version 23 2023-11-16T16:50:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.107+0000 7fea10694b80 1 -- 172.21.15.93:0/4078988049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fe9f810ba80 con 0x557a8ee706b0 2023-11-16T16:50:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.107+0000 7fe9ddffb700 1 -- 172.21.15.93:0/4078988049 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9e001e040 con 0x557a8ee706b0 2023-11-16T16:50:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.108+0000 7fe9ddffb700 1 -- 172.21.15.93:0/4078988049 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe9e801f430 con 0x557a8ee706b0 2023-11-16T16:50:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.108+0000 7fea10694b80 10 client.25493 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:50:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.108+0000 7fea10694b80 10 client.25493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:50:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.108+0000 7fea10694b80 20 client.25493 populate_metadata read hostname 'smithi093' 2023-11-16T16:50:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.108+0000 7fea10694b80 1 --2- 172.21.15.93:0/4078988049 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557a8f07eb60 0x557a8f080f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.108+0000 7fea10694b80 4 client.25493 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:50:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.108+0000 7fea10694b80 1 -- 172.21.15.93:0/4078988049 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a8ef34600 con 0x557a8f07eb60 2023-11-16T16:50:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.108+0000 7fe9effff700 1 --2- 172.21.15.93:0/4078988049 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557a8f07eb60 0x557a8f080f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.109+0000 7fe9ddffb700 10 client.25493 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.109+0000 7fe9effff700 1 --2- 172.21.15.93:0/4078988049 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557a8f07eb60 0x557a8f080f70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e4ebab46c69e1f server_cookie=7997823cad72a19a in_seq=0 out_seq=0 2023-11-16T16:50:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.133+0000 7fe9ddffb700 1 -- 172.21.15.93:0/4078988049 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a8ef34600 con 0x557a8f07eb60 2023-11-16T16:50:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.133+0000 7fe9ddffb700 10 client.25493 handle_command_reply: tid=0 2023-11-16T16:50:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.133+0000 7fea10694b80 1 client.25493 shutdown 2023-11-16T16:50:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.134+0000 7fea10694b80 1 -- 172.21.15.93:0/4078988049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8ee706b0 msgr2=0x557a8f0213e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.134+0000 7fea10694b80 1 --2- 172.21.15.93:0/4078988049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8ee706b0 0x557a8f0213e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe9e801ff10 tx=0x7fe9e8005bb0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.134+0000 7fea10694b80 1 -- 172.21.15.93:0/4078988049 shutdown_connections 2023-11-16T16:50:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.134+0000 7fea10694b80 1 --2- 172.21.15.93:0/4078988049 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557a8f07eb60 0x557a8f080f70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.134+0000 7fea10694b80 1 --2- 172.21.15.93:0/4078988049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a8f015b10 0x557a8f024720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.134+0000 7fea10694b80 1 --2- 172.21.15.93:0/4078988049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a8ee706b0 0x557a8f0213e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.134+0000 7fe9ddffb700 0 client.25493 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.134+0000 7fea10694b80 1 --2- 172.21.15.93:0/4078988049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a8f014db0 0x557a8f021920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.134+0000 7fea10694b80 1 -- 172.21.15.93:0/4078988049 >> 172.21.15.93:0/4078988049 conn(0x557a8f009f50 msgr2=0x557a8f018a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.134+0000 7fea10694b80 1 -- 172.21.15.93:0/4078988049 shutdown_connections 2023-11-16T16:50:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.134+0000 7fea10694b80 1 -- 172.21.15.93:0/4078988049 wait complete. 2023-11-16T16:50:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.134+0000 7fea10694b80 20 client.25493 trim_cache size 0 max 16384 2023-11-16T16:50:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:50:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:50:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:50:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:50:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.135+0000 7fe9fcdff700 1 -- 172.21.15.93:0/1839527156 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe9e40a1150 msgr2=0x7fe9e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.135+0000 7fe9fcdff700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe9e40a1150 0x7fe9e40a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8101310 tx=0x7fe9e8009260 comp rx=0 tx=0).stop 2023-11-16T16:50:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.135+0000 7fe9fcdff700 1 -- 172.21.15.93:0/1839527156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f8101e40 msgr2=0x7fe9f819d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.135+0000 7fe9fcdff700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f8101e40 0x7fe9f819d5c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000ef30 tx=0x7fe9f000c3a0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.135+0000 7fe9fcdff700 1 -- 172.21.15.93:0/1839527156 shutdown_connections 2023-11-16T16:50:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.135+0000 7fe9fcdff700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe9e40a1150 0x7fe9e40a3600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.135+0000 7fe9fcdff700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9f81002c0 0x7fe9f8196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.135+0000 7fe9fcdff700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f8101e40 0x7fe9f819d5c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.135+0000 7fe9fcdff700 1 --2- 172.21.15.93:0/1839527156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9f81014b0 0x7fe9f8196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.135+0000 7fe9fcdff700 1 -- 172.21.15.93:0/1839527156 >> 172.21.15.93:0/1839527156 conn(0x7fe9f80fb8e0 msgr2=0x7fe9f80fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.135+0000 7fe9fcdff700 1 -- 172.21.15.93:0/1839527156 shutdown_connections 2023-11-16T16:50:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:43.135+0000 7fe9fcdff700 1 -- 172.21.15.93:0/1839527156 wait complete. 2023-11-16T16:50:43.501 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:43.502 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:43.657 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:43.657 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:43.666 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:43.667 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:43.667 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 57 -lt 100 ']' 2023-11-16T16:50:43.667 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 57 + 1 2023-11-16T16:50:43.688 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=58 2023-11-16T16:50:43.688 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:43.721 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:43.721 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:43.724 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:43.724 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:43.724 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:44.101 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:44.101 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:44.245 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:44.245 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:44.266 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:44.266 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:44.266 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 58 -lt 100 ']' 2023-11-16T16:50:44.266 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 58 + 1 2023-11-16T16:50:44.287 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=59 2023-11-16T16:50:44.287 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:44.328 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:44.328 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:44.331 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:44.331 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:44.332 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:44.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:44 smithi093 ceph-mon[97863]: pgmap v408: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 9 op/s 2023-11-16T16:50:44.734 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:44 smithi159 ceph-mon[131528]: pgmap v408: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 9 op/s 2023-11-16T16:50:44.784 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:44.785 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:44.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:44 smithi125 ceph-mon[131076]: pgmap v408: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 9 op/s 2023-11-16T16:50:44.928 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:44.928 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:44.950 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:44.950 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:44.950 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 59 -lt 100 ']' 2023-11-16T16:50:44.951 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 59 + 1 2023-11-16T16:50:44.976 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=60 2023-11-16T16:50:44.976 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:45.000 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:45.000 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:45.003 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:45.004 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:45.004 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:45.401 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:45.401 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:45.557 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:45.557 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:45.566 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:45.566 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:45.566 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 60 -lt 100 ']' 2023-11-16T16:50:45.566 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 60 + 1 2023-11-16T16:50:45.595 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=61 2023-11-16T16:50:45.596 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:45.626 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:45.626 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:45.629 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:45.630 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:45.630 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:46.018 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:46.018 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:46.178 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:46.178 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:46.192 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:46.192 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:46.192 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 61 -lt 100 ']' 2023-11-16T16:50:46.193 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 61 + 1 2023-11-16T16:50:46.212 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=62 2023-11-16T16:50:46.212 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:46.242 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:46.242 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:46.245 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:46.246 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:46.246 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:46.634 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:46.635 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:46.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:46 smithi159 ceph-mon[131528]: pgmap v409: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-11-16T16:50:46.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:46 smithi125 ceph-mon[131076]: pgmap v409: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-11-16T16:50:46.828 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:46.828 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:46.843 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:46.843 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:46.843 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 62 -lt 100 ']' 2023-11-16T16:50:46.843 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 62 + 1 2023-11-16T16:50:46.887 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=63 2023-11-16T16:50:46.887 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:46.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:46 smithi093 ceph-mon[97863]: pgmap v409: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-11-16T16:50:46.918 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:46.918 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:46.921 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:46.921 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:46.922 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:47.301 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:47.301 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:47.517 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:47.517 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:47.533 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:47.534 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:47.534 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 63 -lt 100 ']' 2023-11-16T16:50:47.534 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 63 + 1 2023-11-16T16:50:47.568 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=64 2023-11-16T16:50:47.568 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:47.606 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:47.607 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:47.609 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:47.610 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:47.611 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:48.050 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:48.051 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:48.186 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:48.186 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:48.199 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:48.199 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:48.199 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 64 -lt 100 ']' 2023-11-16T16:50:48.200 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 64 + 1 2023-11-16T16:50:48.220 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=65 2023-11-16T16:50:48.220 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:48.249 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:48.250 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:48.254 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:48.254 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:48.255 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:48.659 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:48.659 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:48.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:48 smithi159 ceph-mon[131528]: pgmap v410: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-11-16T16:50:48.817 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:48.817 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:48.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:48 smithi125 ceph-mon[131076]: pgmap v410: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-11-16T16:50:48.833 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:48.833 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:48.833 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 65 -lt 100 ']' 2023-11-16T16:50:48.834 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 65 + 1 2023-11-16T16:50:48.853 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=66 2023-11-16T16:50:48.854 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:48.886 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:48.887 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:48.889 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:48.890 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:48.890 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:48.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:48 smithi093 ceph-mon[97863]: pgmap v410: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-11-16T16:50:49.367 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:49.367 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:49.511 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:49.511 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:49.524 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:49.524 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:49.524 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 66 -lt 100 ']' 2023-11-16T16:50:49.525 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 66 + 1 2023-11-16T16:50:49.545 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=67 2023-11-16T16:50:49.545 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:49.576 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:49.576 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:49.579 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:49.580 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:49.580 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:50.075 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:50.076 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:50.219 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:50.219 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:50.233 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:50.233 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:50.233 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 67 -lt 100 ']' 2023-11-16T16:50:50.234 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 67 + 1 2023-11-16T16:50:50.253 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=68 2023-11-16T16:50:50.254 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:50.283 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:50.284 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:50.287 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:50.287 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:50.288 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:50.692 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:50.692 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:50.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:50 smithi159 ceph-mon[131528]: pgmap v411: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-11-16T16:50:50.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:50 smithi125 ceph-mon[131076]: pgmap v411: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-11-16T16:50:50.852 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:50.852 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:50.865 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:50.865 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:50.865 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 68 -lt 100 ']' 2023-11-16T16:50:50.866 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 68 + 1 2023-11-16T16:50:50.895 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=69 2023-11-16T16:50:50.895 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:50.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:50 smithi093 ceph-mon[97863]: pgmap v411: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-11-16T16:50:50.931 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:50.932 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:50.934 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:50.935 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:50.935 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:51.375 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:51.375 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:51.527 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:51.527 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:51.550 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:51.550 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:51.550 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 69 -lt 100 ']' 2023-11-16T16:50:51.551 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 69 + 1 2023-11-16T16:50:51.570 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=70 2023-11-16T16:50:51.570 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:51.599 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:51.600 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:51.602 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:51.603 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:51.603 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:52.025 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:52.025 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:52.160 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:52.160 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:52.182 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:52.182 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:52.182 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 70 -lt 100 ']' 2023-11-16T16:50:52.183 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 70 + 1 2023-11-16T16:50:52.203 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=71 2023-11-16T16:50:52.203 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:52.233 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:52.234 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:52.236 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:52.237 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:52.237 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:52.641 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:52.642 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:52.734 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:52 smithi159 ceph-mon[131528]: pgmap v412: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 16 op/s 2023-11-16T16:50:52.826 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:52.827 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:52.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:52 smithi125 ceph-mon[131076]: pgmap v412: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 16 op/s 2023-11-16T16:50:52.842 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:52.842 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:52.842 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 71 -lt 100 ']' 2023-11-16T16:50:52.842 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 71 + 1 2023-11-16T16:50:52.883 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=72 2023-11-16T16:50:52.883 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:52.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:52 smithi093 ceph-mon[97863]: pgmap v412: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 16 op/s 2023-11-16T16:50:52.944 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:52.945 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:52.947 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:52.948 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:52.948 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:53.425 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:53.425 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:53.560 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:53.560 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:53.573 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:53.573 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:53.573 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 72 -lt 100 ']' 2023-11-16T16:50:53.574 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 72 + 1 2023-11-16T16:50:53.594 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=73 2023-11-16T16:50:53.594 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:53.623 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:53.624 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:53.626 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:53.627 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:53.627 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:54.024 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:54.025 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:54.143 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:54.143 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:54.158 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:54.158 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:54.158 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 73 -lt 100 ']' 2023-11-16T16:50:54.159 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 73 + 1 2023-11-16T16:50:54.183 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=74 2023-11-16T16:50:54.183 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:54.210 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:54.211 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:54.213 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:54.214 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:54.214 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:54.566 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:54.567 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:54.718 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:54.718 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:54.731 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:54.731 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:54.731 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 74 -lt 100 ']' 2023-11-16T16:50:54.732 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 74 + 1 2023-11-16T16:50:54.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:54 smithi159 ceph-mon[131528]: pgmap v413: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2023-11-16T16:50:54.753 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=75 2023-11-16T16:50:54.753 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:54.784 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:54.785 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:54.787 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:54.787 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:54.788 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:54.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:54 smithi125 ceph-mon[131076]: pgmap v413: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2023-11-16T16:50:54.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:54 smithi093 ceph-mon[97863]: pgmap v413: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 15 op/s 2023-11-16T16:50:55.316 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:55.316 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:55.476 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:55.476 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:55.491 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:55.491 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:55.492 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 75 -lt 100 ']' 2023-11-16T16:50:55.492 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 75 + 1 2023-11-16T16:50:55.511 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=76 2023-11-16T16:50:55.511 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:55.543 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:55.544 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:55.546 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:55.547 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:55.547 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:55.966 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:55.966 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:56.093 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:56.093 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:56.115 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:56.115 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:56.115 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 76 -lt 100 ']' 2023-11-16T16:50:56.116 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 76 + 1 2023-11-16T16:50:56.135 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=77 2023-11-16T16:50:56.136 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:56.167 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:56.168 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:56.170 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:56.171 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:56.171 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:56.657 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:56.658 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:56.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:56 smithi159 ceph-mon[131528]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 16 op/s 2023-11-16T16:50:56.793 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:56.793 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:56.815 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:56.815 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:56.815 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 77 -lt 100 ']' 2023-11-16T16:50:56.816 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 77 + 1 2023-11-16T16:50:56.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:56 smithi125 ceph-mon[131076]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 16 op/s 2023-11-16T16:50:56.849 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=78 2023-11-16T16:50:56.849 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:56.892 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:56.893 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:56.895 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:56.896 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:56.896 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:56.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:56 smithi093 ceph-mon[97863]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 16 op/s 2023-11-16T16:50:57.716 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:57.716 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:57.968 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:57.968 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:58.007 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:58.007 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:58.007 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 78 -lt 100 ']' 2023-11-16T16:50:58.008 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 78 + 1 2023-11-16T16:50:58.049 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=79 2023-11-16T16:50:58.049 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:58.110 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:58.110 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:58.113 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:58.113 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:58.114 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.409+0000 7f39eb9ff700 1 -- 172.21.15.93:0/223609555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e41035e0 msgr2=0x7f39e41059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.409+0000 7f39eb9ff700 1 --2- 172.21.15.93:0/223609555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e41035e0 0x7f39e41059c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009990 tx=0x7f39d4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.409+0000 7f39eb9ff700 1 -- 172.21.15.93:0/223609555 shutdown_connections 2023-11-16T16:50:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.409+0000 7f39eb9ff700 1 --2- 172.21.15.93:0/223609555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e4105f00 0x7f39e4108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.409+0000 7f39eb9ff700 1 --2- 172.21.15.93:0/223609555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e4100cc0 0x7f39e41030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.409+0000 7f39eb9ff700 1 --2- 172.21.15.93:0/223609555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e41035e0 0x7f39e41059c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.409+0000 7f39eb9ff700 1 -- 172.21.15.93:0/223609555 >> 172.21.15.93:0/223609555 conn(0x7f39e40fa720 msgr2=0x7f39e40fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.409+0000 7f39eb9ff700 1 -- 172.21.15.93:0/223609555 shutdown_connections 2023-11-16T16:50:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.409+0000 7f39eb9ff700 1 -- 172.21.15.93:0/223609555 wait complete. 2023-11-16T16:50:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.410+0000 7f39eb9ff700 1 Processor -- start 2023-11-16T16:50:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.411+0000 7f39eb9ff700 1 -- start start 2023-11-16T16:50:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.411+0000 7f39eb9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e4100cc0 0x7f39e4198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.411+0000 7f39eb9ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e41035e0 0x7f39e4198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.411+0000 7f39eb9ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4105f00 0x7f39e419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.411+0000 7f39eb9ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e419fda0 con 0x7f39e4105f00 2023-11-16T16:50:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.411+0000 7f39eb9ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e419fee0 con 0x7f39e41035e0 2023-11-16T16:50:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.411+0000 7f39eb9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e41a0070 con 0x7f39e4100cc0 2023-11-16T16:50:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.411+0000 7f39e9f9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4105f00 0x7f39e419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.411+0000 7f39e979b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e4100cc0 0x7f39e4198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.411+0000 7f39e9f9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4105f00 0x7f39e419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35484/0 (socket says 172.21.15.93:35484) 2023-11-16T16:50:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.411+0000 7f39e9f9c700 1 -- 172.21.15.93:0/682966527 learned_addr learned my addr 172.21.15.93:0/682966527 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.411+0000 7f39e8f9a700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e41035e0 0x7f39e4198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39e9f9c700 1 -- 172.21.15.93:0/682966527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e4100cc0 msgr2=0x7f39e4198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39e9f9c700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e4100cc0 0x7f39e4198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39e9f9c700 1 -- 172.21.15.93:0/682966527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e41035e0 msgr2=0x7f39e4198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39e9f9c700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e41035e0 0x7f39e4198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39e9f9c700 1 -- 172.21.15.93:0/682966527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d40095f0 con 0x7f39e4105f00 2023-11-16T16:50:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39e979b700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e4100cc0 0x7f39e4198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:50:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39e8f9a700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e41035e0 0x7f39e4198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:50:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39e9f9c700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4105f00 0x7f39e419f7d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00e970 tx=0x7f39dc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39da7fc700 1 -- 172.21.15.93:0/682966527 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc009760 con 0x7f39e4105f00 2023-11-16T16:50:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39da7fc700 1 -- 172.21.15.93:0/682966527 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39dc00ce20 con 0x7f39e4105f00 2023-11-16T16:50:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39eb9ff700 1 -- 172.21.15.93:0/682966527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e410bc50 con 0x7f39e4105f00 2023-11-16T16:50:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39da7fc700 1 -- 172.21.15.93:0/682966527 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc018420 con 0x7f39e4105f00 2023-11-16T16:50:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.412+0000 7f39eb9ff700 1 -- 172.21.15.93:0/682966527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e410bac0 con 0x7f39e4105f00 2023-11-16T16:50:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.415+0000 7f39da7fc700 1 -- 172.21.15.93:0/682966527 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f39dc0106c0 con 0x7f39e4105f00 2023-11-16T16:50:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.417+0000 7f39da7fc700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f39d00a1150 0x7f39d00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.417+0000 7f39da7fc700 1 -- 172.21.15.93:0/682966527 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f39dc014070 con 0x7f39e4105f00 2023-11-16T16:50:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.417+0000 7f39e979b700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f39d00a1150 0x7f39d00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.418+0000 7f39e979b700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f39d00a1150 0x7f39d00a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009480 tx=0x7f39e000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.476+0000 7f39ff2b1b80 1 Processor -- start 2023-11-16T16:50:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.477+0000 7f39ff2b1b80 1 -- start start 2023-11-16T16:50:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.477+0000 7f39ff2b1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67b2c6b0 0x562d67cd0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.477+0000 7f39ff2b1b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67cd0db0 0x562d67cd15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.477+0000 7f39ff2b1b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd1b10 0x562d67cd1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.477+0000 7f39ff2b1b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d67bd6000 con 0x562d67cd1b10 2023-11-16T16:50:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.477+0000 7f39ff2b1b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d67c28840 con 0x562d67cd0db0 2023-11-16T16:50:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.477+0000 7f39ff2b1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d67c21cd0 con 0x562d67b2c6b0 2023-11-16T16:50:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.477+0000 7f39e9f9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd1b10 0x562d67cd1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.477+0000 7f39e9f9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd1b10 0x562d67cd1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35488/0 (socket says 172.21.15.93:35488) 2023-11-16T16:50:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.477+0000 7f39e9f9c700 1 -- 172.21.15.93:0/3772802234 learned_addr learned my addr 172.21.15.93:0/3772802234 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.477+0000 7f39e8f9a700 1 --2- 172.21.15.93:0/3772802234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67cd0db0 0x562d67cd15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.478+0000 7f39e979b700 1 --2- 172.21.15.93:0/3772802234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67b2c6b0 0x562d67cd0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.478+0000 7f39e9f9c700 1 -- 172.21.15.93:0/3772802234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67b2c6b0 msgr2=0x562d67cd0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.478+0000 7f39e9f9c700 1 --2- 172.21.15.93:0/3772802234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67b2c6b0 0x562d67cd0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.478+0000 7f39e9f9c700 1 -- 172.21.15.93:0/3772802234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67cd0db0 msgr2=0x562d67cd15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.478+0000 7f39e9f9c700 1 --2- 172.21.15.93:0/3772802234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67cd0db0 0x562d67cd15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.478+0000 7f39e9f9c700 1 -- 172.21.15.93:0/3772802234 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d67aa35d0 con 0x562d67cd1b10 2023-11-16T16:50:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.478+0000 7f39e979b700 1 --2- 172.21.15.93:0/3772802234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67b2c6b0 0x562d67cd0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:50:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.478+0000 7f39e9f9c700 1 --2- 172.21.15.93:0/3772802234 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd1b10 0x562d67cd1f80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0a1dd0 tx=0x7f39dc010ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7d34bace3061b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.478+0000 7f39cf7fe700 1 -- 172.21.15.93:0/3772802234 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc09dee0 con 0x562d67cd1b10 2023-11-16T16:50:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.478+0000 7f39cf7fe700 1 -- 172.21.15.93:0/3772802234 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39dc0a2300 con 0x562d67cd1b10 2023-11-16T16:50:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.478+0000 7f39cf7fe700 1 -- 172.21.15.93:0/3772802234 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0a20d0 con 0x562d67cd1b10 2023-11-16T16:50:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.479+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/3772802234 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd1b10 msgr2=0x562d67cd1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.479+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/3772802234 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd1b10 0x562d67cd1f80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0a1dd0 tx=0x7f39dc010ac0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.479+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/3772802234 shutdown_connections 2023-11-16T16:50:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.479+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/3772802234 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd1b10 0x562d67cd1f80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.479+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/3772802234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67b2c6b0 0x562d67cd0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.479+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/3772802234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67cd0db0 0x562d67cd15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.479+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/3772802234 >> 172.21.15.93:0/3772802234 conn(0x562d67cc5f50 msgr2=0x562d67ccb100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.479+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/3772802234 shutdown_connections 2023-11-16T16:50:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.479+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/3772802234 wait complete. 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.480+0000 7f39ff2b1b80 1 -- start start 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.480+0000 7f39ff2b1b80 1 Processor -- start 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.480+0000 7f39ff2b1b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67b2c6b0 0x562d67cd9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.480+0000 7f39ff2b1b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 0x562d67cda210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.480+0000 7f39ff2b1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67ce1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.480+0000 7f39ff2b1b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d67bc3880 con 0x562d67cd0db0 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.480+0000 7f39ff2b1b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d67bc3520 con 0x562d67b2c6b0 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.480+0000 7f39ff2b1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d67aa58c0 con 0x562d67cd1b10 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.481+0000 7f39ff2b1b80 10 client.0 _ll_register_callbacks cb 0x562d67be1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.481+0000 7f39e8f9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 0x562d67cda210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.481+0000 7f39e979b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67b2c6b0 0x562d67cd9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.481+0000 7f39e8f9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 0x562d67cda210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35504/0 (socket says 172.21.15.93:35504) 2023-11-16T16:50:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.481+0000 7f39e8f9a700 1 -- 172.21.15.93:0/1095272480 learned_addr learned my addr 172.21.15.93:0/1095272480 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.481+0000 7f39e9f9c700 1 --2- 172.21.15.93:0/1095272480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67ce1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.481+0000 7f39e979b700 1 -- 172.21.15.93:0/1095272480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 msgr2=0x562d67ce1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.481+0000 7f39e979b700 1 --2- 172.21.15.93:0/1095272480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67ce1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.481+0000 7f39e979b700 1 -- 172.21.15.93:0/1095272480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 msgr2=0x562d67cda210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.481+0000 7f39e979b700 1 --2- 172.21.15.93:0/1095272480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 0x562d67cda210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.481+0000 7f39e979b700 1 -- 172.21.15.93:0/1095272480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e410bac0 con 0x562d67b2c6b0 2023-11-16T16:50:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.482+0000 7f39e979b700 1 --2- 172.21.15.93:0/1095272480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67b2c6b0 0x562d67cd9cd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f39e0007b70 tx=0x7f39e0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed18c48835bf74f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.482+0000 7f39ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:50:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.482+0000 7f39ccff9700 1 -- 172.21.15.93:0/1095272480 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0015610 con 0x562d67b2c6b0 2023-11-16T16:50:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.482+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1095272480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d67cc3710 con 0x562d67b2c6b0 2023-11-16T16:50:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.482+0000 7f39ccff9700 1 -- 172.21.15.93:0/1095272480 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f39e001c070 con 0x562d67b2c6b0 2023-11-16T16:50:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.482+0000 7f39ccff9700 1 -- 172.21.15.93:0/1095272480 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39e0017e80 con 0x562d67b2c6b0 2023-11-16T16:50:58.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.482+0000 7f39ff2b1b80 10 client.25505 fetch_fsmap learned FSMap version 23 2023-11-16T16:50:58.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.482+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1095272480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562d67be8130 con 0x562d67b2c6b0 2023-11-16T16:50:58.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.482+0000 7f39ccff9700 1 -- 172.21.15.93:0/1095272480 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d67cc3710 con 0x562d67b2c6b0 2023-11-16T16:50:58.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.483+0000 7f39ccff9700 1 -- 172.21.15.93:0/1095272480 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39e0022870 con 0x562d67b2c6b0 2023-11-16T16:50:58.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.483+0000 7f39ff2b1b80 10 client.25505 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:50:58.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.483+0000 7f39ff2b1b80 10 client.25505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:50:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.483+0000 7f39ff2b1b80 20 client.25505 populate_metadata read hostname 'smithi093' 2023-11-16T16:50:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.483+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1095272480 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x562d67ce5b30 0x562d67ce7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.483+0000 7f39ff2b1b80 4 client.25505 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:50:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.483+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1095272480 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d67be4040 con 0x562d67ce5b30 2023-11-16T16:50:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.483+0000 7f39e8f9a700 1 --2- 172.21.15.93:0/1095272480 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x562d67ce5b30 0x562d67ce7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.484+0000 7f39ccff9700 10 client.25505 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.484+0000 7f39e8f9a700 1 --2- 172.21.15.93:0/1095272480 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x562d67ce5b30 0x562d67ce7f40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae648d4f5d63478e server_cookie=b00b00460454a800 in_seq=0 out_seq=0 2023-11-16T16:50:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ccff9700 1 -- 172.21.15.93:0/1095272480 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562d67be4040 con 0x562d67ce5b30 2023-11-16T16:50:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ccff9700 10 client.25505 handle_command_reply: tid=0 2023-11-16T16:50:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ff2b1b80 1 client.25505 shutdown 2023-11-16T16:50:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1095272480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67b2c6b0 msgr2=0x562d67cd9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1095272480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67b2c6b0 0x562d67cd9cd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f39e0007b70 tx=0x7f39e0004ad0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1095272480 shutdown_connections 2023-11-16T16:50:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1095272480 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x562d67ce5b30 0x562d67ce7f40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1095272480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67ce1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1095272480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67b2c6b0 0x562d67cd9cd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1095272480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 0x562d67cda210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1095272480 >> 172.21.15.93:0/1095272480 conn(0x562d67cc5f50 msgr2=0x562d67cca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ccff9700 0 client.25505 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1095272480 shutdown_connections 2023-11-16T16:50:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.485+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1095272480 wait complete. 2023-11-16T16:50:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.486+0000 7f39ff2b1b80 20 client.25505 trim_cache size 0 max 16384 2023-11-16T16:50:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.501+0000 7f39ff2b1b80 1 Processor -- start 2023-11-16T16:50:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.501+0000 7f39ff2b1b80 1 -- start start 2023-11-16T16:50:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.501+0000 7f39ff2b1b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67b2c6b0 0x562d67cd4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.501+0000 7f39ff2b1b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67cd0db0 0x562d67cd4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.501+0000 7f39ff2b1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67ce1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.501+0000 7f39ff2b1b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d67cd1420 con 0x562d67b2c6b0 2023-11-16T16:50:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.501+0000 7f39ff2b1b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d67ccbc50 con 0x562d67cd0db0 2023-11-16T16:50:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.501+0000 7f39ff2b1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d67cdace0 con 0x562d67cd1b10 2023-11-16T16:50:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.501+0000 7f39e979b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67b2c6b0 0x562d67cd4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.501+0000 7f39e979b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67b2c6b0 0x562d67cd4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35512/0 (socket says 172.21.15.93:35512) 2023-11-16T16:50:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.501+0000 7f39e979b700 1 -- 172.21.15.93:0/1782614456 learned_addr learned my addr 172.21.15.93:0/1782614456 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.501+0000 7f39e8f9a700 1 --2- 172.21.15.93:0/1782614456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67cd0db0 0x562d67cd4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.502+0000 7f39e9f9c700 1 --2- 172.21.15.93:0/1782614456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67ce1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.502+0000 7f39e979b700 1 -- 172.21.15.93:0/1782614456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 msgr2=0x562d67ce1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.502+0000 7f39e979b700 1 --2- 172.21.15.93:0/1782614456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67ce1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.502+0000 7f39e979b700 1 -- 172.21.15.93:0/1782614456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67cd0db0 msgr2=0x562d67cd4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.502+0000 7f39e979b700 1 --2- 172.21.15.93:0/1782614456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67cd0db0 0x562d67cd4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.502+0000 7f39e979b700 1 -- 172.21.15.93:0/1782614456 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d67be7e80 con 0x562d67b2c6b0 2023-11-16T16:50:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.502+0000 7f39e979b700 1 --2- 172.21.15.93:0/1782614456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67b2c6b0 0x562d67cd4a20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f39e0004aa0 tx=0x7f39e000e4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2064dc28fb1254 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.502+0000 7f39cffff700 1 -- 172.21.15.93:0/1782614456 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0022600 con 0x562d67b2c6b0 2023-11-16T16:50:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.502+0000 7f39cffff700 1 -- 172.21.15.93:0/1782614456 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39e000f040 con 0x562d67b2c6b0 2023-11-16T16:50:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.502+0000 7f39cffff700 1 -- 172.21.15.93:0/1782614456 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0014b60 con 0x562d67b2c6b0 2023-11-16T16:50:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.503+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1782614456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67b2c6b0 msgr2=0x562d67cd4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.503+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1782614456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67b2c6b0 0x562d67cd4a20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f39e0004aa0 tx=0x7f39e000e4f0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.503+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1782614456 shutdown_connections 2023-11-16T16:50:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.503+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1782614456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67b2c6b0 0x562d67cd4a20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.503+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1782614456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67ce1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.503+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1782614456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67cd0db0 0x562d67cd4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.503+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1782614456 >> 172.21.15.93:0/1782614456 conn(0x562d67cc5f50 msgr2=0x562d67cd89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.503+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1782614456 shutdown_connections 2023-11-16T16:50:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.503+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1782614456 wait complete. 2023-11-16T16:50:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.503+0000 7f39ff2b1b80 1 -- start start 2023-11-16T16:50:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39ff2b1b80 1 Processor -- start 2023-11-16T16:50:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39ff2b1b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67b2c6b0 0x562d67ce0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39ff2b1b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 0x562d67ce0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39ff2b1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67cdd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39ff2b1b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d67cdda40 con 0x562d67cd0db0 2023-11-16T16:50:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39ff2b1b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d67cddb80 con 0x562d67b2c6b0 2023-11-16T16:50:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39ff2b1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d67cddd40 con 0x562d67cd1b10 2023-11-16T16:50:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39ff2b1b80 10 client.0 _ll_register_callbacks cb 0x7f39e000ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:50:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39e8f9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 0x562d67ce0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39e8f9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 0x562d67ce0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35528/0 (socket says 172.21.15.93:35528) 2023-11-16T16:50:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39e8f9a700 1 -- 172.21.15.93:0/1331007275 learned_addr learned my addr 172.21.15.93:0/1331007275 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:50:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39e979b700 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67b2c6b0 0x562d67ce0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.504+0000 7f39e9f9c700 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67cdd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.505+0000 7f39e8f9a700 1 -- 172.21.15.93:0/1331007275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 msgr2=0x562d67cdd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.505+0000 7f39e8f9a700 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67cdd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.505+0000 7f39e8f9a700 1 -- 172.21.15.93:0/1331007275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67b2c6b0 msgr2=0x562d67ce0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:50:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.505+0000 7f39e8f9a700 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67b2c6b0 0x562d67ce0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.505+0000 7f39e8f9a700 1 -- 172.21.15.93:0/1331007275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d67be8130 con 0x562d67cd0db0 2023-11-16T16:50:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.505+0000 7f39e9f9c700 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67cdd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:50:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.505+0000 7f39e8f9a700 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 0x562d67ce0fa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f39d40095c0 tx=0x7f39d4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232bc4dc106422f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.505+0000 7f39ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:50:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.505+0000 7f39ccff9700 1 -- 172.21.15.93:0/1331007275 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d401d040 con 0x562d67cd0db0 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.505+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1331007275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39dc0a2300 con 0x562d67cd0db0 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.505+0000 7f39ccff9700 1 -- 172.21.15.93:0/1331007275 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f39d4005260 con 0x562d67cd0db0 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.506+0000 7f39ccff9700 1 -- 172.21.15.93:0/1331007275 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39d4005420 con 0x562d67cd0db0 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.506+0000 7f39ff2b1b80 10 client.15474 fetch_fsmap learned FSMap version 23 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.506+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1331007275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562d67a38c00 con 0x562d67cd0db0 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.506+0000 7f39ccff9700 1 -- 172.21.15.93:0/1331007275 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39dc0a2300 con 0x562d67cd0db0 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.506+0000 7f39ccff9700 1 -- 172.21.15.93:0/1331007275 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39d402f020 con 0x562d67cd0db0 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.506+0000 7f39ff2b1b80 10 client.15474 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.506+0000 7f39ff2b1b80 10 client.15474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.506+0000 7f39ff2b1b80 20 client.15474 populate_metadata read hostname 'smithi093' 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.506+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x562d67d3add0 0x562d67d3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.506+0000 7f39ff2b1b80 4 client.15474 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.506+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1331007275 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562d67bf0600 con 0x562d67d3add0 2023-11-16T16:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.506+0000 7f39e979b700 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x562d67d3add0 0x562d67d3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.507+0000 7f39ccff9700 10 client.15474 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.507+0000 7f39e979b700 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x562d67d3add0 0x562d67d3d1e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15831e8dd089d6c4 server_cookie=d76fd45f8003af94 in_seq=0 out_seq=0 2023-11-16T16:50:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ccff9700 1 -- 172.21.15.93:0/1331007275 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562d67bf0600 con 0x562d67d3add0 2023-11-16T16:50:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ccff9700 10 client.15474 handle_command_reply: tid=0 2023-11-16T16:50:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ff2b1b80 1 client.15474 shutdown 2023-11-16T16:50:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1331007275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 msgr2=0x562d67ce0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 0x562d67ce0fa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f39d40095c0 tx=0x7f39d4005040 comp rx=0 tx=0).stop 2023-11-16T16:50:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1331007275 shutdown_connections 2023-11-16T16:50:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x562d67d3add0 0x562d67d3d1e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d67cd1b10 0x562d67cdd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562d67b2c6b0 0x562d67ce0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ff2b1b80 1 --2- 172.21.15.93:0/1331007275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562d67cd0db0 0x562d67ce0fa0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ccff9700 0 client.15474 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:50:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1331007275 >> 172.21.15.93:0/1331007275 conn(0x562d67cc5f50 msgr2=0x562d67cd4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.536+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1331007275 shutdown_connections 2023-11-16T16:50:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.537+0000 7f39ff2b1b80 1 -- 172.21.15.93:0/1331007275 wait complete. 2023-11-16T16:50:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.537+0000 7f39ff2b1b80 20 client.15474 trim_cache size 0 max 16384 2023-11-16T16:50:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:50:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:50:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:50:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:50:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.537+0000 7f39eb9ff700 1 -- 172.21.15.93:0/682966527 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f39d00a1150 msgr2=0x7f39d00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.537+0000 7f39eb9ff700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f39d00a1150 0x7f39d00a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009480 tx=0x7f39e000b040 comp rx=0 tx=0).stop 2023-11-16T16:50:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.537+0000 7f39eb9ff700 1 -- 172.21.15.93:0/682966527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4105f00 msgr2=0x7f39e419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.537+0000 7f39eb9ff700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4105f00 0x7f39e419f7d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00e970 tx=0x7f39dc00ec80 comp rx=0 tx=0).stop 2023-11-16T16:50:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.537+0000 7f39eb9ff700 1 -- 172.21.15.93:0/682966527 shutdown_connections 2023-11-16T16:50:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.537+0000 7f39eb9ff700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f39d00a1150 0x7f39d00a3600 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.537+0000 7f39eb9ff700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4105f00 0x7f39e419f7d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.537+0000 7f39eb9ff700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e4100cc0 0x7f39e4198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.537+0000 7f39eb9ff700 1 --2- 172.21.15.93:0/682966527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e41035e0 0x7f39e4198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.537+0000 7f39eb9ff700 1 -- 172.21.15.93:0/682966527 >> 172.21.15.93:0/682966527 conn(0x7f39e40fa720 msgr2=0x7f39e40ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.538+0000 7f39eb9ff700 1 -- 172.21.15.93:0/682966527 shutdown_connections 2023-11-16T16:50:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:50:58.538+0000 7f39eb9ff700 1 -- 172.21.15.93:0/682966527 wait complete. 2023-11-16T16:50:58.686 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:58 smithi093 ceph-mon[97863]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-11-16T16:50:58.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:58 smithi159 ceph-mon[131528]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-11-16T16:50:58.774 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:58.774 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:58.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:58 smithi125 ceph-mon[131076]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-11-16T16:50:58.926 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:58.926 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:58.938 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:58.939 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:58.939 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 79 -lt 100 ']' 2023-11-16T16:50:58.939 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 79 + 1 2023-11-16T16:50:58.960 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=80 2023-11-16T16:50:58.960 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:58.993 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:58.993 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:58.996 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:58.996 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:58.996 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:59.440 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:59.441 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:59.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:50:59 smithi093 ceph-mon[97863]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 8 op/s 2023-11-16T16:50:59.676 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:50:59.676 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:50:59.690 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:59.690 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:50:59.690 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 80 -lt 100 ']' 2023-11-16T16:50:59.691 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 80 + 1 2023-11-16T16:50:59.710 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=81 2023-11-16T16:50:59.710 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:50:59.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:50:59 smithi159 ceph-mon[131528]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 8 op/s 2023-11-16T16:50:59.740 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:50:59.741 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:50:59.743 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:50:59.744 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:50:59.744 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:50:59.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:50:59 smithi125 ceph-mon[131076]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 8 op/s 2023-11-16T16:51:00.190 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:00.191 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:00.588 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:00.588 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:00.597 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:00.597 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:00.597 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 81 -lt 100 ']' 2023-11-16T16:51:00.597 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 81 + 1 2023-11-16T16:51:00.618 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=82 2023-11-16T16:51:00.618 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:00.651 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:00.651 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:00.653 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:00.654 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:00.654 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:01.124 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:01.124 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:01.259 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:01.259 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:01.273 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:01.273 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:01.273 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 82 -lt 100 ']' 2023-11-16T16:51:01.273 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 82 + 1 2023-11-16T16:51:01.293 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=83 2023-11-16T16:51:01.293 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:01.326 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:01.326 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:01.329 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:01.329 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:01.330 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:01.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:01 smithi093 ceph-mon[97863]: pgmap v417: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-11-16T16:51:01.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:01 smithi159 ceph-mon[131528]: pgmap v417: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-11-16T16:51:01.741 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:01.741 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:01.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:01 smithi125 ceph-mon[131076]: pgmap v417: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-11-16T16:51:01.930 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:01.930 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:01.939 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:01.939 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:01.939 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 83 -lt 100 ']' 2023-11-16T16:51:01.940 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 83 + 1 2023-11-16T16:51:01.960 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=84 2023-11-16T16:51:01.960 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:02.000 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:02.001 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:02.004 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:02.004 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:02.005 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:02.415 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:02.415 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:02.575 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:02.575 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:02.590 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:02.590 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:02.590 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 84 -lt 100 ']' 2023-11-16T16:51:02.590 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 84 + 1 2023-11-16T16:51:02.610 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=85 2023-11-16T16:51:02.610 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:02.639 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:02.639 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:02.642 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:02.643 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:02.643 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:03.032 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:03.032 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:03.217 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:03.217 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:03.248 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:03.248 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:03.248 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 85 -lt 100 ']' 2023-11-16T16:51:03.248 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 85 + 1 2023-11-16T16:51:03.276 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=86 2023-11-16T16:51:03.276 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:03.305 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:03.306 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:03.309 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:03.309 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:03.310 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:03.706 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:03.707 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:03.875 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:03.875 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:03.890 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:03.890 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:03.891 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 86 -lt 100 ']' 2023-11-16T16:51:03.891 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 86 + 1 2023-11-16T16:51:03.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:03 smithi093 ceph-mon[97863]: pgmap v418: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 15 op/s 2023-11-16T16:51:03.940 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=87 2023-11-16T16:51:03.940 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:03.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:03 smithi159 ceph-mon[131528]: pgmap v418: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 15 op/s 2023-11-16T16:51:03.987 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:03.988 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:03.991 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:03.991 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:03.992 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:04.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:03 smithi125 ceph-mon[131076]: pgmap v418: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 15 op/s 2023-11-16T16:51:04.382 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:04.382 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:04.517 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:04.517 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:04.530 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:04.531 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:04.531 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 87 -lt 100 ']' 2023-11-16T16:51:04.531 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 87 + 1 2023-11-16T16:51:04.551 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=88 2023-11-16T16:51:04.551 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:04.588 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:04.589 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:04.591 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:04.592 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:04.592 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:04.957 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:04.957 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:05.100 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:05.100 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:05.114 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:05.114 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:05.114 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 88 -lt 100 ']' 2023-11-16T16:51:05.115 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 88 + 1 2023-11-16T16:51:05.134 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=89 2023-11-16T16:51:05.134 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:05.165 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:05.165 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:05.168 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:05.168 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:05.169 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:05.556 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:05.557 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:05.721 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:05.721 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:05.730 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:05.730 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:05.730 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 89 -lt 100 ']' 2023-11-16T16:51:05.731 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 89 + 1 2023-11-16T16:51:05.751 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=90 2023-11-16T16:51:05.751 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:05.789 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:05.789 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:05.792 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:05.793 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:05.793 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:06.273 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:06.273 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:06.425 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:06.425 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:06.439 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:06.439 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:06.439 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 90 -lt 100 ']' 2023-11-16T16:51:06.439 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 90 + 1 2023-11-16T16:51:06.459 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=91 2023-11-16T16:51:06.459 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:06.490 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:06.490 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:06.493 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:06.493 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:06.494 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:06.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:06 smithi093 ceph-mon[97863]: pgmap v419: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 16 op/s 2023-11-16T16:51:06.734 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:06 smithi159 ceph-mon[131528]: pgmap v419: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 16 op/s 2023-11-16T16:51:06.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:06 smithi125 ceph-mon[131076]: pgmap v419: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 16 op/s 2023-11-16T16:51:06.856 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:06.856 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:06.974 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:06.975 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:06.988 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:06.988 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:06.988 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 91 -lt 100 ']' 2023-11-16T16:51:06.989 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 91 + 1 2023-11-16T16:51:07.009 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=92 2023-11-16T16:51:07.009 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:07.039 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:07.040 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:07.042 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:07.043 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:07.043 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:07.523 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:07.523 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:07.675 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:07.675 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:07.697 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:07.697 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:07.697 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 92 -lt 100 ']' 2023-11-16T16:51:07.698 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 92 + 1 2023-11-16T16:51:07.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=93 2023-11-16T16:51:07.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:07.750 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:07.750 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:07.753 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:07.754 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:07.754 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:08.206 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:08.207 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:08.415 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:08.415 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:08.430 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:08.430 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:08.430 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 93 -lt 100 ']' 2023-11-16T16:51:08.431 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 93 + 1 2023-11-16T16:51:08.451 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=94 2023-11-16T16:51:08.451 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:08.484 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:08.484 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:08.487 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:08.488 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:08.488 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:08.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:08 smithi159 ceph-mon[131528]: pgmap v420: 129 pgs: 129 active+clean; 1.5 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 15 op/s 2023-11-16T16:51:08.742 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:08 smithi093 ceph-mon[97863]: pgmap v420: 129 pgs: 129 active+clean; 1.5 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 15 op/s 2023-11-16T16:51:08.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:08 smithi125 ceph-mon[131076]: pgmap v420: 129 pgs: 129 active+clean; 1.5 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 15 op/s 2023-11-16T16:51:08.856 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:08.856 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:09.008 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:09.008 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:09.022 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:09.022 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:09.022 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 94 -lt 100 ']' 2023-11-16T16:51:09.023 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 94 + 1 2023-11-16T16:51:09.043 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=95 2023-11-16T16:51:09.043 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:09.075 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:09.076 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:09.078 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:09.079 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:09.079 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:09.506 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:09.506 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:09.658 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:09.658 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:09.679 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:09.680 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:09.680 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 95 -lt 100 ']' 2023-11-16T16:51:09.680 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 95 + 1 2023-11-16T16:51:09.700 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=96 2023-11-16T16:51:09.700 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:09.733 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:09.734 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:09.736 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:09.737 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:09.737 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:10.123 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:10.123 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:10.324 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:10.325 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:10.338 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:10.339 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:10.339 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 96 -lt 100 ']' 2023-11-16T16:51:10.339 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 96 + 1 2023-11-16T16:51:10.359 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=97 2023-11-16T16:51:10.359 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:10.392 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:10.392 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:10.395 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:10.396 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:10.396 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:10.684 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:10 smithi093 ceph-mon[97863]: pgmap v421: 129 pgs: 129 active+clean; 1.5 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-11-16T16:51:10.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:10 smithi159 ceph-mon[131528]: pgmap v421: 129 pgs: 129 active+clean; 1.5 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-11-16T16:51:10.806 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:10.806 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:10.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:10 smithi125 ceph-mon[131076]: pgmap v421: 129 pgs: 129 active+clean; 1.5 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-11-16T16:51:10.897 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:10.897 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:10.914 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:10.914 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:10.914 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 97 -lt 100 ']' 2023-11-16T16:51:10.915 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 97 + 1 2023-11-16T16:51:10.939 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=98 2023-11-16T16:51:10.939 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:10.946 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:10.947 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:10.950 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:10.951 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:10.951 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:11.430 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:11.431 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:11.590 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:11.590 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:11.622 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:11.622 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:11.622 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 98 -lt 100 ']' 2023-11-16T16:51:11.623 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 98 + 1 2023-11-16T16:51:11.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:51:11.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:11.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:11.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:11.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:11 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:11.680 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=99 2023-11-16T16:51:11.680 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:11.732 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:11.732 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:11.734 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:51:11.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:11.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:11.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:11.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:11 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:11.735 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:11.735 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:11.736 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:11.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:51:11.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:11.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:11.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:11.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:11 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:12.489 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:12.489 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:12.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:12 smithi093 ceph-mon[97863]: pgmap v422: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 16 op/s 2023-11-16T16:51:12.715 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:12.715 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:12.734 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:12 smithi159 ceph-mon[131528]: pgmap v422: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 16 op/s 2023-11-16T16:51:12.746 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:12.747 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:12.747 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 99 -lt 100 ']' 2023-11-16T16:51:12.747 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 99 + 1 2023-11-16T16:51:12.805 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=100 2023-11-16T16:51:12.805 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:12.817 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:12.817 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:12.820 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-11-16T16:51:12.820 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-11-16T16:51:12.821 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:51:12.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:12 smithi125 ceph-mon[131076]: pgmap v422: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 16 op/s 2023-11-16T16:51:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.615+0000 7fc8c59ff700 1 -- 172.21.15.93:0/1029191447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8c0103640 msgr2=0x7fc8c0105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.615+0000 7fc8c59ff700 1 --2- 172.21.15.93:0/1029191447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8c0103640 0x7fc8c0105a20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009990 tx=0x7fc8a8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.616+0000 7fc8c59ff700 1 -- 172.21.15.93:0/1029191447 shutdown_connections 2023-11-16T16:51:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.616+0000 7fc8c59ff700 1 --2- 172.21.15.93:0/1029191447 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8c0105f60 0x7fc8c0108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.616+0000 7fc8c59ff700 1 --2- 172.21.15.93:0/1029191447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0100d20 0x7fc8c0103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.616+0000 7fc8c59ff700 1 --2- 172.21.15.93:0/1029191447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8c0103640 0x7fc8c0105a20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.616+0000 7fc8c59ff700 1 -- 172.21.15.93:0/1029191447 >> 172.21.15.93:0/1029191447 conn(0x7fc8c00fa720 msgr2=0x7fc8c00fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.616+0000 7fc8c59ff700 1 -- 172.21.15.93:0/1029191447 shutdown_connections 2023-11-16T16:51:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.616+0000 7fc8c59ff700 1 -- 172.21.15.93:0/1029191447 wait complete. 2023-11-16T16:51:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.628+0000 7fc8c59ff700 1 Processor -- start 2023-11-16T16:51:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.629+0000 7fc8c59ff700 1 -- start start 2023-11-16T16:51:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.629+0000 7fc8c59ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8c0100d20 0x7fc8c01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.629+0000 7fc8c59ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0103640 0x7fc8c0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.629+0000 7fc8c59ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8c0105f60 0x7fc8c019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.629+0000 7fc8c59ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c019fd40 con 0x7fc8c0105f60 2023-11-16T16:51:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.629+0000 7fc8c59ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c019fe80 con 0x7fc8c0100d20 2023-11-16T16:51:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.629+0000 7fc8c59ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01a0010 con 0x7fc8c0103640 2023-11-16T16:51:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.629+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8c0105f60 0x7fc8c019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.629+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8c0105f60 0x7fc8c019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50846/0 (socket says 172.21.15.93:50846) 2023-11-16T16:51:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.629+0000 7fc8bf7fe700 1 -- 172.21.15.93:0/4125076654 learned_addr learned my addr 172.21.15.93:0/4125076654 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.629+0000 7fc8beffd700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8c0100d20 0x7fc8c01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.629+0000 7fc8b67fc700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0103640 0x7fc8c0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.630+0000 7fc8bf7fe700 1 -- 172.21.15.93:0/4125076654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0103640 msgr2=0x7fc8c0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.630+0000 7fc8bf7fe700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0103640 0x7fc8c0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.630+0000 7fc8bf7fe700 1 -- 172.21.15.93:0/4125076654 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8c0100d20 msgr2=0x7fc8c01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.630+0000 7fc8bf7fe700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8c0100d20 0x7fc8c01981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.630+0000 7fc8bf7fe700 1 -- 172.21.15.93:0/4125076654 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80095f0 con 0x7fc8c0105f60 2023-11-16T16:51:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.630+0000 7fc8b67fc700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0103640 0x7fc8c0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:51:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.630+0000 7fc8bf7fe700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8c0105f60 0x7fc8c019f770 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800eb80 tx=0x7fc8b800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.630+0000 7fc8c49fd700 1 -- 172.21.15.93:0/4125076654 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8018070 con 0x7fc8c0105f60 2023-11-16T16:51:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.630+0000 7fc8c49fd700 1 -- 172.21.15.93:0/4125076654 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8b800f040 con 0x7fc8c0105f60 2023-11-16T16:51:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.630+0000 7fc8c49fd700 1 -- 172.21.15.93:0/4125076654 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800ce80 con 0x7fc8c0105f60 2023-11-16T16:51:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.630+0000 7fc8c59ff700 1 -- 172.21.15.93:0/4125076654 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c010bc50 con 0x7fc8c0105f60 2023-11-16T16:51:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.630+0000 7fc8c59ff700 1 -- 172.21.15.93:0/4125076654 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c010bac0 con 0x7fc8c0105f60 2023-11-16T16:51:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.633+0000 7fc8c49fd700 1 -- 172.21.15.93:0/4125076654 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc8b8007500 con 0x7fc8c0105f60 2023-11-16T16:51:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.635+0000 7fc8c49fd700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc8ac0a1150 0x7fc8ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.635+0000 7fc8c49fd700 1 -- 172.21.15.93:0/4125076654 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fc8b80cc930 con 0x7fc8c0105f60 2023-11-16T16:51:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.635+0000 7fc8beffd700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc8ac0a1150 0x7fc8ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.636+0000 7fc8beffd700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc8ac0a1150 0x7fc8ac0a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009c60 tx=0x7fc8b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:13.713 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-11-16T16:51:13.714 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-11-16T16:51:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.787+0000 7fc8d92a4b80 1 Processor -- start 2023-11-16T16:51:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.788+0000 7fc8d92a4b80 1 -- start start 2023-11-16T16:51:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.788+0000 7fc8d92a4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a478ce6b0 0x556a47a72870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.788+0000 7fc8d92a4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a72db0 0x556a47a735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.788+0000 7fc8d92a4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a47a73b10 0x556a47a73f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.788+0000 7fc8d92a4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a47978000 con 0x556a478ce6b0 2023-11-16T16:51:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.788+0000 7fc8d92a4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a479ca840 con 0x556a47a72db0 2023-11-16T16:51:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.788+0000 7fc8d92a4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a479c3cd0 con 0x556a47a73b10 2023-11-16T16:51:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.788+0000 7fc8beffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a478ce6b0 0x556a47a72870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.788+0000 7fc8b67fc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a72db0 0x556a47a735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.788+0000 7fc8b67fc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a72db0 0x556a47a735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:55144/0 (socket says 172.21.15.93:55144) 2023-11-16T16:51:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.788+0000 7fc8b67fc700 1 -- 172.21.15.93:0/111382557 learned_addr learned my addr 172.21.15.93:0/111382557 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.788+0000 7fc8bf7fe700 1 --2- 172.21.15.93:0/111382557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a47a73b10 0x556a47a73f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.789+0000 7fc8b67fc700 1 -- 172.21.15.93:0/111382557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a47a73b10 msgr2=0x556a47a73f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.789+0000 7fc8b67fc700 1 --2- 172.21.15.93:0/111382557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a47a73b10 0x556a47a73f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.789+0000 7fc8b67fc700 1 -- 172.21.15.93:0/111382557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a478ce6b0 msgr2=0x556a47a72870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.789+0000 7fc8b67fc700 1 --2- 172.21.15.93:0/111382557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a478ce6b0 0x556a47a72870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.789+0000 7fc8b67fc700 1 -- 172.21.15.93:0/111382557 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a478455d0 con 0x556a47a72db0 2023-11-16T16:51:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.789+0000 7fc8bf7fe700 1 --2- 172.21.15.93:0/111382557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a47a73b10 0x556a47a73f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:51:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.789+0000 7fc8b67fc700 1 --2- 172.21.15.93:0/111382557 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a72db0 0x556a47a735d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800b2e0 tx=0x7fc8a8005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db2abcb438ee5afa server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.789+0000 7fc8b5ffb700 1 -- 172.21.15.93:0/111382557 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800beb0 con 0x556a47a72db0 2023-11-16T16:51:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.790+0000 7fc8b5ffb700 1 -- 172.21.15.93:0/111382557 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8a801e040 con 0x556a47a72db0 2023-11-16T16:51:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.790+0000 7fc8b5ffb700 1 -- 172.21.15.93:0/111382557 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8004ba0 con 0x556a47a72db0 2023-11-16T16:51:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.790+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/111382557 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a72db0 msgr2=0x556a47a735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.790+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/111382557 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a72db0 0x556a47a735d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800b2e0 tx=0x7fc8a8005c00 comp rx=0 tx=0).stop 2023-11-16T16:51:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.790+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/111382557 shutdown_connections 2023-11-16T16:51:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.790+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/111382557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a478ce6b0 0x556a47a72870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.790+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/111382557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a47a73b10 0x556a47a73f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.790+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/111382557 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a72db0 0x556a47a735d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.790+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/111382557 >> 172.21.15.93:0/111382557 conn(0x556a47a67f50 msgr2=0x556a47a6d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.790+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/111382557 shutdown_connections 2023-11-16T16:51:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.790+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/111382557 wait complete. 2023-11-16T16:51:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.791+0000 7fc8d92a4b80 1 -- start start 2023-11-16T16:51:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.791+0000 7fc8d92a4b80 1 Processor -- start 2023-11-16T16:51:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.792+0000 7fc8d92a4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a478ce6b0 0x556a47a7bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.792+0000 7fc8d92a4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 0x556a47a7c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.792+0000 7fc8d92a4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a83220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.792+0000 7fc8d92a4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a47965880 con 0x556a47a72db0 2023-11-16T16:51:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.792+0000 7fc8d92a4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a47965520 con 0x556a47a73b10 2023-11-16T16:51:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.792+0000 7fc8d92a4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a478478c0 con 0x556a478ce6b0 2023-11-16T16:51:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.792+0000 7fc8d92a4b80 10 client.0 _ll_register_callbacks cb 0x556a47983720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:51:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.792+0000 7fc8b67fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 0x556a47a7c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc8beffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a478ce6b0 0x556a47a7bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc8b67fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 0x556a47a7c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50874/0 (socket says 172.21.15.93:50874) 2023-11-16T16:51:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc8b67fc700 1 -- 172.21.15.93:0/2819299077 learned_addr learned my addr 172.21.15.93:0/2819299077 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc8bf7fe700 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a83220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc8beffd700 1 -- 172.21.15.93:0/2819299077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 msgr2=0x556a47a83220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc8beffd700 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a83220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc8beffd700 1 -- 172.21.15.93:0/2819299077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 msgr2=0x556a47a7c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc8beffd700 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 0x556a47a7c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc8beffd700 1 -- 172.21.15.93:0/2819299077 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c0108f50 con 0x556a478ce6b0 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc8b67fc700 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 0x556a47a7c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc8bf7fe700 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a83220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc8beffd700 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a478ce6b0 0x556a47a7bca0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0011ab0 tx=0x7fc8b00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=631c89a9c0b33567 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.793+0000 7fc89effd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.794+0000 7fc89effd700 1 -- 172.21.15.93:0/2819299077 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0022070 con 0x556a478ce6b0 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.794+0000 7fc89effd700 1 -- 172.21.15.93:0/2819299077 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fc8b001d620 con 0x556a478ce6b0 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.794+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/2819299077 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a47a65710 con 0x556a478ce6b0 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.794+0000 7fc89effd700 1 -- 172.21.15.93:0/2819299077 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8b001d7e0 con 0x556a478ce6b0 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.794+0000 7fc8d92a4b80 10 client.25516 fetch_fsmap learned FSMap version 23 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.794+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/2819299077 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556a4798a130 con 0x556a478ce6b0 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.795+0000 7fc89effd700 1 -- 172.21.15.93:0/2819299077 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a47a65710 con 0x556a478ce6b0 2023-11-16T16:51:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.795+0000 7fc89effd700 1 -- 172.21.15.93:0/2819299077 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8b0027b40 con 0x556a478ce6b0 2023-11-16T16:51:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.795+0000 7fc8d92a4b80 10 client.25516 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:51:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.795+0000 7fc8d92a4b80 10 client.25516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:51:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.795+0000 7fc8d92a4b80 20 client.25516 populate_metadata read hostname 'smithi093' 2023-11-16T16:51:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.795+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x556a47a87b00 0x556a47a89f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.795+0000 7fc8d92a4b80 4 client.25516 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:51:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.795+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/2819299077 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a47986040 con 0x556a47a87b00 2023-11-16T16:51:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.795+0000 7fc8b67fc700 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x556a47a87b00 0x556a47a89f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.796+0000 7fc89effd700 10 client.25516 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:51:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.796+0000 7fc8b67fc700 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x556a47a87b00 0x556a47a89f10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781ce59901d96cad server_cookie=671f134735da998b in_seq=0 out_seq=0 2023-11-16T16:51:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc89effd700 1 -- 172.21.15.93:0/2819299077 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556a47986040 con 0x556a47a87b00 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc89effd700 10 client.25516 handle_command_reply: tid=0 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc8d92a4b80 1 client.25516 shutdown 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/2819299077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a478ce6b0 msgr2=0x556a47a7bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a478ce6b0 0x556a47a7bca0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0011ab0 tx=0x7fc8b00047c0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/2819299077 shutdown_connections 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x556a47a87b00 0x556a47a89f10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a83220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a478ce6b0 0x556a47a7bca0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc89effd700 0 client.25516 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/2819299077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 0x556a47a7c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/2819299077 >> 172.21.15.93:0/2819299077 conn(0x556a47a67f50 msgr2=0x556a47a6c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/2819299077 shutdown_connections 2023-11-16T16:51:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/2819299077 wait complete. 2023-11-16T16:51:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.797+0000 7fc8d92a4b80 20 client.25516 trim_cache size 0 max 16384 2023-11-16T16:51:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8d92a4b80 1 Processor -- start 2023-11-16T16:51:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8d92a4b80 1 -- start start 2023-11-16T16:51:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8d92a4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a478ce6b0 0x556a47a6e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8d92a4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a47a72db0 0x556a47a831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8d92a4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a7b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8d92a4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a47a7b180 con 0x556a478ce6b0 2023-11-16T16:51:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8d92a4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a47a733f0 con 0x556a47a73b10 2023-11-16T16:51:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8d92a4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a47a79c50 con 0x556a47a72db0 2023-11-16T16:51:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8beffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a478ce6b0 0x556a47a6e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8beffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a478ce6b0 0x556a47a6e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50880/0 (socket says 172.21.15.93:50880) 2023-11-16T16:51:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8beffd700 1 -- 172.21.15.93:0/3999194982 learned_addr learned my addr 172.21.15.93:0/3999194982 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8bf7fe700 1 --2- 172.21.15.93:0/3999194982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a7b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.813+0000 7fc8b67fc700 1 --2- 172.21.15.93:0/3999194982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a47a72db0 0x556a47a831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.814+0000 7fc8beffd700 1 -- 172.21.15.93:0/3999194982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a47a72db0 msgr2=0x556a47a831e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.814+0000 7fc8beffd700 1 --2- 172.21.15.93:0/3999194982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a47a72db0 0x556a47a831e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.814+0000 7fc8beffd700 1 -- 172.21.15.93:0/3999194982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 msgr2=0x556a47a7b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.814+0000 7fc8beffd700 1 --2- 172.21.15.93:0/3999194982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a7b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.814+0000 7fc8beffd700 1 -- 172.21.15.93:0/3999194982 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a47989e80 con 0x556a478ce6b0 2023-11-16T16:51:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.814+0000 7fc8beffd700 1 --2- 172.21.15.93:0/3999194982 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a478ce6b0 0x556a47a6e1a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0000c00 tx=0x7fc8b00093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94c29eca97b82d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.814+0000 7fc8b6ffd700 1 -- 172.21.15.93:0/3999194982 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0039070 con 0x556a478ce6b0 2023-11-16T16:51:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.814+0000 7fc8b6ffd700 1 -- 172.21.15.93:0/3999194982 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8b00045c0 con 0x556a478ce6b0 2023-11-16T16:51:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.814+0000 7fc8b6ffd700 1 -- 172.21.15.93:0/3999194982 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0022030 con 0x556a478ce6b0 2023-11-16T16:51:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.815+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/3999194982 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a478ce6b0 msgr2=0x556a47a6e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.815+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/3999194982 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a478ce6b0 0x556a47a6e1a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0000c00 tx=0x7fc8b00093b0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.815+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/3999194982 shutdown_connections 2023-11-16T16:51:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.815+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/3999194982 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a478ce6b0 0x556a47a6e1a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.815+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/3999194982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a7b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.815+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/3999194982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a47a72db0 0x556a47a831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.815+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/3999194982 >> 172.21.15.93:0/3999194982 conn(0x556a47a67f50 msgr2=0x556a47a6d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.815+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/3999194982 shutdown_connections 2023-11-16T16:51:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.815+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/3999194982 wait complete. 2023-11-16T16:51:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.815+0000 7fc8d92a4b80 1 -- start start 2023-11-16T16:51:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.816+0000 7fc8d92a4b80 1 Processor -- start 2023-11-16T16:51:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.816+0000 7fc8d92a4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a478ce6b0 0x556a47a7e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.816+0000 7fc8d92a4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 0x556a47a7ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.816+0000 7fc8d92a4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a81bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.816+0000 7fc8d92a4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a47a82130 con 0x556a47a72db0 2023-11-16T16:51:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.816+0000 7fc8d92a4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a47a822a0 con 0x556a47a73b10 2023-11-16T16:51:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.816+0000 7fc8d92a4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a47a82460 con 0x556a478ce6b0 2023-11-16T16:51:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.816+0000 7fc8d92a4b80 10 client.0 _ll_register_callbacks cb 0x556a47983720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:51:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.816+0000 7fc8b67fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 0x556a47a7ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.816+0000 7fc8b67fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 0x556a47a7ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50886/0 (socket says 172.21.15.93:50886) 2023-11-16T16:51:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.816+0000 7fc8b67fc700 1 -- 172.21.15.93:0/1825420230 learned_addr learned my addr 172.21.15.93:0/1825420230 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.816+0000 7fc8beffd700 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a478ce6b0 0x556a47a7e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.817+0000 7fc8bf7fe700 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a81bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.817+0000 7fc8b67fc700 1 -- 172.21.15.93:0/1825420230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a478ce6b0 msgr2=0x556a47a7e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.817+0000 7fc8b67fc700 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a478ce6b0 0x556a47a7e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.817+0000 7fc8b67fc700 1 -- 172.21.15.93:0/1825420230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 msgr2=0x556a47a81bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.817+0000 7fc8b67fc700 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a81bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.817+0000 7fc8b67fc700 1 -- 172.21.15.93:0/1825420230 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a4798a130 con 0x556a47a72db0 2023-11-16T16:51:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.817+0000 7fc8b67fc700 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 0x556a47a7ebd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80098e0 tx=0x7fc8a8005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee9c1a74dad81f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.817+0000 7fc89effd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:51:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.817+0000 7fc89effd700 1 -- 172.21.15.93:0/1825420230 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8031d90 con 0x556a47a72db0 2023-11-16T16:51:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.817+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/1825420230 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8a801e040 con 0x556a47a72db0 2023-11-16T16:51:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.817+0000 7fc89effd700 1 -- 172.21.15.93:0/1825420230 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fc8a8004790 con 0x556a47a72db0 2023-11-16T16:51:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.818+0000 7fc89effd700 1 -- 172.21.15.93:0/1825420230 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8a8031150 con 0x556a47a72db0 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.818+0000 7fc8d92a4b80 10 client.15492 fetch_fsmap learned FSMap version 23 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.818+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/1825420230 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556a477dac00 con 0x556a47a72db0 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.818+0000 7fc89effd700 1 -- 172.21.15.93:0/1825420230 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8a801e040 con 0x556a47a72db0 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.818+0000 7fc89effd700 1 -- 172.21.15.93:0/1825420230 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8a800f460 con 0x556a47a72db0 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.818+0000 7fc8d92a4b80 10 client.15492 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.818+0000 7fc8d92a4b80 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.818+0000 7fc8d92a4b80 20 client.15492 populate_metadata read hostname 'smithi093' 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.818+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x556a47ae43d0 0x556a47ae67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.818+0000 7fc8d92a4b80 4 client.15492 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.818+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/1825420230 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556a47992600 con 0x556a47ae43d0 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.819+0000 7fc8beffd700 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x556a47ae43d0 0x556a47ae67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.819+0000 7fc8beffd700 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x556a47ae43d0 0x556a47ae67b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f4dc79f39adec8 server_cookie=b1c2836deb7e0b53 in_seq=0 out_seq=0 2023-11-16T16:51:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.819+0000 7fc89effd700 10 client.15492 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:51:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.846+0000 7fc89effd700 1 -- 172.21.15.93:0/1825420230 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556a47992600 con 0x556a47ae43d0 2023-11-16T16:51:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.846+0000 7fc89effd700 10 client.15492 handle_command_reply: tid=0 2023-11-16T16:51:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.846+0000 7fc8d92a4b80 1 client.15492 shutdown 2023-11-16T16:51:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.846+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/1825420230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 msgr2=0x556a47a7ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.846+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 0x556a47a7ebd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80098e0 tx=0x7fc8a8005ab0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.847+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/1825420230 shutdown_connections 2023-11-16T16:51:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.847+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x556a47ae43d0 0x556a47ae67b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.847+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556a47a73b10 0x556a47a81bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.847+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a478ce6b0 0x556a47a7e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.847+0000 7fc89effd700 0 client.15492 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:51:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.847+0000 7fc8d92a4b80 1 --2- 172.21.15.93:0/1825420230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556a47a72db0 0x556a47a7ebd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.847+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/1825420230 >> 172.21.15.93:0/1825420230 conn(0x556a47a67f50 msgr2=0x556a47a7a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.847+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/1825420230 shutdown_connections 2023-11-16T16:51:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.847+0000 7fc8d92a4b80 1 -- 172.21.15.93:0/1825420230 wait complete. 2023-11-16T16:51:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.847+0000 7fc8d92a4b80 20 client.15492 trim_cache size 0 max 16384 2023-11-16T16:51:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:51:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:51:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:51:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:51:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.848+0000 7fc8c59ff700 1 -- 172.21.15.93:0/4125076654 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc8ac0a1150 msgr2=0x7fc8ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.848+0000 7fc8c59ff700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc8ac0a1150 0x7fc8ac0a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009c60 tx=0x7fc8b0009260 comp rx=0 tx=0).stop 2023-11-16T16:51:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.848+0000 7fc8c59ff700 1 -- 172.21.15.93:0/4125076654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8c0105f60 msgr2=0x7fc8c019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.848+0000 7fc8c59ff700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8c0105f60 0x7fc8c019f770 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800eb80 tx=0x7fc8b800ee90 comp rx=0 tx=0).stop 2023-11-16T16:51:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.848+0000 7fc8c59ff700 1 -- 172.21.15.93:0/4125076654 shutdown_connections 2023-11-16T16:51:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.848+0000 7fc8c59ff700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc8ac0a1150 0x7fc8ac0a3600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.848+0000 7fc8c59ff700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8c0105f60 0x7fc8c019f770 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.848+0000 7fc8c59ff700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8c0100d20 0x7fc8c01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.848+0000 7fc8c59ff700 1 --2- 172.21.15.93:0/4125076654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0103640 0x7fc8c0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.848+0000 7fc8c59ff700 1 -- 172.21.15.93:0/4125076654 >> 172.21.15.93:0/4125076654 conn(0x7fc8c00fa720 msgr2=0x7fc8c00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.848+0000 7fc8c59ff700 1 -- 172.21.15.93:0/4125076654 shutdown_connections 2023-11-16T16:51:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:13.848+0000 7fc8c59ff700 1 -- 172.21.15.93:0/4125076654 wait complete. 2023-11-16T16:51:13.890 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-11-16T16:51:13.890 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:13.930 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:13.930 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:51:13.930 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 100 -lt 100 ']' 2023-11-16T16:51:13.942 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-11-16T16:51:14.164 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2023-11-16T16:51:14.220 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2023-11-16T16:51:14.224 INFO:tasks.workunit.client.0.smithi093.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2023-11-16T16:51:14.224 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n system.posix_acl_access d1 2023-11-16T16:51:14.262 INFO:tasks.workunit.client.0.smithi093.stderr:d1: system.posix_acl_access: No such attribute 2023-11-16T16:51:14.262 INFO:tasks.workunit.client.0.smithi093.stderr:+ return 0 2023-11-16T16:51:14.262 INFO:tasks.workunit.client.0.smithi093.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2023-11-16T16:51:14.262 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n system.posix_acl_default . 2023-11-16T16:51:14.264 INFO:tasks.workunit.client.0.smithi093.stderr:.: system.posix_acl_default: No such attribute 2023-11-16T16:51:14.264 INFO:tasks.workunit.client.0.smithi093.stderr:+ return 0 2023-11-16T16:51:14.264 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-11-16T16:51:14.269 INFO:tasks.workunit.client.0.smithi093.stderr:+ cd .. 2023-11-16T16:51:14.269 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir testdir 2023-11-16T16:51:14.281 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo OK 2023-11-16T16:51:14.282 INFO:tasks.workunit.client.0.smithi093.stdout:OK 2023-11-16T16:51:14.282 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:51:14.282 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:51:14.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:14 smithi159 ceph-mon[131528]: pgmap v423: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-11-16T16:51:14.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:14 smithi125 ceph-mon[131076]: pgmap v423: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-11-16T16:51:14.890 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:14 smithi093 ceph-mon[97863]: pgmap v423: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-11-16T16:51:14.920 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2023-11-16T16:51:14.920 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2023-11-16T16:51:15.211 INFO:tasks.workunit.client.0.smithi093.stderr:+ set -e 2023-11-16T16:51:15.211 INFO:tasks.workunit.client.0.smithi093.stderr:+ file=test_chmod.150517 2023-11-16T16:51:15.212 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo foo 2023-11-16T16:51:15.217 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-11-16T16:51:15.217 INFO:tasks.workunit.client.0.smithi093.stderr:+ chmod 400 test_chmod.150517 2023-11-16T16:51:15.225 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-11-16T16:51:15.225 INFO:tasks.workunit.client.0.smithi093.stderr:+ check_perms test_chmod.150517 -r-------- 2023-11-16T16:51:15.225 INFO:tasks.workunit.client.0.smithi093.stderr:+ file=test_chmod.150517 2023-11-16T16:51:15.225 INFO:tasks.workunit.client.0.smithi093.stderr:++ ls -la test_chmod.150517 2023-11-16T16:51:15.236 INFO:tasks.workunit.client.0.smithi093.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Nov 16 16:51 test_chmod.150517' 2023-11-16T16:51:15.237 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-11-16T16:51:15.237 INFO:tasks.workunit.client.0.smithi093.stderr:+ perms=-r-------- 2023-11-16T16:51:15.238 INFO:tasks.workunit.client.0.smithi093.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Nov 16 16:51 test_chmod.150517 2023-11-16T16:51:15.238 INFO:tasks.workunit.client.0.smithi093.stderr:++ awk '{print $1}' 2023-11-16T16:51:15.278 INFO:tasks.workunit.client.0.smithi093.stderr:+ test -r-------- '!=' -r--------. 2023-11-16T16:51:15.280 INFO:tasks.workunit.client.0.smithi093.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Nov 16 16:51 test_chmod.150517 2023-11-16T16:51:15.280 INFO:tasks.workunit.client.0.smithi093.stderr:++ awk '{print $1}' 2023-11-16T16:51:15.284 INFO:tasks.workunit.client.0.smithi093.stderr:+ test -r--------. '!=' -r--------. 2023-11-16T16:51:15.284 INFO:tasks.workunit.client.0.smithi093.stderr:+ set +e 2023-11-16T16:51:15.284 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo bar 2023-11-16T16:51:15.284 INFO:tasks.workunit.client.0.smithi093.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.150517: Permission denied 2023-11-16T16:51:15.284 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 1 = 0 2023-11-16T16:51:15.285 INFO:tasks.workunit.client.0.smithi093.stderr:+ set -e 2023-11-16T16:51:15.285 INFO:tasks.workunit.client.0.smithi093.stderr:+ chmod 600 test_chmod.150517 2023-11-16T16:51:15.287 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo bar 2023-11-16T16:51:15.287 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-11-16T16:51:15.287 INFO:tasks.workunit.client.0.smithi093.stderr:+ check_perms test_chmod.150517 -rw------- 2023-11-16T16:51:15.287 INFO:tasks.workunit.client.0.smithi093.stderr:+ file=test_chmod.150517 2023-11-16T16:51:15.287 INFO:tasks.workunit.client.0.smithi093.stderr:++ ls -la test_chmod.150517 2023-11-16T16:51:15.291 INFO:tasks.workunit.client.0.smithi093.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Nov 16 16:51 test_chmod.150517' 2023-11-16T16:51:15.291 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-11-16T16:51:15.291 INFO:tasks.workunit.client.0.smithi093.stderr:+ perms=-rw------- 2023-11-16T16:51:15.292 INFO:tasks.workunit.client.0.smithi093.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Nov 16 16:51 test_chmod.150517 2023-11-16T16:51:15.292 INFO:tasks.workunit.client.0.smithi093.stderr:++ awk '{print $1}' 2023-11-16T16:51:15.296 INFO:tasks.workunit.client.0.smithi093.stderr:+ test -rw------- '!=' -rw-------. 2023-11-16T16:51:15.297 INFO:tasks.workunit.client.0.smithi093.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Nov 16 16:51 test_chmod.150517 2023-11-16T16:51:15.298 INFO:tasks.workunit.client.0.smithi093.stderr:++ awk '{print $1}' 2023-11-16T16:51:15.301 INFO:tasks.workunit.client.0.smithi093.stderr:+ test -rw-------. '!=' -rw-------. 2023-11-16T16:51:15.301 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo foo 2023-11-16T16:51:15.301 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-11-16T16:51:15.302 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:51:15.302 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:51:15.390 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2023-11-16T16:51:15.390 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2023-11-16T16:51:15.448 INFO:tasks.workunit.client.0.smithi093.stderr:+ set -e 2023-11-16T16:51:15.448 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir -p testdir 2023-11-16T16:51:15.452 INFO:tasks.workunit.client.0.smithi093.stderr:+ file=test_chmod.150582 2023-11-16T16:51:15.452 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo foo 2023-11-16T16:51:15.453 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo chmod 600 testdir 2023-11-16T16:51:15.487 INFO:tasks.workunit.client.0.smithi093.stderr:+ expect_failure cat testdir/test_chmod.150582 2023-11-16T16:51:15.487 INFO:tasks.workunit.client.0.smithi093.stderr:+ cat testdir/test_chmod.150582 2023-11-16T16:51:15.507 INFO:tasks.workunit.client.0.smithi093.stderr:cat: testdir/test_chmod.150582: Permission denied 2023-11-16T16:51:15.507 INFO:tasks.workunit.client.0.smithi093.stderr:+ return 0 2023-11-16T16:51:15.507 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo cat testdir/test_chmod.150582 2023-11-16T16:51:15.532 INFO:tasks.workunit.client.0.smithi093.stdout:foo 2023-11-16T16:51:15.536 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:51:15.536 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:51:15.626 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2023-11-16T16:51:15.627 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2023-11-16T16:51:16.197 INFO:tasks.workunit.client.0.smithi093.stdout:ok 2023-11-16T16:51:16.203 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:51:16.203 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:51:16.289 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2023-11-16T16:51:16.290 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2023-11-16T16:51:16.364 INFO:tasks.workunit.client.0.smithi093.stdout:creating folder hierarchy 2023-11-16T16:51:16.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:16 smithi093 ceph-mon[97863]: pgmap v424: 129 pgs: 129 active+clean; 1.7 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-11-16T16:51:16.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:16 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:51:16.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:16 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:51:16.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:16 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:16.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:16 smithi159 ceph-mon[131528]: pgmap v424: 129 pgs: 129 active+clean; 1.7 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-11-16T16:51:16.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:16 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:51:16.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:16 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:51:16.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:16 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:16.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:16 smithi125 ceph-mon[131076]: pgmap v424: 129 pgs: 129 active+clean; 1.7 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-11-16T16:51:16.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:16 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:51:16.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:16 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:51:16.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:16 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:51:17.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:17 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:51:17.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:17 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:51:17.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:17 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:51:17.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:17 smithi159 ceph-mon[131528]: Service mds.all has no related daemon already placed. 2023-11-16T16:51:17.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:17 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:51:17.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:17 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:51:17.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:17 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:51:17.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:17 smithi125 ceph-mon[131076]: Service mds.all has no related daemon already placed. 2023-11-16T16:51:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:17 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:51:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:17 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:51:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:17 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:51:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:17 smithi093 ceph-mon[97863]: Service mds.all has no related daemon already placed. 2023-11-16T16:51:18.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:18 smithi159 ceph-mon[131528]: pgmap v425: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-11-16T16:51:18.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:18 smithi125 ceph-mon[131076]: pgmap v425: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-11-16T16:51:18.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:18 smithi093 ceph-mon[97863]: pgmap v425: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-11-16T16:51:20.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:20 smithi159 ceph-mon[131528]: pgmap v426: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-11-16T16:51:20.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:20 smithi125 ceph-mon[131076]: pgmap v426: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-11-16T16:51:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:20 smithi093 ceph-mon[97863]: pgmap v426: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-11-16T16:51:22.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:22 smithi159 ceph-mon[131528]: pgmap v427: 129 pgs: 129 active+clean; 1.8 MiB data, 91 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 KiB/s wr, 17 op/s 2023-11-16T16:51:22.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:22 smithi125 ceph-mon[131076]: pgmap v427: 129 pgs: 129 active+clean; 1.8 MiB data, 91 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 KiB/s wr, 17 op/s 2023-11-16T16:51:22.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:22 smithi093 ceph-mon[97863]: pgmap v427: 129 pgs: 129 active+clean; 1.8 MiB data, 91 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 KiB/s wr, 17 op/s 2023-11-16T16:51:24.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:24 smithi159 ceph-mon[131528]: pgmap v428: 129 pgs: 129 active+clean; 1.8 MiB data, 94 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 51 KiB/s wr, 20 op/s 2023-11-16T16:51:24.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:24 smithi125 ceph-mon[131076]: pgmap v428: 129 pgs: 129 active+clean; 1.8 MiB data, 94 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 51 KiB/s wr, 20 op/s 2023-11-16T16:51:24.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:24 smithi093 ceph-mon[97863]: pgmap v428: 129 pgs: 129 active+clean; 1.8 MiB data, 94 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 51 KiB/s wr, 20 op/s 2023-11-16T16:51:26.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:26 smithi159 ceph-mon[131528]: pgmap v429: 129 pgs: 129 active+clean; 8.6 MiB data, 116 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 657 KiB/s wr, 28 op/s 2023-11-16T16:51:26.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:26 smithi125 ceph-mon[131076]: pgmap v429: 129 pgs: 129 active+clean; 8.6 MiB data, 116 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 657 KiB/s wr, 28 op/s 2023-11-16T16:51:26.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:26 smithi093 ceph-mon[97863]: pgmap v429: 129 pgs: 129 active+clean; 8.6 MiB data, 116 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 657 KiB/s wr, 28 op/s 2023-11-16T16:51:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.297+0000 7f15327ff700 1 -- 172.21.15.93:0/2354110906 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c101470 msgr2=0x7f152c1018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.297+0000 7f15327ff700 1 --2- 172.21.15.93:0/2354110906 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c101470 0x7f152c1018c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1520009990 tx=0x7f1520009ca0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.297+0000 7f15327ff700 1 -- 172.21.15.93:0/2354110906 shutdown_connections 2023-11-16T16:51:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.297+0000 7f15327ff700 1 --2- 172.21.15.93:0/2354110906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f152c100280 0x7f152c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.297+0000 7f15327ff700 1 --2- 172.21.15.93:0/2354110906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f152c101e00 0x7f152c1086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.297+0000 7f15327ff700 1 --2- 172.21.15.93:0/2354110906 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c101470 0x7f152c1018c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.297+0000 7f15327ff700 1 -- 172.21.15.93:0/2354110906 >> 172.21.15.93:0/2354110906 conn(0x7f152c0fb8a0 msgr2=0x7f152c0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.297+0000 7f15327ff700 1 -- 172.21.15.93:0/2354110906 shutdown_connections 2023-11-16T16:51:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.297+0000 7f15327ff700 1 -- 172.21.15.93:0/2354110906 wait complete. 2023-11-16T16:51:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.298+0000 7f15327ff700 1 Processor -- start 2023-11-16T16:51:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.298+0000 7f15327ff700 1 -- start start 2023-11-16T16:51:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.298+0000 7f15327ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f152c100280 0x7f152c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.298+0000 7f15327ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c101470 0x7f152c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.298+0000 7f15327ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f152c101e00 0x7f152c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.298+0000 7f15327ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c19fd70 con 0x7f152c101e00 2023-11-16T16:51:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.298+0000 7f15327ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c19feb0 con 0x7f152c101470 2023-11-16T16:51:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.298+0000 7f15327ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c1a0040 con 0x7f152c100280 2023-11-16T16:51:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.299+0000 7f152bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f152c100280 0x7f152c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.299+0000 7f1530d9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f152c101e00 0x7f152c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.299+0000 7f152b7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c101470 0x7f152c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.299+0000 7f152bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f152c100280 0x7f152c1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:54764/0 (socket says 172.21.15.93:54764) 2023-11-16T16:51:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.299+0000 7f152bfff700 1 -- 172.21.15.93:0/739068559 learned_addr learned my addr 172.21.15.93:0/739068559 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.299+0000 7f152b7fe700 1 -- 172.21.15.93:0/739068559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f152c100280 msgr2=0x7f152c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.299+0000 7f152b7fe700 1 --2- 172.21.15.93:0/739068559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f152c100280 0x7f152c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.299+0000 7f152b7fe700 1 -- 172.21.15.93:0/739068559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f152c101e00 msgr2=0x7f152c19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.299+0000 7f152b7fe700 1 --2- 172.21.15.93:0/739068559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f152c101e00 0x7f152c19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.299+0000 7f152b7fe700 1 -- 172.21.15.93:0/739068559 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c009520 con 0x7f152c101470 2023-11-16T16:51:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.299+0000 7f152b7fe700 1 --2- 172.21.15.93:0/739068559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c101470 0x7f152c198730 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f15200098e0 tx=0x7f1520004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.300+0000 7f15297fa700 1 -- 172.21.15.93:0/739068559 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152001d070 con 0x7f152c101470 2023-11-16T16:51:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.300+0000 7f15297fa700 1 -- 172.21.15.93:0/739068559 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1520005030 con 0x7f152c101470 2023-11-16T16:51:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.300+0000 7f15327ff700 1 -- 172.21.15.93:0/739068559 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15200095f0 con 0x7f152c101470 2023-11-16T16:51:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.300+0000 7f15297fa700 1 -- 172.21.15.93:0/739068559 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520004b20 con 0x7f152c101470 2023-11-16T16:51:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.300+0000 7f15327ff700 1 -- 172.21.15.93:0/739068559 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c10bc40 con 0x7f152c101470 2023-11-16T16:51:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.303+0000 7f15297fa700 1 -- 172.21.15.93:0/739068559 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1520004cc0 con 0x7f152c101470 2023-11-16T16:51:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.305+0000 7f1546134b80 1 Processor -- start 2023-11-16T16:51:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.305+0000 7f1546134b80 1 -- start start 2023-11-16T16:51:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.305+0000 7f1546134b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cba1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.305+0000 7f1546134b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba1db0 0x55678cba25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.305+0000 7f1546134b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba2b10 0x55678cba2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.305+0000 7f1546134b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678caa7000 con 0x55678cba2b10 2023-11-16T16:51:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.305+0000 7f1546134b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678caf9840 con 0x55678c9fd6b0 2023-11-16T16:51:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.305+0000 7f1546134b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678caf2cd0 con 0x55678cba1db0 2023-11-16T16:51:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.305+0000 7f15297fa700 1 --2- 172.21.15.93:0/739068559 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15140a1150 0x7f15140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.305+0000 7f15297fa700 1 -- 172.21.15.93:0/739068559 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f15200cde80 con 0x7f152c101470 2023-11-16T16:51:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f1530d9c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cba1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f152bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba1db0 0x55678cba25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f1530d9c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cba1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:40998/0 (socket says 172.21.15.93:40998) 2023-11-16T16:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f1530d9c700 1 -- 172.21.15.93:0/2201667016 learned_addr learned my addr 172.21.15.93:0/2201667016 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f152b7fe700 1 --2- 172.21.15.93:0/2201667016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba2b10 0x55678cba2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f1530d9c700 1 --2- 172.21.15.93:0/739068559 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15140a1150 0x7f15140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f1530d9c700 1 -- 172.21.15.93:0/2201667016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba1db0 msgr2=0x55678cba25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f1530d9c700 1 --2- 172.21.15.93:0/2201667016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba1db0 0x55678cba25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f1530d9c700 1 -- 172.21.15.93:0/2201667016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba2b10 msgr2=0x55678cba2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f1530d9c700 1 --2- 172.21.15.93:0/2201667016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba2b10 0x55678cba2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f1530d9c700 1 -- 172.21.15.93:0/2201667016 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55678c9745d0 con 0x55678c9fd6b0 2023-11-16T16:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f152b7fe700 1 --2- 172.21.15.93:0/2201667016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba2b10 0x55678cba2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f152bfff700 1 --2- 172.21.15.93:0/2201667016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba1db0 0x55678cba25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.306+0000 7f1530d9c700 1 --2- 172.21.15.93:0/2201667016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cba1870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1524007d40 tx=0x7f1524019650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf0457480fee77a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.307+0000 7f1530d9c700 1 --2- 172.21.15.93:0/739068559 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15140a1150 0x7f15140a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f15240199a0 tx=0x7f152400d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.307+0000 7f15127fc700 1 -- 172.21.15.93:0/2201667016 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524007e90 con 0x55678c9fd6b0 2023-11-16T16:51:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.307+0000 7f15127fc700 1 -- 172.21.15.93:0/2201667016 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f152400f040 con 0x55678c9fd6b0 2023-11-16T16:51:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.307+0000 7f15127fc700 1 -- 172.21.15.93:0/2201667016 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524004e10 con 0x55678c9fd6b0 2023-11-16T16:51:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.307+0000 7f1546134b80 1 -- 172.21.15.93:0/2201667016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 msgr2=0x55678cba1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.307+0000 7f1546134b80 1 --2- 172.21.15.93:0/2201667016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cba1870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1524007d40 tx=0x7f1524019650 comp rx=0 tx=0).stop 2023-11-16T16:51:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.308+0000 7f1546134b80 1 -- 172.21.15.93:0/2201667016 shutdown_connections 2023-11-16T16:51:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.308+0000 7f1546134b80 1 --2- 172.21.15.93:0/2201667016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba2b10 0x55678cba2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.308+0000 7f1546134b80 1 --2- 172.21.15.93:0/2201667016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cba1870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.308+0000 7f1546134b80 1 --2- 172.21.15.93:0/2201667016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba1db0 0x55678cba25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.308+0000 7f1546134b80 1 -- 172.21.15.93:0/2201667016 >> 172.21.15.93:0/2201667016 conn(0x55678cb96f50 msgr2=0x55678cb9c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.308+0000 7f1546134b80 1 -- 172.21.15.93:0/2201667016 shutdown_connections 2023-11-16T16:51:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.308+0000 7f1546134b80 1 -- 172.21.15.93:0/2201667016 wait complete. 2023-11-16T16:51:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.308+0000 7f1546134b80 1 -- start start 2023-11-16T16:51:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f1546134b80 1 Processor -- start 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f1546134b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cbaaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f1546134b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba1db0 0x55678cbab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f1546134b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba2b10 0x55678cbb2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f1546134b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678ca94880 con 0x55678cba2b10 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f1546134b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678ca94520 con 0x55678c9fd6b0 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f1546134b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678c9768c0 con 0x55678cba1db0 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f1546134b80 10 client.0 _ll_register_callbacks cb 0x55678cab2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f152bfff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cbaaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f1530d9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba2b10 0x55678cbb2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f152bfff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cbaaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:41014/0 (socket says 172.21.15.93:41014) 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f1530d9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba2b10 0x55678cbb2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44906/0 (socket says 172.21.15.93:44906) 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.309+0000 7f152bfff700 1 -- 172.21.15.93:0/3702350747 learned_addr learned my addr 172.21.15.93:0/3702350747 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f152b7fe700 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba1db0 0x55678cbab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f152bfff700 1 -- 172.21.15.93:0/3702350747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba1db0 msgr2=0x55678cbab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f152bfff700 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba1db0 0x55678cbab1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f152bfff700 1 -- 172.21.15.93:0/3702350747 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba2b10 msgr2=0x55678cbb2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f152bfff700 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba2b10 0x55678cbb2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f152bfff700 1 -- 172.21.15.93:0/3702350747 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1524011040 con 0x55678c9fd6b0 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f152b7fe700 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba1db0 0x55678cbab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f152bfff700 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cbaaca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f151c011d10 tx=0x7f151c011d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41c9d38d4b9a1b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f150b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f150b7fe700 1 -- 172.21.15.93:0/3702350747 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c016070 con 0x55678c9fd6b0 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f150b7fe700 1 -- 172.21.15.93:0/3702350747 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f151c00fa40 con 0x55678c9fd6b0 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f1546134b80 1 -- 172.21.15.93:0/3702350747 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55678cb94710 con 0x55678c9fd6b0 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.310+0000 7f150b7fe700 1 -- 172.21.15.93:0/3702350747 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f151c00fc00 con 0x55678c9fd6b0 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.311+0000 7f1546134b80 10 client.25559 fetch_fsmap learned FSMap version 23 2023-11-16T16:51:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.311+0000 7f1546134b80 1 -- 172.21.15.93:0/3702350747 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55678cab9130 con 0x55678c9fd6b0 2023-11-16T16:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.311+0000 7f150b7fe700 1 -- 172.21.15.93:0/3702350747 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55678cb94710 con 0x55678c9fd6b0 2023-11-16T16:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.311+0000 7f150b7fe700 1 -- 172.21.15.93:0/3702350747 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f151c019390 con 0x55678c9fd6b0 2023-11-16T16:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.311+0000 7f1546134b80 10 client.25559 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.311+0000 7f1546134b80 10 client.25559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.311+0000 7f1546134b80 20 client.25559 populate_metadata read hostname 'smithi093' 2023-11-16T16:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.311+0000 7f1546134b80 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55678cbb6bd0 0x55678cbb8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.311+0000 7f1546134b80 4 client.25559 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.311+0000 7f1546134b80 1 -- 172.21.15.93:0/3702350747 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55678cab5040 con 0x55678cbb6bd0 2023-11-16T16:51:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.311+0000 7f152b7fe700 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55678cbb6bd0 0x55678cbb8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.312+0000 7f150b7fe700 10 client.25559 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:51:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.312+0000 7f152b7fe700 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55678cbb6bd0 0x55678cbb8fe0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9055df47aedcc server_cookie=1b00fa5a4e650b03 in_seq=0 out_seq=0 2023-11-16T16:51:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f150b7fe700 1 -- 172.21.15.93:0/3702350747 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55678cab5040 con 0x55678cbb6bd0 2023-11-16T16:51:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f150b7fe700 10 client.25559 handle_command_reply: tid=0 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f1546134b80 1 client.25559 shutdown 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f1546134b80 1 -- 172.21.15.93:0/3702350747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 msgr2=0x55678cbaaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f1546134b80 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cbaaca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f151c011d10 tx=0x7f151c011d40 comp rx=0 tx=0).stop 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f1546134b80 1 -- 172.21.15.93:0/3702350747 shutdown_connections 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f1546134b80 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55678cbb6bd0 0x55678cbb8fe0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f1546134b80 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba2b10 0x55678cbb2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f1546134b80 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cbaaca0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f150b7fe700 0 client.25559 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f1546134b80 1 --2- 172.21.15.93:0/3702350747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba1db0 0x55678cbab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f1546134b80 1 -- 172.21.15.93:0/3702350747 >> 172.21.15.93:0/3702350747 conn(0x55678cb96f50 msgr2=0x55678cb9b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.313+0000 7f1546134b80 1 -- 172.21.15.93:0/3702350747 shutdown_connections 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.314+0000 7f1546134b80 1 -- 172.21.15.93:0/3702350747 wait complete. 2023-11-16T16:51:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.314+0000 7f1546134b80 20 client.25559 trim_cache size 0 max 16384 2023-11-16T16:51:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.331+0000 7f1546134b80 1 Processor -- start 2023-11-16T16:51:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.331+0000 7f1546134b80 1 -- start start 2023-11-16T16:51:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.331+0000 7f1546134b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cba5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.331+0000 7f1546134b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba1db0 0x55678cba5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.331+0000 7f1546134b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba2b10 0x55678cbb2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.331+0000 7f1546134b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cbaa870 con 0x55678cba1db0 2023-11-16T16:51:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.331+0000 7f1546134b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cb9ccf0 con 0x55678c9fd6b0 2023-11-16T16:51:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.331+0000 7f1546134b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cbabcb0 con 0x55678cba2b10 2023-11-16T16:51:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.331+0000 7f152bfff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cba5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.331+0000 7f152bfff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cba5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:41018/0 (socket says 172.21.15.93:41018) 2023-11-16T16:51:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.331+0000 7f152bfff700 1 -- 172.21.15.93:0/898856136 learned_addr learned my addr 172.21.15.93:0/898856136 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.331+0000 7f152b7fe700 1 --2- 172.21.15.93:0/898856136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba1db0 0x55678cba5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.332+0000 7f1530d9c700 1 --2- 172.21.15.93:0/898856136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba2b10 0x55678cbb2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.332+0000 7f152bfff700 1 -- 172.21.15.93:0/898856136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba2b10 msgr2=0x55678cbb2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.332+0000 7f152bfff700 1 --2- 172.21.15.93:0/898856136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba2b10 0x55678cbb2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.332+0000 7f152bfff700 1 -- 172.21.15.93:0/898856136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba1db0 msgr2=0x55678cba5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.332+0000 7f152bfff700 1 --2- 172.21.15.93:0/898856136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba1db0 0x55678cba5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.332+0000 7f152bfff700 1 -- 172.21.15.93:0/898856136 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55678cab8e80 con 0x55678c9fd6b0 2023-11-16T16:51:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.332+0000 7f152b7fe700 1 --2- 172.21.15.93:0/898856136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba1db0 0x55678cba5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:51:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.332+0000 7f152bfff700 1 --2- 172.21.15.93:0/898856136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cba5a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f151c011ce0 tx=0x7f151c009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d94afd6a312c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.332+0000 7f1512ffd700 1 -- 172.21.15.93:0/898856136 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c016070 con 0x55678c9fd6b0 2023-11-16T16:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.332+0000 7f1512ffd700 1 -- 172.21.15.93:0/898856136 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151c019450 con 0x55678c9fd6b0 2023-11-16T16:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.332+0000 7f1512ffd700 1 -- 172.21.15.93:0/898856136 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c019b70 con 0x55678c9fd6b0 2023-11-16T16:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.333+0000 7f1546134b80 1 -- 172.21.15.93:0/898856136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 msgr2=0x55678cba5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.333+0000 7f1546134b80 1 --2- 172.21.15.93:0/898856136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cba5a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f151c011ce0 tx=0x7f151c009880 comp rx=0 tx=0).stop 2023-11-16T16:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.333+0000 7f1546134b80 1 -- 172.21.15.93:0/898856136 shutdown_connections 2023-11-16T16:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.333+0000 7f1546134b80 1 --2- 172.21.15.93:0/898856136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba2b10 0x55678cbb2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.333+0000 7f1546134b80 1 --2- 172.21.15.93:0/898856136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678c9fd6b0 0x55678cba5a00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.333+0000 7f1546134b80 1 --2- 172.21.15.93:0/898856136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678cba1db0 0x55678cba5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.333+0000 7f1546134b80 1 -- 172.21.15.93:0/898856136 >> 172.21.15.93:0/898856136 conn(0x55678cb96f50 msgr2=0x55678cba9990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.333+0000 7f1546134b80 1 -- 172.21.15.93:0/898856136 shutdown_connections 2023-11-16T16:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.333+0000 7f1546134b80 1 -- 172.21.15.93:0/898856136 wait complete. 2023-11-16T16:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.333+0000 7f1546134b80 1 -- start start 2023-11-16T16:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.334+0000 7f1546134b80 1 Processor -- start 2023-11-16T16:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.334+0000 7f1546134b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678c9fd6b0 0x55678cbadf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.334+0000 7f1546134b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678cba1db0 0x55678cbae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.334+0000 7f1546134b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba2b10 0x55678cbb14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.334+0000 7f1546134b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cbb1a10 con 0x55678c9fd6b0 2023-11-16T16:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.334+0000 7f1546134b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cbb1b80 con 0x55678cba1db0 2023-11-16T16:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.334+0000 7f1546134b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678cbb1d40 con 0x55678cba2b10 2023-11-16T16:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.334+0000 7f1546134b80 10 client.0 _ll_register_callbacks cb 0x7f151c0117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.334+0000 7f152b7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678cba1db0 0x55678cbae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.334+0000 7f152bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678c9fd6b0 0x55678cbadf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f152b7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678cba1db0 0x55678cbae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:41024/0 (socket says 172.21.15.93:41024) 2023-11-16T16:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f152b7fe700 1 -- 172.21.15.93:0/3063156539 learned_addr learned my addr 172.21.15.93:0/3063156539 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f1530d9c700 1 --2- 172.21.15.93:0/3063156539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba2b10 0x55678cbb14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f152b7fe700 1 -- 172.21.15.93:0/3063156539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba2b10 msgr2=0x55678cbb14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f152b7fe700 1 --2- 172.21.15.93:0/3063156539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba2b10 0x55678cbb14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f152b7fe700 1 -- 172.21.15.93:0/3063156539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678c9fd6b0 msgr2=0x55678cbadf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f152b7fe700 1 --2- 172.21.15.93:0/3063156539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678c9fd6b0 0x55678cbadf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f152b7fe700 1 -- 172.21.15.93:0/3063156539 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55678cab9130 con 0x55678cba1db0 2023-11-16T16:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f152b7fe700 1 --2- 172.21.15.93:0/3063156539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678cba1db0 0x55678cbae4b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1520000c00 tx=0x7f1520097230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56a282ee340338 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f150b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f150b7fe700 1 -- 172.21.15.93:0/3063156539 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15200993b0 con 0x55678cba1db0 2023-11-16T16:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f150b7fe700 1 -- 172.21.15.93:0/3063156539 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f152009b150 con 0x55678cba1db0 2023-11-16T16:51:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f1546134b80 1 -- 172.21.15.93:0/3063156539 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f152400f040 con 0x55678cba1db0 2023-11-16T16:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f150b7fe700 1 -- 172.21.15.93:0/3063156539 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1520005610 con 0x55678cba1db0 2023-11-16T16:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f1546134b80 10 client.25565 fetch_fsmap learned FSMap version 23 2023-11-16T16:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.335+0000 7f1546134b80 1 -- 172.21.15.93:0/3063156539 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55678c909c00 con 0x55678cba1db0 2023-11-16T16:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.336+0000 7f150b7fe700 1 -- 172.21.15.93:0/3063156539 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f152400f040 con 0x55678cba1db0 2023-11-16T16:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.336+0000 7f150b7fe700 1 -- 172.21.15.93:0/3063156539 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f15200a2e20 con 0x55678cba1db0 2023-11-16T16:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.336+0000 7f1546134b80 10 client.25565 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.336+0000 7f1546134b80 10 client.25565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.336+0000 7f1546134b80 20 client.25565 populate_metadata read hostname 'smithi093' 2023-11-16T16:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.336+0000 7f1546134b80 1 --2- 172.21.15.93:0/3063156539 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55678cc0b5e0 0x55678cc0d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.336+0000 7f1546134b80 4 client.25565 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.336+0000 7f1546134b80 1 -- 172.21.15.93:0/3063156539 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55678cac1600 con 0x55678cc0b5e0 2023-11-16T16:51:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.336+0000 7f152bfff700 1 --2- 172.21.15.93:0/3063156539 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55678cc0b5e0 0x55678cc0d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.337+0000 7f152bfff700 1 --2- 172.21.15.93:0/3063156539 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55678cc0b5e0 0x55678cc0d9f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae184ea1c3461ef3 server_cookie=b77dd6beef46999e in_seq=0 out_seq=0 2023-11-16T16:51:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:27.337+0000 7f150b7fe700 10 client.25565 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:51:28.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:28 smithi159 ceph-mon[131528]: pgmap v430: 129 pgs: 129 active+clean; 17 MiB data, 131 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 53 op/s 2023-11-16T16:51:28.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:28 smithi125 ceph-mon[131076]: pgmap v430: 129 pgs: 129 active+clean; 17 MiB data, 131 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 53 op/s 2023-11-16T16:51:28.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:28 smithi093 ceph-mon[97863]: pgmap v430: 129 pgs: 129 active+clean; 17 MiB data, 131 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 53 op/s 2023-11-16T16:51:30.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:30 smithi159 ceph-mon[131528]: pgmap v431: 129 pgs: 129 active+clean; 17 MiB data, 144 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 190 op/s 2023-11-16T16:51:30.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:30 smithi125 ceph-mon[131076]: pgmap v431: 129 pgs: 129 active+clean; 17 MiB data, 144 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 190 op/s 2023-11-16T16:51:30.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:30 smithi093 ceph-mon[97863]: pgmap v431: 129 pgs: 129 active+clean; 17 MiB data, 144 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 190 op/s 2023-11-16T16:51:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.092+0000 7f150b7fe700 1 -- 172.21.15.93:0/3063156539 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55678cac1600 con 0x55678cc0b5e0 2023-11-16T16:51:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.092+0000 7f150b7fe700 10 client.25565 handle_command_reply: tid=0 2023-11-16T16:51:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.092+0000 7f1546134b80 1 client.25565 shutdown 2023-11-16T16:51:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.092+0000 7f1546134b80 1 -- 172.21.15.93:0/3063156539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678cba1db0 msgr2=0x55678cbae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.092+0000 7f1546134b80 1 --2- 172.21.15.93:0/3063156539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678cba1db0 0x55678cbae4b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1520000c00 tx=0x7f1520097230 comp rx=0 tx=0).stop 2023-11-16T16:51:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.092+0000 7f1546134b80 1 -- 172.21.15.93:0/3063156539 shutdown_connections 2023-11-16T16:51:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.092+0000 7f1546134b80 1 --2- 172.21.15.93:0/3063156539 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55678cc0b5e0 0x55678cc0d9f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.092+0000 7f1546134b80 1 --2- 172.21.15.93:0/3063156539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55678c9fd6b0 0x55678cbadf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.092+0000 7f1546134b80 1 --2- 172.21.15.93:0/3063156539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55678cba2b10 0x55678cbb14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.092+0000 7f1546134b80 1 --2- 172.21.15.93:0/3063156539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55678cba1db0 0x55678cbae4b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.092+0000 7f1546134b80 1 -- 172.21.15.93:0/3063156539 >> 172.21.15.93:0/3063156539 conn(0x55678cb96f50 msgr2=0x55678cba5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.092+0000 7f150b7fe700 0 client.25565 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:51:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.093+0000 7f1546134b80 1 -- 172.21.15.93:0/3063156539 shutdown_connections 2023-11-16T16:51:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.093+0000 7f1546134b80 1 -- 172.21.15.93:0/3063156539 wait complete. 2023-11-16T16:51:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.093+0000 7f1546134b80 20 client.25565 trim_cache size 0 max 16384 2023-11-16T16:51:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:51:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:51:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:51:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:51:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.094+0000 7f15327ff700 1 -- 172.21.15.93:0/739068559 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15140a1150 msgr2=0x7f15140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.094+0000 7f15327ff700 1 --2- 172.21.15.93:0/739068559 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15140a1150 0x7f15140a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f15240199a0 tx=0x7f152400d000 comp rx=0 tx=0).stop 2023-11-16T16:51:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.094+0000 7f15327ff700 1 -- 172.21.15.93:0/739068559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c101470 msgr2=0x7f152c198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.094+0000 7f15327ff700 1 --2- 172.21.15.93:0/739068559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c101470 0x7f152c198730 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f15200098e0 tx=0x7f1520004730 comp rx=0 tx=0).stop 2023-11-16T16:51:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.094+0000 7f15327ff700 1 -- 172.21.15.93:0/739068559 shutdown_connections 2023-11-16T16:51:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.094+0000 7f15327ff700 1 --2- 172.21.15.93:0/739068559 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15140a1150 0x7f15140a3600 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.094+0000 7f15327ff700 1 --2- 172.21.15.93:0/739068559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f152c101e00 0x7f152c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.094+0000 7f15327ff700 1 --2- 172.21.15.93:0/739068559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f152c100280 0x7f152c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.094+0000 7f15327ff700 1 --2- 172.21.15.93:0/739068559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c101470 0x7f152c198730 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.094+0000 7f15327ff700 1 -- 172.21.15.93:0/739068559 >> 172.21.15.93:0/739068559 conn(0x7f152c0fb8a0 msgr2=0x7f152c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.094+0000 7f15327ff700 1 -- 172.21.15.93:0/739068559 shutdown_connections 2023-11-16T16:51:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:31.094+0000 7f15327ff700 1 -- 172.21.15.93:0/739068559 wait complete. 2023-11-16T16:51:32.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:32 smithi159 ceph-mon[131528]: pgmap v432: 129 pgs: 129 active+clean; 21 MiB data, 198 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 797 op/s 2023-11-16T16:51:32.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:32 smithi125 ceph-mon[131076]: pgmap v432: 129 pgs: 129 active+clean; 21 MiB data, 198 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 797 op/s 2023-11-16T16:51:32.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:32 smithi093 ceph-mon[97863]: pgmap v432: 129 pgs: 129 active+clean; 21 MiB data, 198 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 797 op/s 2023-11-16T16:51:34.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:34 smithi159 ceph-mon[131528]: pgmap v433: 129 pgs: 129 active+clean; 19 MiB data, 250 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 1.79k op/s 2023-11-16T16:51:34.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:34 smithi125 ceph-mon[131076]: pgmap v433: 129 pgs: 129 active+clean; 19 MiB data, 250 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 1.79k op/s 2023-11-16T16:51:34.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:34 smithi093 ceph-mon[97863]: pgmap v433: 129 pgs: 129 active+clean; 19 MiB data, 250 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 1.79k op/s 2023-11-16T16:51:36.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:36 smithi159 ceph-mon[131528]: pgmap v434: 129 pgs: 129 active+clean; 11 MiB data, 289 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 2.13k op/s 2023-11-16T16:51:36.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:36 smithi125 ceph-mon[131076]: pgmap v434: 129 pgs: 129 active+clean; 11 MiB data, 289 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 2.13k op/s 2023-11-16T16:51:36.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:36 smithi093 ceph-mon[97863]: pgmap v434: 129 pgs: 129 active+clean; 11 MiB data, 289 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 2.13k op/s 2023-11-16T16:51:38.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:38 smithi159 ceph-mon[131528]: pgmap v435: 129 pgs: 129 active+clean; 10 MiB data, 302 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 2.19k op/s 2023-11-16T16:51:38.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:38 smithi125 ceph-mon[131076]: pgmap v435: 129 pgs: 129 active+clean; 10 MiB data, 302 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 2.19k op/s 2023-11-16T16:51:38.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:38 smithi093 ceph-mon[97863]: pgmap v435: 129 pgs: 129 active+clean; 10 MiB data, 302 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 2.19k op/s 2023-11-16T16:51:40.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:40 smithi159 ceph-mon[131528]: pgmap v436: 129 pgs: 129 active+clean; 19 MiB data, 315 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 2.17k op/s 2023-11-16T16:51:40.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:40 smithi159 ceph-mon[131528]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:51:40.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:40 smithi125 ceph-mon[131076]: pgmap v436: 129 pgs: 129 active+clean; 19 MiB data, 315 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 2.17k op/s 2023-11-16T16:51:40.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:40 smithi125 ceph-mon[131076]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:51:40.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:40 smithi093 ceph-mon[97863]: pgmap v436: 129 pgs: 129 active+clean; 19 MiB data, 315 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 2.17k op/s 2023-11-16T16:51:40.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:40 smithi093 ceph-mon[97863]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:51:42.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:42 smithi125 ceph-mon[131076]: pgmap v437: 129 pgs: 129 active+clean; 23 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 2.03k op/s 2023-11-16T16:51:43.745 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:42 smithi159 ceph-mon[131528]: pgmap v437: 129 pgs: 129 active+clean; 23 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 2.03k op/s 2023-11-16T16:51:43.745 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:42 smithi093 ceph-mon[97863]: pgmap v437: 129 pgs: 129 active+clean; 23 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 2.03k op/s 2023-11-16T16:51:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.297+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2444330885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344101dc0 msgr2=0x7ff344108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.297+0000 7ff34b7ff700 1 --2- 172.21.15.93:0/2444330885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344101dc0 0x7ff344108680 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0098f0 tx=0x7ff33c009c00 comp rx=0 tx=0).stop 2023-11-16T16:51:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.297+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2444330885 shutdown_connections 2023-11-16T16:51:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.297+0000 7ff34b7ff700 1 --2- 172.21.15.93:0/2444330885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344101dc0 0x7ff344108680 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.297+0000 7ff34b7ff700 1 --2- 172.21.15.93:0/2444330885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff344100240 0x7ff344100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.297+0000 7ff34b7ff700 1 --2- 172.21.15.93:0/2444330885 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff344101430 0x7ff344101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.297+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2444330885 >> 172.21.15.93:0/2444330885 conn(0x7ff3440fb840 msgr2=0x7ff3440fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.297+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2444330885 shutdown_connections 2023-11-16T16:51:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.297+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2444330885 wait complete. 2023-11-16T16:51:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.298+0000 7ff34b7ff700 1 Processor -- start 2023-11-16T16:51:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.299+0000 7ff34b7ff700 1 -- start start 2023-11-16T16:51:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.299+0000 7ff34b7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344100240 0x7ff34406f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.299+0000 7ff34b7ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff344101430 0x7ff34406f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.299+0000 7ff34b7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff344101dc0 0x7ff3441a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.299+0000 7ff34b7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff344072940 con 0x7ff344101dc0 2023-11-16T16:51:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.299+0000 7ff34b7ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff344072ab0 con 0x7ff344101430 2023-11-16T16:51:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.299+0000 7ff34b7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3441a66d0 con 0x7ff344100240 2023-11-16T16:51:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.299+0000 7ff348d9a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff344101430 0x7ff34406f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.300+0000 7ff348d9a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff344101430 0x7ff34406f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:54012/0 (socket says 172.21.15.93:54012) 2023-11-16T16:51:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.300+0000 7ff348d9a700 1 -- 172.21.15.93:0/2959369442 learned_addr learned my addr 172.21.15.93:0/2959369442 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.300+0000 7ff34959b700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344100240 0x7ff34406f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.300+0000 7ff349d9c700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff344101dc0 0x7ff3441a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.300+0000 7ff348d9a700 1 -- 172.21.15.93:0/2959369442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344100240 msgr2=0x7ff34406f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.300+0000 7ff348d9a700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344100240 0x7ff34406f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.300+0000 7ff348d9a700 1 -- 172.21.15.93:0/2959369442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff344101dc0 msgr2=0x7ff3441a6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.300+0000 7ff348d9a700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff344101dc0 0x7ff3441a6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.300+0000 7ff348d9a700 1 -- 172.21.15.93:0/2959369442 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c009520 con 0x7ff344101430 2023-11-16T16:51:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.301+0000 7ff348d9a700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff344101430 0x7ff34406f860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff33400d980 tx=0x7ff33400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.301+0000 7ff33a7fc700 1 -- 172.21.15.93:0/2959369442 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33400f890 con 0x7ff344101430 2023-11-16T16:51:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.302+0000 7ff33a7fc700 1 -- 172.21.15.93:0/2959369442 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff334010460 con 0x7ff344101430 2023-11-16T16:51:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.302+0000 7ff33a7fc700 1 -- 172.21.15.93:0/2959369442 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3340195c0 con 0x7ff344101430 2023-11-16T16:51:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.302+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2959369442 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34410bba0 con 0x7ff344101430 2023-11-16T16:51:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.302+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2959369442 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34410ba10 con 0x7ff344101430 2023-11-16T16:51:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.306+0000 7ff33a7fc700 1 -- 172.21.15.93:0/2959369442 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff3340105d0 con 0x7ff344101430 2023-11-16T16:51:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.308+0000 7ff33a7fc700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff3300a12d0 0x7ff3300a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.308+0000 7ff33a7fc700 1 -- 172.21.15.93:0/2959369442 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7ff3340cc860 con 0x7ff344101430 2023-11-16T16:51:44.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.308+0000 7ff34959b700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff3300a12d0 0x7ff3300a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.309+0000 7ff35f1ecb80 1 Processor -- start 2023-11-16T16:51:44.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.309+0000 7ff35f1ecb80 1 -- start start 2023-11-16T16:51:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff35f1ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d50956c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff35f1ecb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d50956d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff35f1ecb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d50956df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff35f1ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d509472000 con 0x55d5093c86b0 2023-11-16T16:51:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff35f1ecb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5094c4840 con 0x55d50956db10 2023-11-16T16:51:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff35f1ecb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5094bdcd0 con 0x55d50956cdb0 2023-11-16T16:51:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff348d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d50956d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff34959b700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff3300a12d0 0x7ff3300a3780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff340009550 tx=0x7ff3400091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff348d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d50956d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:52072/0 (socket says 172.21.15.93:52072) 2023-11-16T16:51:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff348d9a700 1 -- 172.21.15.93:0/286528174 learned_addr learned my addr 172.21.15.93:0/286528174 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff349d9c700 1 --2- 172.21.15.93:0/286528174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d50956df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff348d9a700 1 -- 172.21.15.93:0/286528174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 msgr2=0x55d50956df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff348d9a700 1 --2- 172.21.15.93:0/286528174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d50956df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff348d9a700 1 -- 172.21.15.93:0/286528174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 msgr2=0x55d50956c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff348d9a700 1 --2- 172.21.15.93:0/286528174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d50956c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff348d9a700 1 -- 172.21.15.93:0/286528174 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d50933f5d0 con 0x55d50956cdb0 2023-11-16T16:51:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.310+0000 7ff349d9c700 1 --2- 172.21.15.93:0/286528174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d50956df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:51:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.311+0000 7ff348d9a700 1 --2- 172.21.15.93:0/286528174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d50956d5d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff334099830 tx=0x7ff334000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aac25b1950cedc92 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.311+0000 7ff32f7fe700 1 -- 172.21.15.93:0/286528174 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3340a1cf0 con 0x55d50956cdb0 2023-11-16T16:51:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.311+0000 7ff32f7fe700 1 -- 172.21.15.93:0/286528174 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff334097c40 con 0x55d50956cdb0 2023-11-16T16:51:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.311+0000 7ff32f7fe700 1 -- 172.21.15.93:0/286528174 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3340a1ac0 con 0x55d50956cdb0 2023-11-16T16:51:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.312+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/286528174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 msgr2=0x55d50956d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.312+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/286528174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d50956d5d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff334099830 tx=0x7ff334000bc0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.312+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/286528174 shutdown_connections 2023-11-16T16:51:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.312+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/286528174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d50956c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.312+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/286528174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d50956df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.312+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/286528174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d50956d5d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.312+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/286528174 >> 172.21.15.93:0/286528174 conn(0x55d509561f50 msgr2=0x55d509567100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.313+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/286528174 shutdown_connections 2023-11-16T16:51:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.313+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/286528174 wait complete. 2023-11-16T16:51:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.314+0000 7ff35f1ecb80 1 -- start start 2023-11-16T16:51:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.314+0000 7ff35f1ecb80 1 Processor -- start 2023-11-16T16:51:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.315+0000 7ff35f1ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d509575bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.315+0000 7ff35f1ecb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d509576130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.315+0000 7ff35f1ecb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d50957d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.315+0000 7ff35f1ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50945f880 con 0x55d5093c86b0 2023-11-16T16:51:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.315+0000 7ff35f1ecb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50945f520 con 0x55d50956db10 2023-11-16T16:51:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.315+0000 7ff35f1ecb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5093418c0 con 0x55d50956cdb0 2023-11-16T16:51:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.315+0000 7ff35f1ecb80 10 client.0 _ll_register_callbacks cb 0x55d50947d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:51:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.315+0000 7ff34959b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d509575bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.316+0000 7ff34959b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d509575bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41402/0 (socket says 172.21.15.93:41402) 2023-11-16T16:51:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.316+0000 7ff34959b700 1 -- 172.21.15.93:0/3601697991 learned_addr learned my addr 172.21.15.93:0/3601697991 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.316+0000 7ff349d9c700 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d50957d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.316+0000 7ff348d9a700 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d509576130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.316+0000 7ff349d9c700 1 -- 172.21.15.93:0/3601697991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 msgr2=0x55d509576130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.316+0000 7ff349d9c700 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d509576130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.316+0000 7ff349d9c700 1 -- 172.21.15.93:0/3601697991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 msgr2=0x55d509575bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.316+0000 7ff349d9c700 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d509575bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.316+0000 7ff349d9c700 1 -- 172.21.15.93:0/3601697991 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34410ba10 con 0x55d50956db10 2023-11-16T16:51:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.316+0000 7ff34959b700 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d509575bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:51:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.317+0000 7ff349d9c700 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d50957d170 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009810 tx=0x7ff33c010fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17dc778ab6e7094f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.317+0000 7ff32cff9700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:51:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.317+0000 7ff32cff9700 1 -- 172.21.15.93:0/3601697991 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c01c050 con 0x55d50956db10 2023-11-16T16:51:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.317+0000 7ff32cff9700 1 -- 172.21.15.93:0/3601697991 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7ff33c0054e0 con 0x55d50956db10 2023-11-16T16:51:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.318+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/3601697991 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d50955f710 con 0x55d50956db10 2023-11-16T16:51:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.318+0000 7ff32cff9700 1 -- 172.21.15.93:0/3601697991 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff33c00f9c0 con 0x55d50956db10 2023-11-16T16:51:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.319+0000 7ff35f1ecb80 10 client.25574 fetch_fsmap learned FSMap version 24 2023-11-16T16:51:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.319+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/3601697991 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d509484130 con 0x55d50956db10 2023-11-16T16:51:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.319+0000 7ff32cff9700 1 -- 172.21.15.93:0/3601697991 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff33c01d040 con 0x55d50956db10 2023-11-16T16:51:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.319+0000 7ff32cff9700 1 -- 172.21.15.93:0/3601697991 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff33c015020 con 0x55d50956db10 2023-11-16T16:51:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.319+0000 7ff35f1ecb80 10 client.25574 fetch_fsmap finished waiting for FSMap version 24 2023-11-16T16:51:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.319+0000 7ff35f1ecb80 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:51:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.319+0000 7ff35f1ecb80 20 client.25574 populate_metadata read hostname 'smithi093' 2023-11-16T16:51:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.319+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d5095929c0 0x55d509594dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.319+0000 7ff35f1ecb80 4 client.25574 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:51:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.319+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/3601697991 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d50955f710 con 0x55d5095929c0 2023-11-16T16:51:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.320+0000 7ff34959b700 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d5095929c0 0x55d509594dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.321+0000 7ff34959b700 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d5095929c0 0x55d509594dd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fcf2c8655ad6d50 server_cookie=4600be0d1d78e529 in_seq=0 out_seq=0 2023-11-16T16:51:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.321+0000 7ff32cff9700 10 client.25574 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:51:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.322+0000 7ff32cff9700 1 -- 172.21.15.93:0/3601697991 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d50955f710 con 0x55d5095929c0 2023-11-16T16:51:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.322+0000 7ff32cff9700 10 client.25574 handle_command_reply: tid=0 2023-11-16T16:51:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.322+0000 7ff35f1ecb80 1 client.25574 shutdown 2023-11-16T16:51:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.323+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/3601697991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 msgr2=0x55d50957d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.323+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d50957d170 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009810 tx=0x7ff33c010fa0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.324+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/3601697991 shutdown_connections 2023-11-16T16:51:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.324+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d5095929c0 0x55d509594dd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.324+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d509575bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.324+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d50957d170 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.324+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/3601697991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d509576130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.324+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/3601697991 >> 172.21.15.93:0/3601697991 conn(0x55d509561f50 msgr2=0x55d5095667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.324+0000 7ff32cff9700 0 client.25574 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:51:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.324+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/3601697991 shutdown_connections 2023-11-16T16:51:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.324+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/3601697991 wait complete. 2023-11-16T16:51:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.325+0000 7ff35f1ecb80 20 client.25574 trim_cache size 0 max 16384 2023-11-16T16:51:44.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.347+0000 7ff35f1ecb80 1 Processor -- start 2023-11-16T16:51:44.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.347+0000 7ff35f1ecb80 1 -- start start 2023-11-16T16:51:44.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.348+0000 7ff35f1ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d509599730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.348+0000 7ff35f1ecb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d50957d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.348+0000 7ff35f1ecb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d5095949d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.348+0000 7ff35f1ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d509568930 con 0x55d5093c86b0 2023-11-16T16:51:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.348+0000 7ff35f1ecb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d509573c60 con 0x55d50956db10 2023-11-16T16:51:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.348+0000 7ff35f1ecb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d509571550 con 0x55d50956cdb0 2023-11-16T16:51:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.348+0000 7ff348d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d50957d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.348+0000 7ff349d9c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d5095949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.348+0000 7ff348d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d50957d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:52100/0 (socket says 172.21.15.93:52100) 2023-11-16T16:51:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.348+0000 7ff348d9a700 1 -- 172.21.15.93:0/1394837079 learned_addr learned my addr 172.21.15.93:0/1394837079 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.348+0000 7ff349d9c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d5095949d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:54036/0 (socket says 172.21.15.93:54036) 2023-11-16T16:51:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.348+0000 7ff34959b700 1 --2- 172.21.15.93:0/1394837079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d509599730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.349+0000 7ff349d9c700 1 -- 172.21.15.93:0/1394837079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 msgr2=0x55d50957d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.349+0000 7ff349d9c700 1 --2- 172.21.15.93:0/1394837079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d50957d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.349+0000 7ff349d9c700 1 -- 172.21.15.93:0/1394837079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 msgr2=0x55d509599730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.349+0000 7ff349d9c700 1 --2- 172.21.15.93:0/1394837079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d509599730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.349+0000 7ff349d9c700 1 -- 172.21.15.93:0/1394837079 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d509483e80 con 0x55d50956db10 2023-11-16T16:51:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.349+0000 7ff349d9c700 1 --2- 172.21.15.93:0/1394837079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d5095949d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00fb90 tx=0x7ff33c00e3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eed68516d05d49 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.350+0000 7ff32ffff700 1 -- 172.21.15.93:0/1394837079 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c01c030 con 0x55d50956db10 2023-11-16T16:51:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.350+0000 7ff32ffff700 1 -- 172.21.15.93:0/1394837079 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff33c00b2c0 con 0x55d50956db10 2023-11-16T16:51:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.350+0000 7ff32ffff700 1 -- 172.21.15.93:0/1394837079 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c02bc00 con 0x55d50956db10 2023-11-16T16:51:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.351+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/1394837079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 msgr2=0x55d5095949d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.351+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/1394837079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d5095949d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00fb90 tx=0x7ff33c00e3e0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.352+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/1394837079 shutdown_connections 2023-11-16T16:51:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.352+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/1394837079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5093c86b0 0x55d509599730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.352+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/1394837079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d50956db10 0x55d5095949d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.352+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/1394837079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956cdb0 0x55d50957d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.352+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/1394837079 >> 172.21.15.93:0/1394837079 conn(0x55d509561f50 msgr2=0x55d509574920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.352+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/1394837079 shutdown_connections 2023-11-16T16:51:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.352+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/1394837079 wait complete. 2023-11-16T16:51:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.352+0000 7ff35f1ecb80 1 -- start start 2023-11-16T16:51:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.353+0000 7ff35f1ecb80 1 Processor -- start 2023-11-16T16:51:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.353+0000 7ff35f1ecb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5093c86b0 0x55d509578770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.353+0000 7ff35f1ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d50956cdb0 0x55d509578cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.353+0000 7ff35f1ecb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956db10 0x55d50957bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.353+0000 7ff35f1ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50957c180 con 0x55d50956cdb0 2023-11-16T16:51:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.353+0000 7ff35f1ecb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50957c2f0 con 0x55d5093c86b0 2023-11-16T16:51:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.353+0000 7ff35f1ecb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50957c4b0 con 0x55d50956db10 2023-11-16T16:51:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.353+0000 7ff35f1ecb80 10 client.0 _ll_register_callbacks cb 0x7ff33c00f790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:51:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.353+0000 7ff34959b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5093c86b0 0x55d509578770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.353+0000 7ff34959b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5093c86b0 0x55d509578770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:54040/0 (socket says 172.21.15.93:54040) 2023-11-16T16:51:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.353+0000 7ff34959b700 1 -- 172.21.15.93:0/2096294886 learned_addr learned my addr 172.21.15.93:0/2096294886 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:51:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.353+0000 7ff349d9c700 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956db10 0x55d50957bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.354+0000 7ff348d9a700 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d50956cdb0 0x55d509578cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.354+0000 7ff34959b700 1 -- 172.21.15.93:0/2096294886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956db10 msgr2=0x55d50957bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.354+0000 7ff34959b700 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956db10 0x55d50957bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.354+0000 7ff34959b700 1 -- 172.21.15.93:0/2096294886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d50956cdb0 msgr2=0x55d509578cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:51:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.354+0000 7ff34959b700 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d50956cdb0 0x55d509578cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.354+0000 7ff34959b700 1 -- 172.21.15.93:0/2096294886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d509484130 con 0x55d5093c86b0 2023-11-16T16:51:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.354+0000 7ff34959b700 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5093c86b0 0x55d509578770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff340009fd0 tx=0x7ff340004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4646d59e8ff1e2f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:51:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.354+0000 7ff32cff9700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:51:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.354+0000 7ff32cff9700 1 -- 172.21.15.93:0/2096294886 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34002c070 con 0x55d5093c86b0 2023-11-16T16:51:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.354+0000 7ff32cff9700 1 -- 172.21.15.93:0/2096294886 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7ff340015a50 con 0x55d5093c86b0 2023-11-16T16:51:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.355+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/2096294886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff334097c40 con 0x55d5093c86b0 2023-11-16T16:51:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.355+0000 7ff32cff9700 1 -- 172.21.15.93:0/2096294886 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff340015820 con 0x55d5093c86b0 2023-11-16T16:51:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.355+0000 7ff35f1ecb80 10 client.25580 fetch_fsmap learned FSMap version 24 2023-11-16T16:51:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.355+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/2096294886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d5092d4c00 con 0x55d5093c86b0 2023-11-16T16:51:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.355+0000 7ff32cff9700 1 -- 172.21.15.93:0/2096294886 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff334097c40 con 0x55d5093c86b0 2023-11-16T16:51:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.356+0000 7ff32cff9700 1 -- 172.21.15.93:0/2096294886 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff34002f020 con 0x55d5093c86b0 2023-11-16T16:51:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.356+0000 7ff35f1ecb80 10 client.25580 fetch_fsmap finished waiting for FSMap version 24 2023-11-16T16:51:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.356+0000 7ff35f1ecb80 10 client.25580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:51:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.356+0000 7ff35f1ecb80 20 client.25580 populate_metadata read hostname 'smithi093' 2023-11-16T16:51:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.356+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d5095e71e0 0x55d5095e95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:51:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.356+0000 7ff35f1ecb80 4 client.25580 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:51:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.356+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/2096294886 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d509480040 con 0x55d5095e71e0 2023-11-16T16:51:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.356+0000 7ff348d9a700 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d5095e71e0 0x55d5095e95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:51:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.357+0000 7ff32cff9700 10 client.25580 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:51:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:44.357+0000 7ff348d9a700 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d5095e71e0 0x55d5095e95f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d7f898bca42d391 server_cookie=c7f130d3b7b7b537 in_seq=0 out_seq=0 2023-11-16T16:51:44.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:44 smithi125 ceph-mon[131076]: pgmap v438: 129 pgs: 129 active+clean; 25 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 1.43k op/s 2023-11-16T16:51:44.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:44 smithi093 ceph-mon[97863]: pgmap v438: 129 pgs: 129 active+clean; 25 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 1.43k op/s 2023-11-16T16:51:44.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:44 smithi159 ceph-mon[131528]: pgmap v438: 129 pgs: 129 active+clean; 25 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 1.43k op/s 2023-11-16T16:51:46.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:46 smithi125 ceph-mon[131076]: pgmap v439: 129 pgs: 129 active+clean; 40 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 459 op/s 2023-11-16T16:51:46.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:46 smithi093 ceph-mon[97863]: pgmap v439: 129 pgs: 129 active+clean; 40 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 459 op/s 2023-11-16T16:51:46.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:46 smithi159 ceph-mon[131528]: pgmap v439: 129 pgs: 129 active+clean; 40 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 459 op/s 2023-11-16T16:51:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.178+0000 7ff32cff9700 1 -- 172.21.15.93:0/2096294886 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d509480040 con 0x55d5095e71e0 2023-11-16T16:51:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.178+0000 7ff32cff9700 10 client.25580 handle_command_reply: tid=0 2023-11-16T16:51:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.179+0000 7ff35f1ecb80 1 client.25580 shutdown 2023-11-16T16:51:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.179+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/2096294886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5093c86b0 msgr2=0x55d509578770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.179+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5093c86b0 0x55d509578770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff340009fd0 tx=0x7ff340004ad0 comp rx=0 tx=0).stop 2023-11-16T16:51:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.180+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/2096294886 shutdown_connections 2023-11-16T16:51:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.180+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55d5095e71e0 0x55d5095e95f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.180+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d50956db10 0x55d50957bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.180+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5093c86b0 0x55d509578770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.180+0000 7ff35f1ecb80 1 --2- 172.21.15.93:0/2096294886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d50956cdb0 0x55d509578cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.180+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/2096294886 >> 172.21.15.93:0/2096294886 conn(0x55d509561f50 msgr2=0x55d509574920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.180+0000 7ff32cff9700 0 client.25580 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:51:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.181+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/2096294886 shutdown_connections 2023-11-16T16:51:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.181+0000 7ff35f1ecb80 1 -- 172.21.15.93:0/2096294886 wait complete. 2023-11-16T16:51:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.182+0000 7ff35f1ecb80 20 client.25580 trim_cache size 0 max 16384 2023-11-16T16:51:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:51:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:51:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:51:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:51:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.183+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2959369442 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff3300a12d0 msgr2=0x7ff3300a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.183+0000 7ff34b7ff700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff3300a12d0 0x7ff3300a3780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff340009550 tx=0x7ff3400091b0 comp rx=0 tx=0).stop 2023-11-16T16:51:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.183+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2959369442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff344101430 msgr2=0x7ff34406f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:51:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.183+0000 7ff34b7ff700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff344101430 0x7ff34406f860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff33400d980 tx=0x7ff33400dc90 comp rx=0 tx=0).stop 2023-11-16T16:51:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.183+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2959369442 shutdown_connections 2023-11-16T16:51:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.183+0000 7ff34b7ff700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff3300a12d0 0x7ff3300a3780 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.183+0000 7ff34b7ff700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff344101dc0 0x7ff3441a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.183+0000 7ff34b7ff700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344100240 0x7ff34406f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.183+0000 7ff34b7ff700 1 --2- 172.21.15.93:0/2959369442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff344101430 0x7ff34406f860 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:51:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.183+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2959369442 >> 172.21.15.93:0/2959369442 conn(0x7ff3440fb840 msgr2=0x7ff3441048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:51:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.185+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2959369442 shutdown_connections 2023-11-16T16:51:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:51:47.186+0000 7ff34b7ff700 1 -- 172.21.15.93:0/2959369442 wait complete. 2023-11-16T16:51:48.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:48 smithi125 ceph-mon[131076]: pgmap v440: 129 pgs: 129 active+clean; 40 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 463 op/s 2023-11-16T16:51:48.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:48 smithi093 ceph-mon[97863]: pgmap v440: 129 pgs: 129 active+clean; 40 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 463 op/s 2023-11-16T16:51:48.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:48 smithi159 ceph-mon[131528]: pgmap v440: 129 pgs: 129 active+clean; 40 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 463 op/s 2023-11-16T16:51:50.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:50 smithi125 ceph-mon[131076]: pgmap v441: 129 pgs: 129 active+clean; 29 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 1.06k op/s 2023-11-16T16:51:50.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:50 smithi093 ceph-mon[97863]: pgmap v441: 129 pgs: 129 active+clean; 29 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 1.06k op/s 2023-11-16T16:51:50.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:50 smithi159 ceph-mon[131528]: pgmap v441: 129 pgs: 129 active+clean; 29 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 1.06k op/s 2023-11-16T16:51:52.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:52 smithi125 ceph-mon[131076]: pgmap v442: 129 pgs: 129 active+clean; 17 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 1.51k op/s 2023-11-16T16:51:52.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:52 smithi093 ceph-mon[97863]: pgmap v442: 129 pgs: 129 active+clean; 17 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 1.51k op/s 2023-11-16T16:51:52.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:52 smithi159 ceph-mon[131528]: pgmap v442: 129 pgs: 129 active+clean; 17 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 1.51k op/s 2023-11-16T16:51:54.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:54 smithi125 ceph-mon[131076]: pgmap v443: 129 pgs: 129 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 1.66k op/s 2023-11-16T16:51:54.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:54 smithi093 ceph-mon[97863]: pgmap v443: 129 pgs: 129 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 1.66k op/s 2023-11-16T16:51:54.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:54 smithi159 ceph-mon[131528]: pgmap v443: 129 pgs: 129 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 1.66k op/s 2023-11-16T16:51:56.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:56 smithi125 ceph-mon[131076]: pgmap v444: 129 pgs: 129 active+clean; 29 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 1.66k op/s 2023-11-16T16:51:56.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:56 smithi093 ceph-mon[97863]: pgmap v444: 129 pgs: 129 active+clean; 29 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 1.66k op/s 2023-11-16T16:51:56.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:56 smithi159 ceph-mon[131528]: pgmap v444: 129 pgs: 129 active+clean; 29 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 1.66k op/s 2023-11-16T16:51:57.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:57 smithi125 ceph-mon[131076]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:51:57.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:57 smithi093 ceph-mon[97863]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:51:57.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:57 smithi159 ceph-mon[131528]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:51:58.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:51:58 smithi125 ceph-mon[131076]: pgmap v445: 129 pgs: 129 active+clean; 30 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 1.64k op/s 2023-11-16T16:51:58.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:51:58 smithi093 ceph-mon[97863]: pgmap v445: 129 pgs: 129 active+clean; 30 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 1.64k op/s 2023-11-16T16:51:58.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:51:58 smithi159 ceph-mon[131528]: pgmap v445: 129 pgs: 129 active+clean; 30 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 1.64k op/s 2023-11-16T16:52:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.354+0000 7fc285b9d700 1 -- 172.21.15.93:0/3380596355 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2801012d0 msgr2=0x7fc280101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.354+0000 7fc285b9d700 1 --2- 172.21.15.93:0/3380596355 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2801012d0 0x7fc280101740 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0098f0 tx=0x7fc26c009c00 comp rx=0 tx=0).stop 2023-11-16T16:52:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.354+0000 7fc285b9d700 1 -- 172.21.15.93:0/3380596355 shutdown_connections 2023-11-16T16:52:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.354+0000 7fc285b9d700 1 --2- 172.21.15.93:0/3380596355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc280101c80 0x7fc280108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.354+0000 7fc285b9d700 1 --2- 172.21.15.93:0/3380596355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc280100040 0x7fc280100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.354+0000 7fc285b9d700 1 --2- 172.21.15.93:0/3380596355 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2801012d0 0x7fc280101740 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.354+0000 7fc285b9d700 1 -- 172.21.15.93:0/3380596355 >> 172.21.15.93:0/3380596355 conn(0x7fc2800fb640 msgr2=0x7fc2800fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.354+0000 7fc285b9d700 1 -- 172.21.15.93:0/3380596355 shutdown_connections 2023-11-16T16:52:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.354+0000 7fc285b9d700 1 -- 172.21.15.93:0/3380596355 wait complete. 2023-11-16T16:52:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.355+0000 7fc285b9d700 1 Processor -- start 2023-11-16T16:52:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.355+0000 7fc285b9d700 1 -- start start 2023-11-16T16:52:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.356+0000 7fc285b9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc280100040 0x7fc280198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.356+0000 7fc285b9d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2801012d0 0x7fc280198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.356+0000 7fc285b9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc280101c80 0x7fc28019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.356+0000 7fc285b9d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28019fbd0 con 0x7fc280100040 2023-11-16T16:52:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.356+0000 7fc285b9d700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28019fd10 con 0x7fc2801012d0 2023-11-16T16:52:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.356+0000 7fc285b9d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28019fea0 con 0x7fc280101c80 2023-11-16T16:52:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.356+0000 7fc27ffff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2801012d0 0x7fc280198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.356+0000 7fc28539c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc280101c80 0x7fc28019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.356+0000 7fc27ffff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2801012d0 0x7fc280198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:39296/0 (socket says 172.21.15.93:39296) 2023-11-16T16:52:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.356+0000 7fc28539c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc280101c80 0x7fc28019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:33742/0 (socket says 172.21.15.93:33742) 2023-11-16T16:52:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.356+0000 7fc27ffff700 1 -- 172.21.15.93:0/2069886618 learned_addr learned my addr 172.21.15.93:0/2069886618 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.356+0000 7fc284b9b700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc280100040 0x7fc280198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.357+0000 7fc27ffff700 1 -- 172.21.15.93:0/2069886618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc280101c80 msgr2=0x7fc28019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.357+0000 7fc27ffff700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc280101c80 0x7fc28019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.357+0000 7fc27ffff700 1 -- 172.21.15.93:0/2069886618 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc280100040 msgr2=0x7fc280198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.357+0000 7fc27ffff700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc280100040 0x7fc280198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.357+0000 7fc27ffff700 1 -- 172.21.15.93:0/2069886618 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc278009d00 con 0x7fc2801012d0 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.357+0000 7fc284b9b700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc280100040 0x7fc280198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.357+0000 7fc27ffff700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2801012d0 0x7fc280198590 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009fd0 tx=0x7fc26c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.357+0000 7fc27dffb700 1 -- 172.21.15.93:0/2069886618 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c01c070 con 0x7fc2801012d0 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.357+0000 7fc27dffb700 1 -- 172.21.15.93:0/2069886618 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc26c00f7d0 con 0x7fc2801012d0 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.357+0000 7fc27dffb700 1 -- 172.21.15.93:0/2069886618 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c004ad0 con 0x7fc2801012d0 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.357+0000 7fc285b9d700 1 -- 172.21.15.93:0/2069886618 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26c009520 con 0x7fc2801012d0 2023-11-16T16:52:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.357+0000 7fc285b9d700 1 -- 172.21.15.93:0/2069886618 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28010ba00 con 0x7fc2801012d0 2023-11-16T16:52:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.360+0000 7fc27dffb700 1 -- 172.21.15.93:0/2069886618 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc26c020070 con 0x7fc2801012d0 2023-11-16T16:52:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.362+0000 7fc29a7e1b80 1 Processor -- start 2023-11-16T16:52:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.362+0000 7fc29a7e1b80 1 -- start start 2023-11-16T16:52:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.362+0000 7fc29a7e1b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544eba76b0 0x56544ed4ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.362+0000 7fc29a7e1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544ed4bfb0 0x56544ed4c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.362+0000 7fc27dffb700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc2700a11d0 0x7fc2700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.362+0000 7fc29a7e1b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed4d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.362+0000 7fc29a7e1b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544ec51000 con 0x56544eba76b0 2023-11-16T16:52:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.362+0000 7fc29a7e1b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544eca3840 con 0x56544ed4cd60 2023-11-16T16:52:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.362+0000 7fc29a7e1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544ec9ccd0 con 0x56544ed4bfb0 2023-11-16T16:52:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.362+0000 7fc27dffb700 1 -- 172.21.15.93:0/2069886618 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fc26c0cd8d0 con 0x7fc2801012d0 2023-11-16T16:52:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc27ffff700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc2700a11d0 0x7fc2700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc284b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544ed4bfb0 0x56544ed4c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc284b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544ed4bfb0 0x56544ed4c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:33758/0 (socket says 172.21.15.93:33758) 2023-11-16T16:52:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc284b9b700 1 -- 172.21.15.93:0/1777138826 learned_addr learned my addr 172.21.15.93:0/1777138826 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc28539c700 1 --2- 172.21.15.93:0/1777138826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed4d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc28539c700 1 --2- 172.21.15.93:0/1777138826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544eba76b0 0x56544ed4ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc284b9b700 1 -- 172.21.15.93:0/1777138826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 msgr2=0x56544ed4d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc27ffff700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc2700a11d0 0x7fc2700a3680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00fa40 tx=0x7fc26c0a0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc284b9b700 1 --2- 172.21.15.93:0/1777138826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed4d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc284b9b700 1 -- 172.21.15.93:0/1777138826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544eba76b0 msgr2=0x56544ed4ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc284b9b700 1 --2- 172.21.15.93:0/1777138826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544eba76b0 0x56544ed4ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc284b9b700 1 -- 172.21.15.93:0/1777138826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56544eb1e5d0 con 0x56544ed4bfb0 2023-11-16T16:52:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.363+0000 7fc28539c700 1 --2- 172.21.15.93:0/1777138826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed4d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:52:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.364+0000 7fc284b9b700 1 --2- 172.21.15.93:0/1777138826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544ed4bfb0 0x56544ed4c820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2740115f0 tx=0x7fc274011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29877660d2bc474a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.364+0000 7fc26affd700 1 -- 172.21.15.93:0/1777138826 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc274019630 con 0x56544ed4bfb0 2023-11-16T16:52:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.364+0000 7fc26affd700 1 -- 172.21.15.93:0/1777138826 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc274019c50 con 0x56544ed4bfb0 2023-11-16T16:52:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.364+0000 7fc26affd700 1 -- 172.21.15.93:0/1777138826 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27400eb00 con 0x56544ed4bfb0 2023-11-16T16:52:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.365+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/1777138826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544ed4bfb0 msgr2=0x56544ed4c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.365+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/1777138826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544ed4bfb0 0x56544ed4c820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2740115f0 tx=0x7fc274011900 comp rx=0 tx=0).stop 2023-11-16T16:52:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.365+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/1777138826 shutdown_connections 2023-11-16T16:52:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.365+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/1777138826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544eba76b0 0x56544ed4ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.365+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/1777138826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed4d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.365+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/1777138826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544ed4bfb0 0x56544ed4c820 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.365+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/1777138826 >> 172.21.15.93:0/1777138826 conn(0x56544ed411a0 msgr2=0x56544ed462e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.365+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/1777138826 shutdown_connections 2023-11-16T16:52:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.365+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/1777138826 wait complete. 2023-11-16T16:52:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.365+0000 7fc29a7e1b80 1 -- start start 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.366+0000 7fc29a7e1b80 1 Processor -- start 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.366+0000 7fc29a7e1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 0x56544ed54ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.366+0000 7fc29a7e1b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.366+0000 7fc29a7e1b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed5c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.366+0000 7fc29a7e1b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544ec3e880 con 0x56544ed4bfb0 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.366+0000 7fc29a7e1b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544ec3e520 con 0x56544ed4cd60 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.366+0000 7fc29a7e1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544eb208c0 con 0x56544eba76b0 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.366+0000 7fc29a7e1b80 10 client.0 _ll_register_callbacks cb 0x56544ec5c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.366+0000 7fc28539c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 0x56544ed54ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.366+0000 7fc28539c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 0x56544ed54ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:33766/0 (socket says 172.21.15.93:33766) 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.366+0000 7fc28539c700 1 -- 172.21.15.93:0/4017841545 learned_addr learned my addr 172.21.15.93:0/4017841545 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.366+0000 7fc28539c700 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed5c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.367+0000 7fc284b9b700 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.367+0000 7fc28539c700 1 -- 172.21.15.93:0/4017841545 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 msgr2=0x56544ed5c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.367+0000 7fc28539c700 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed5c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.367+0000 7fc28539c700 1 -- 172.21.15.93:0/4017841545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 msgr2=0x56544ed553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.367+0000 7fc28539c700 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.367+0000 7fc28539c700 1 -- 172.21.15.93:0/4017841545 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc274009dd0 con 0x56544eba76b0 2023-11-16T16:52:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.367+0000 7fc284b9b700 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:52:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.367+0000 7fc28539c700 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 0x56544ed54ea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc27801c820 tx=0x7fc278004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c33696c15d07473 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.367+0000 7fc25ffff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-11-16T16:52:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.367+0000 7fc25ffff700 1 -- 172.21.15.93:0/4017841545 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27800e050 con 0x56544eba76b0 2023-11-16T16:52:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.367+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4017841545 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56544ed3e960 con 0x56544eba76b0 2023-11-16T16:52:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.367+0000 7fc25ffff700 1 -- 172.21.15.93:0/4017841545 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fc278004bb0 con 0x56544eba76b0 2023-11-16T16:52:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.368+0000 7fc25ffff700 1 -- 172.21.15.93:0/4017841545 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc278004d70 con 0x56544eba76b0 2023-11-16T16:52:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.368+0000 7fc29a7e1b80 10 client.25588 fetch_fsmap learned FSMap version 25 2023-11-16T16:52:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.368+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4017841545 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56544ec63130 con 0x56544eba76b0 2023-11-16T16:52:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.368+0000 7fc25ffff700 1 -- 172.21.15.93:0/4017841545 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56544ed3e960 con 0x56544eba76b0 2023-11-16T16:52:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.368+0000 7fc25ffff700 1 -- 172.21.15.93:0/4017841545 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc278018720 con 0x56544eba76b0 2023-11-16T16:52:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.368+0000 7fc29a7e1b80 10 client.25588 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:52:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.368+0000 7fc29a7e1b80 10 client.25588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:52:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.368+0000 7fc29a7e1b80 20 client.25588 populate_metadata read hostname 'smithi093' 2023-11-16T16:52:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.368+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56544ed60cb0 0x56544ed630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.368+0000 7fc29a7e1b80 4 client.25588 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:52:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.368+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4017841545 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56544ec5f040 con 0x56544ed60cb0 2023-11-16T16:52:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.369+0000 7fc284b9b700 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56544ed60cb0 0x56544ed630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.369+0000 7fc284b9b700 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56544ed60cb0 0x56544ed630c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86708b29f3c10d96 server_cookie=7a1b2af732d8d78a in_seq=0 out_seq=0 2023-11-16T16:52:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.369+0000 7fc25ffff700 10 client.25588 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.370+0000 7fc25ffff700 1 -- 172.21.15.93:0/4017841545 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56544ec5f040 con 0x56544ed60cb0 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.370+0000 7fc25ffff700 10 client.25588 handle_command_reply: tid=0 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc29a7e1b80 1 client.25588 shutdown 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4017841545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 msgr2=0x56544ed54ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 0x56544ed54ea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc27801c820 tx=0x7fc278004b40 comp rx=0 tx=0).stop 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4017841545 shutdown_connections 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56544ed60cb0 0x56544ed630c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed5c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc25ffff700 0 client.25588 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 0x56544ed54ea0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/4017841545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4017841545 >> 172.21.15.93:0/4017841545 conn(0x56544ed411a0 msgr2=0x56544ed45a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4017841545 shutdown_connections 2023-11-16T16:52:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4017841545 wait complete. 2023-11-16T16:52:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.371+0000 7fc29a7e1b80 20 client.25588 trim_cache size 0 max 16384 2023-11-16T16:52:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.387+0000 7fc29a7e1b80 1 Processor -- start 2023-11-16T16:52:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.387+0000 7fc29a7e1b80 1 -- start start 2023-11-16T16:52:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.387+0000 7fc29a7e1b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544eba76b0 0x56544ed4d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.387+0000 7fc29a7e1b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed5c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.387+0000 7fc29a7e1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544ed4cd60 0x56544ed679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.387+0000 7fc29a7e1b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544ed4d670 con 0x56544ed4bfb0 2023-11-16T16:52:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.387+0000 7fc29a7e1b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544ed4c640 con 0x56544eba76b0 2023-11-16T16:52:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.387+0000 7fc29a7e1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544ed52e30 con 0x56544ed4cd60 2023-11-16T16:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc28539c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544eba76b0 0x56544ed4d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc284b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed5c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc284b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed5c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42288/0 (socket says 172.21.15.93:42288) 2023-11-16T16:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc28539c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544eba76b0 0x56544ed4d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:39310/0 (socket says 172.21.15.93:39310) 2023-11-16T16:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc284b9b700 1 -- 172.21.15.93:0/2251915657 learned_addr learned my addr 172.21.15.93:0/2251915657 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc28539c700 1 --2- 172.21.15.93:0/2251915657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544ed4cd60 0x56544ed679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc28539c700 1 -- 172.21.15.93:0/2251915657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544ed4cd60 msgr2=0x56544ed679e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc28539c700 1 --2- 172.21.15.93:0/2251915657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544ed4cd60 0x56544ed679e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc28539c700 1 -- 172.21.15.93:0/2251915657 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 msgr2=0x56544ed5c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc28539c700 1 --2- 172.21.15.93:0/2251915657 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed5c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc28539c700 1 -- 172.21.15.93:0/2251915657 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56544ec62e80 con 0x56544eba76b0 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc284b9b700 1 --2- 172.21.15.93:0/2251915657 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed5c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.388+0000 7fc28539c700 1 --2- 172.21.15.93:0/2251915657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544eba76b0 0x56544ed4d130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc27800e180 tx=0x7fc278004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b05a84dd4c2a829d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.389+0000 7fc26b7fe700 1 -- 172.21.15.93:0/2251915657 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278008a90 con 0x56544eba76b0 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.389+0000 7fc26b7fe700 1 -- 172.21.15.93:0/2251915657 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc278022030 con 0x56544eba76b0 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.389+0000 7fc26b7fe700 1 -- 172.21.15.93:0/2251915657 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27801a650 con 0x56544eba76b0 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.389+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/2251915657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544eba76b0 msgr2=0x56544ed4d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.389+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/2251915657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544eba76b0 0x56544ed4d130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc27800e180 tx=0x7fc278004ad0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.389+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/2251915657 shutdown_connections 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.389+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/2251915657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544ed4cd60 0x56544ed679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.389+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/2251915657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544eba76b0 0x56544ed4d130 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.389+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/2251915657 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed5c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.389+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/2251915657 >> 172.21.15.93:0/2251915657 conn(0x56544ed411a0 msgr2=0x56544ed53c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.389+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/2251915657 shutdown_connections 2023-11-16T16:52:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.390+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/2251915657 wait complete. 2023-11-16T16:52:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.390+0000 7fc29a7e1b80 1 -- start start 2023-11-16T16:52:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.390+0000 7fc29a7e1b80 1 Processor -- start 2023-11-16T16:52:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc29a7e1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 0x56544ed57fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc29a7e1b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed58520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc29a7e1b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed5b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc29a7e1b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544ed5b9a0 con 0x56544ed4bfb0 2023-11-16T16:52:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc29a7e1b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544ed5bb10 con 0x56544ed4cd60 2023-11-16T16:52:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc29a7e1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544ed5bcd0 con 0x56544eba76b0 2023-11-16T16:52:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc29a7e1b80 10 client.0 _ll_register_callbacks cb 0x56544ec5c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:52:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc28539c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 0x56544ed57fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc284b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed58520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc28539c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 0x56544ed57fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:33778/0 (socket says 172.21.15.93:33778) 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc28539c700 1 -- 172.21.15.93:0/4262863333 learned_addr learned my addr 172.21.15.93:0/4262863333 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc284b9b700 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed58520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42304/0 (socket says 172.21.15.93:42304) 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc28539c700 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed5b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc284b9b700 1 -- 172.21.15.93:0/4262863333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 msgr2=0x56544ed57fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc284b9b700 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 0x56544ed57fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc284b9b700 1 -- 172.21.15.93:0/4262863333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 msgr2=0x56544ed5b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc284b9b700 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed5b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc284b9b700 1 -- 172.21.15.93:0/4262863333 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56544eb1e5d0 con 0x56544ed4bfb0 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc284b9b700 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed58520 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc274011530 tx=0x7fc274009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c593c45e3a363853 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.391+0000 7fc25ffff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc25ffff700 1 -- 172.21.15.93:0/4262863333 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2740056c0 con 0x56544ed4bfb0 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4262863333 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc274019c50 con 0x56544ed4bfb0 2023-11-16T16:52:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc25ffff700 1 -- 172.21.15.93:0/4262863333 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fc27402a070 con 0x56544ed4bfb0 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc25ffff700 1 -- 172.21.15.93:0/4262863333 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc274019430 con 0x56544ed4bfb0 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc29a7e1b80 10 client.15561 fetch_fsmap learned FSMap version 25 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4262863333 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56544ec63130 con 0x56544ed4bfb0 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc25ffff700 1 -- 172.21.15.93:0/4262863333 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc274019c50 con 0x56544ed4bfb0 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc25ffff700 1 -- 172.21.15.93:0/4262863333 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 25) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc27402c070 con 0x56544ed4bfb0 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc29a7e1b80 10 client.15561 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc29a7e1b80 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc29a7e1b80 20 client.15561 populate_metadata read hostname 'smithi093' 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56544edb5800 0x56544edb7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc29a7e1b80 4 client.15561 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.392+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4262863333 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56544ec6b600 con 0x56544edb5800 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.393+0000 7fc28539c700 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56544edb5800 0x56544edb7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.393+0000 7fc25ffff700 10 client.15561 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:00.393+0000 7fc28539c700 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56544edb5800 0x56544edb7c10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d27928c0d6f5635 server_cookie=506c653c3724506f in_seq=0 out_seq=0 2023-11-16T16:52:00.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:00 smithi125 ceph-mon[131076]: pgmap v446: 129 pgs: 129 active+clean; 34 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 1.30k op/s 2023-11-16T16:52:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:00 smithi093 ceph-mon[97863]: pgmap v446: 129 pgs: 129 active+clean; 34 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 1.30k op/s 2023-11-16T16:52:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:00 smithi159 ceph-mon[131528]: pgmap v446: 129 pgs: 129 active+clean; 34 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 1.30k op/s 2023-11-16T16:52:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc25ffff700 1 -- 172.21.15.93:0/4262863333 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56544ec6b600 con 0x56544edb5800 2023-11-16T16:52:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc25ffff700 10 client.15561 handle_command_reply: tid=0 2023-11-16T16:52:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc29a7e1b80 1 client.15561 shutdown 2023-11-16T16:52:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4262863333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 msgr2=0x56544ed58520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed58520 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc274011530 tx=0x7fc274009750 comp rx=0 tx=0).stop 2023-11-16T16:52:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4262863333 shutdown_connections 2023-11-16T16:52:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56544edb5800 0x56544edb7c10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56544ed4cd60 0x56544ed5b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544eba76b0 0x56544ed57fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc29a7e1b80 1 --2- 172.21.15.93:0/4262863333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56544ed4bfb0 0x56544ed58520 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4262863333 >> 172.21.15.93:0/4262863333 conn(0x56544ed411a0 msgr2=0x56544ed679e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc25ffff700 0 client.15561 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4262863333 shutdown_connections 2023-11-16T16:52:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.808+0000 7fc29a7e1b80 1 -- 172.21.15.93:0/4262863333 wait complete. 2023-11-16T16:52:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.809+0000 7fc29a7e1b80 20 client.15561 trim_cache size 0 max 16384 2023-11-16T16:52:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:52:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:52:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:52:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:52:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.810+0000 7fc285b9d700 1 -- 172.21.15.93:0/2069886618 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc2700a11d0 msgr2=0x7fc2700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.810+0000 7fc285b9d700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc2700a11d0 0x7fc2700a3680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00fa40 tx=0x7fc26c0a0040 comp rx=0 tx=0).stop 2023-11-16T16:52:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.810+0000 7fc285b9d700 1 -- 172.21.15.93:0/2069886618 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2801012d0 msgr2=0x7fc280198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.810+0000 7fc285b9d700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2801012d0 0x7fc280198590 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009fd0 tx=0x7fc26c00b790 comp rx=0 tx=0).stop 2023-11-16T16:52:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.810+0000 7fc285b9d700 1 -- 172.21.15.93:0/2069886618 shutdown_connections 2023-11-16T16:52:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.810+0000 7fc285b9d700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fc2700a11d0 0x7fc2700a3680 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.810+0000 7fc285b9d700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc280100040 0x7fc280198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.810+0000 7fc285b9d700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc280101c80 0x7fc28019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.810+0000 7fc285b9d700 1 --2- 172.21.15.93:0/2069886618 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2801012d0 0x7fc280198590 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.810+0000 7fc285b9d700 1 -- 172.21.15.93:0/2069886618 >> 172.21.15.93:0/2069886618 conn(0x7fc2800fb640 msgr2=0x7fc280106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.810+0000 7fc285b9d700 1 -- 172.21.15.93:0/2069886618 shutdown_connections 2023-11-16T16:52:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:02.811+0000 7fc285b9d700 1 -- 172.21.15.93:0/2069886618 wait complete. 2023-11-16T16:52:02.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:02 smithi125 ceph-mon[131076]: pgmap v447: 129 pgs: 129 active+clean; 42 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 667 op/s 2023-11-16T16:52:02.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:02 smithi125 ceph-mon[131076]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:52:02.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:02 smithi093 ceph-mon[97863]: pgmap v447: 129 pgs: 129 active+clean; 42 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 667 op/s 2023-11-16T16:52:02.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:02 smithi093 ceph-mon[97863]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:52:02.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:02 smithi159 ceph-mon[131528]: pgmap v447: 129 pgs: 129 active+clean; 42 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 667 op/s 2023-11-16T16:52:02.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:02 smithi159 ceph-mon[131528]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:52:04.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:04 smithi125 ceph-mon[131076]: pgmap v448: 129 pgs: 129 active+clean; 45 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 450 op/s 2023-11-16T16:52:04.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:04 smithi093 ceph-mon[97863]: pgmap v448: 129 pgs: 129 active+clean; 45 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 450 op/s 2023-11-16T16:52:04.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:04 smithi159 ceph-mon[131528]: pgmap v448: 129 pgs: 129 active+clean; 45 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 450 op/s 2023-11-16T16:52:06.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:06 smithi125 ceph-mon[131076]: pgmap v449: 129 pgs: 129 active+clean; 44 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 786 op/s 2023-11-16T16:52:06.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:06 smithi093 ceph-mon[97863]: pgmap v449: 129 pgs: 129 active+clean; 44 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 786 op/s 2023-11-16T16:52:06.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:06 smithi159 ceph-mon[131528]: pgmap v449: 129 pgs: 129 active+clean; 44 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 786 op/s 2023-11-16T16:52:08.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:08 smithi125 ceph-mon[131076]: pgmap v450: 129 pgs: 129 active+clean; 50 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 1.17k op/s 2023-11-16T16:52:08.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:08 smithi093 ceph-mon[97863]: pgmap v450: 129 pgs: 129 active+clean; 50 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 1.17k op/s 2023-11-16T16:52:08.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:08 smithi159 ceph-mon[131528]: pgmap v450: 129 pgs: 129 active+clean; 50 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 1.17k op/s 2023-11-16T16:52:10.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:10 smithi125 ceph-mon[131076]: pgmap v451: 129 pgs: 129 active+clean; 54 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 1.36k op/s 2023-11-16T16:52:16.234 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:12 smithi125 ceph-mon[131076]: pgmap v452: 129 pgs: 129 active+clean; 69 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 1.38k op/s 2023-11-16T16:52:16.235 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:14 smithi125 ceph-mon[131076]: pgmap v453: 129 pgs: 129 active+clean; 78 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 1.36k op/s 2023-11-16T16:52:16.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:10 smithi159 ceph-mon[131528]: pgmap v451: 129 pgs: 129 active+clean; 54 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 1.36k op/s 2023-11-16T16:52:16.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:12 smithi159 ceph-mon[131528]: pgmap v452: 129 pgs: 129 active+clean; 69 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 1.38k op/s 2023-11-16T16:52:16.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:14 smithi159 ceph-mon[131528]: pgmap v453: 129 pgs: 129 active+clean; 78 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 1.36k op/s 2023-11-16T16:52:16.235 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:10 smithi093 ceph-mon[97863]: pgmap v451: 129 pgs: 129 active+clean; 54 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 1.36k op/s 2023-11-16T16:52:16.235 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:12 smithi093 ceph-mon[97863]: pgmap v452: 129 pgs: 129 active+clean; 69 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 1.38k op/s 2023-11-16T16:52:16.235 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:14 smithi093 ceph-mon[97863]: pgmap v453: 129 pgs: 129 active+clean; 78 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 1.36k op/s 2023-11-16T16:52:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.984+0000 7f4b641ff700 1 -- 172.21.15.93:0/3598919070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b5c100240 msgr2=0x7f4b5c100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.984+0000 7f4b641ff700 1 --2- 172.21.15.93:0/3598919070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b5c100240 0x7f4b5c100650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009990 tx=0x7f4b58009ca0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.984+0000 7f4b641ff700 1 -- 172.21.15.93:0/3598919070 shutdown_connections 2023-11-16T16:52:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.984+0000 7f4b641ff700 1 --2- 172.21.15.93:0/3598919070 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b5c101dc0 0x7f4b5c108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.984+0000 7f4b641ff700 1 --2- 172.21.15.93:0/3598919070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b5c100240 0x7f4b5c100650 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.984+0000 7f4b641ff700 1 --2- 172.21.15.93:0/3598919070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b5c101430 0x7f4b5c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.984+0000 7f4b641ff700 1 -- 172.21.15.93:0/3598919070 >> 172.21.15.93:0/3598919070 conn(0x7f4b5c0fb840 msgr2=0x7f4b5c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.984+0000 7f4b641ff700 1 -- 172.21.15.93:0/3598919070 shutdown_connections 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.984+0000 7f4b641ff700 1 -- 172.21.15.93:0/3598919070 wait complete. 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.986+0000 7f4b641ff700 1 Processor -- start 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.986+0000 7f4b641ff700 1 -- start start 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.986+0000 7f4b641ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b5c100240 0x7f4b5c19c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.986+0000 7f4b641ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b5c101430 0x7f4b5c19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.987+0000 7f4b641ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b5c101dc0 0x7f4b5c1a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.987+0000 7f4b641ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c1a4180 con 0x7f4b5c100240 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.987+0000 7f4b641ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c1a42c0 con 0x7f4b5c101dc0 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.987+0000 7f4b641ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c1a4450 con 0x7f4b5c101430 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.987+0000 7f4b6179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b5c101430 0x7f4b5c19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.987+0000 7f4b61f9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b5c100240 0x7f4b5c19c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.987+0000 7f4b6179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b5c101430 0x7f4b5c19cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:56616/0 (socket says 172.21.15.93:56616) 2023-11-16T16:52:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.987+0000 7f4b6179a700 1 -- 172.21.15.93:0/2521969105 learned_addr learned my addr 172.21.15.93:0/2521969105 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.987+0000 7f4b61f9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b5c100240 0x7f4b5c19c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47142/0 (socket says 172.21.15.93:47142) 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.987+0000 7f4b6279c700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b5c101dc0 0x7f4b5c1a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.988+0000 7f4b61f9b700 1 -- 172.21.15.93:0/2521969105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b5c101430 msgr2=0x7f4b5c19cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.988+0000 7f4b61f9b700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b5c101430 0x7f4b5c19cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.988+0000 7f4b61f9b700 1 -- 172.21.15.93:0/2521969105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b5c101dc0 msgr2=0x7f4b5c1a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.988+0000 7f4b61f9b700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b5c101dc0 0x7f4b5c1a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.988+0000 7f4b61f9b700 1 -- 172.21.15.93:0/2521969105 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b580095f0 con 0x7f4b5c100240 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.988+0000 7f4b6179a700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b5c101430 0x7f4b5c19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.988+0000 7f4b61f9b700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b5c100240 0x7f4b5c19c600 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4b580098e0 tx=0x7f4b580057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.989+0000 7f4b52ffd700 1 -- 172.21.15.93:0/2521969105 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5801d070 con 0x7f4b5c100240 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.989+0000 7f4b52ffd700 1 -- 172.21.15.93:0/2521969105 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5800fb80 con 0x7f4b5c100240 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.989+0000 7f4b52ffd700 1 -- 172.21.15.93:0/2521969105 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58017650 con 0x7f4b5c100240 2023-11-16T16:52:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.989+0000 7f4b641ff700 1 -- 172.21.15.93:0/2521969105 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c10bba0 con 0x7f4b5c100240 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.989+0000 7f4b641ff700 1 -- 172.21.15.93:0/2521969105 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5c10ba10 con 0x7f4b5c100240 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.993+0000 7f4b52ffd700 1 -- 172.21.15.93:0/2521969105 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4b58022470 con 0x7f4b5c100240 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.995+0000 7f4b52ffd700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4b480a11d0 0x7f4b480a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.995+0000 7f4b52ffd700 1 -- 172.21.15.93:0/2521969105 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f4b580ce2a0 con 0x7f4b5c100240 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.996+0000 7f4b6179a700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4b480a11d0 0x7f4b480a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.996+0000 7f4b77aceb80 1 Processor -- start 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.997+0000 7f4b77aceb80 1 -- start start 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.997+0000 7f4b6179a700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4b480a11d0 0x7f4b480a3680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009d70 tx=0x7f4b4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.997+0000 7f4b77aceb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x563805910870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.997+0000 7f4b77aceb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805910db0 0x5638059115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.997+0000 7f4b77aceb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 0x563805911f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.997+0000 7f4b77aceb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805816000 con 0x563805911b10 2023-11-16T16:52:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.997+0000 7f4b77aceb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805868840 con 0x563805910db0 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.997+0000 7f4b77aceb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805861cd0 con 0x56380576c6b0 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.997+0000 7f4b61f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x563805910870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.998+0000 7f4b61f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x563805910870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:56626/0 (socket says 172.21.15.93:56626) 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.998+0000 7f4b61f9b700 1 -- 172.21.15.93:0/3816675402 learned_addr learned my addr 172.21.15.93:0/3816675402 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.998+0000 7f4b61f9b700 1 -- 172.21.15.93:0/3816675402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805910db0 msgr2=0x5638059115d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.998+0000 7f4b61f9b700 1 --2- 172.21.15.93:0/3816675402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805910db0 0x5638059115d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.998+0000 7f4b61f9b700 1 -- 172.21.15.93:0/3816675402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 msgr2=0x563805911f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.998+0000 7f4b61f9b700 1 --2- 172.21.15.93:0/3816675402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 0x563805911f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.998+0000 7f4b61f9b700 1 -- 172.21.15.93:0/3816675402 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638056e35d0 con 0x56380576c6b0 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.998+0000 7f4b61f9b700 1 --2- 172.21.15.93:0/3816675402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x563805910870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b58097de0 tx=0x7f4b58000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4a7cdf556e99416 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.999+0000 7f4b47fff700 1 -- 172.21.15.93:0/3816675402 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5809f3a0 con 0x56380576c6b0 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.999+0000 7f4b47fff700 1 -- 172.21.15.93:0/3816675402 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b580cbe90 con 0x56380576c6b0 2023-11-16T16:52:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:15.999+0000 7f4b47fff700 1 -- 172.21.15.93:0/3816675402 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5809a860 con 0x56380576c6b0 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.000+0000 7f4b77aceb80 1 -- 172.21.15.93:0/3816675402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 msgr2=0x563805910870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.000+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/3816675402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x563805910870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b58097de0 tx=0x7f4b58000bc0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.000+0000 7f4b77aceb80 1 -- 172.21.15.93:0/3816675402 shutdown_connections 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.000+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/3816675402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 0x563805911f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.000+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/3816675402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x563805910870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.000+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/3816675402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805910db0 0x5638059115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.000+0000 7f4b77aceb80 1 -- 172.21.15.93:0/3816675402 >> 172.21.15.93:0/3816675402 conn(0x563805905f50 msgr2=0x56380590b100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.000+0000 7f4b77aceb80 1 -- 172.21.15.93:0/3816675402 shutdown_connections 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.000+0000 7f4b77aceb80 1 -- 172.21.15.93:0/3816675402 wait complete. 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.001+0000 7f4b77aceb80 1 -- start start 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.001+0000 7f4b77aceb80 1 Processor -- start 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.002+0000 7f4b77aceb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380576c6b0 0x563805919970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.002+0000 7f4b77aceb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563805910db0 0x563805919eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.002+0000 7f4b77aceb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 0x563805920ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.002+0000 7f4b77aceb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805803880 con 0x563805911b10 2023-11-16T16:52:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.002+0000 7f4b77aceb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805803520 con 0x56380576c6b0 2023-11-16T16:52:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.002+0000 7f4b77aceb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638056e58c0 con 0x563805910db0 2023-11-16T16:52:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.002+0000 7f4b77aceb80 10 client.0 _ll_register_callbacks cb 0x563805821720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:52:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.002+0000 7f4b6179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563805910db0 0x563805919eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.002+0000 7f4b6179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563805910db0 0x563805919eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:56628/0 (socket says 172.21.15.93:56628) 2023-11-16T16:52:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.002+0000 7f4b6179a700 1 -- 172.21.15.93:0/1139045563 learned_addr learned my addr 172.21.15.93:0/1139045563 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.003+0000 7f4b6279c700 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 0x563805920ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.003+0000 7f4b6179a700 1 -- 172.21.15.93:0/1139045563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380576c6b0 msgr2=0x563805919970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:52:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.003+0000 7f4b6179a700 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380576c6b0 0x563805919970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.003+0000 7f4b6179a700 1 -- 172.21.15.93:0/1139045563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 msgr2=0x563805920ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.003+0000 7f4b6179a700 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 0x563805920ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.003+0000 7f4b6179a700 1 -- 172.21.15.93:0/1139045563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5c10ba10 con 0x563805910db0 2023-11-16T16:52:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.003+0000 7f4b6279c700 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 0x563805920ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:52:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.003+0000 7f4b6179a700 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563805910db0 0x563805919eb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009fd0 tx=0x7f4b4c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=274aed4fb318f315 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.003+0000 7f4b457fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-11-16T16:52:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.004+0000 7f4b457fa700 1 -- 172.21.15.93:0/1139045563 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00f9e0 con 0x563805910db0 2023-11-16T16:52:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.004+0000 7f4b457fa700 1 -- 172.21.15.93:0/1139045563 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f4b4c018070 con 0x563805910db0 2023-11-16T16:52:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.004+0000 7f4b77aceb80 1 -- 172.21.15.93:0/1139045563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563805903710 con 0x563805910db0 2023-11-16T16:52:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.004+0000 7f4b457fa700 1 -- 172.21.15.93:0/1139045563 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b4c00fb40 con 0x563805910db0 2023-11-16T16:52:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.004+0000 7f4b77aceb80 10 client.25606 fetch_fsmap learned FSMap version 26 2023-11-16T16:52:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.004+0000 7f4b77aceb80 1 -- 172.21.15.93:0/1139045563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563805828130 con 0x563805910db0 2023-11-16T16:52:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.004+0000 7f4b457fa700 1 -- 172.21.15.93:0/1139045563 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563805903710 con 0x563805910db0 2023-11-16T16:52:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.004+0000 7f4b457fa700 1 -- 172.21.15.93:0/1139045563 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 26) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4b4c025020 con 0x563805910db0 2023-11-16T16:52:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.004+0000 7f4b77aceb80 10 client.25606 fetch_fsmap finished waiting for FSMap version 26 2023-11-16T16:52:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.005+0000 7f4b77aceb80 10 client.25606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:52:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.005+0000 7f4b77aceb80 20 client.25606 populate_metadata read hostname 'smithi093' 2023-11-16T16:52:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.005+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x563805925780 0x563805927b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.005+0000 7f4b77aceb80 4 client.25606 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:52:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.005+0000 7f4b77aceb80 1 -- 172.21.15.93:0/1139045563 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563805824040 con 0x563805925780 2023-11-16T16:52:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.005+0000 7f4b61f9b700 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x563805925780 0x563805927b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.006+0000 7f4b61f9b700 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x563805925780 0x563805927b90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf6268df4e201452 server_cookie=5bfe2d0ca88811b8 in_seq=0 out_seq=0 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.006+0000 7f4b457fa700 10 client.25606 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.006+0000 7f4b457fa700 1 -- 172.21.15.93:0/1139045563 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563805824040 con 0x563805925780 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.006+0000 7f4b457fa700 10 client.25606 handle_command_reply: tid=0 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.006+0000 7f4b77aceb80 1 client.25606 shutdown 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.007+0000 7f4b77aceb80 1 -- 172.21.15.93:0/1139045563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563805910db0 msgr2=0x563805919eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.007+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563805910db0 0x563805919eb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009fd0 tx=0x7f4b4c0047c0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.007+0000 7f4b77aceb80 1 -- 172.21.15.93:0/1139045563 shutdown_connections 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.007+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x563805925780 0x563805927b90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.007+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 0x563805920ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.007+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56380576c6b0 0x563805919970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.007+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/1139045563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563805910db0 0x563805919eb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.007+0000 7f4b457fa700 0 client.25606 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.007+0000 7f4b77aceb80 1 -- 172.21.15.93:0/1139045563 >> 172.21.15.93:0/1139045563 conn(0x563805905f50 msgr2=0x56380590a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.007+0000 7f4b77aceb80 1 -- 172.21.15.93:0/1139045563 shutdown_connections 2023-11-16T16:52:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.007+0000 7f4b77aceb80 1 -- 172.21.15.93:0/1139045563 wait complete. 2023-11-16T16:52:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.007+0000 7f4b77aceb80 20 client.25606 trim_cache size 0 max 16384 2023-11-16T16:52:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.029+0000 7f4b77aceb80 1 Processor -- start 2023-11-16T16:52:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.029+0000 7f4b77aceb80 1 -- start start 2023-11-16T16:52:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.029+0000 7f4b77aceb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x563805919720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.029+0000 7f4b77aceb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805910db0 0x563805920eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.029+0000 7f4b77aceb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 0x563805919c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.029+0000 7f4b77aceb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805919540 con 0x563805911b10 2023-11-16T16:52:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.029+0000 7f4b77aceb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805905950 con 0x563805910db0 2023-11-16T16:52:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.029+0000 7f4b77aceb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380591a8a0 con 0x56380576c6b0 2023-11-16T16:52:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.030+0000 7f4b6179a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805910db0 0x563805920eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.030+0000 7f4b61f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x563805919720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.030+0000 7f4b6279c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 0x563805919c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.030+0000 7f4b6179a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805910db0 0x563805920eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:54958/0 (socket says 172.21.15.93:54958) 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.030+0000 7f4b6179a700 1 -- 172.21.15.93:0/197152075 learned_addr learned my addr 172.21.15.93:0/197152075 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.030+0000 7f4b61f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x563805919720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:56640/0 (socket says 172.21.15.93:56640) 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.030+0000 7f4b6179a700 1 -- 172.21.15.93:0/197152075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 msgr2=0x563805919720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.030+0000 7f4b6179a700 1 --2- 172.21.15.93:0/197152075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x563805919720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.030+0000 7f4b6179a700 1 -- 172.21.15.93:0/197152075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 msgr2=0x563805919c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.030+0000 7f4b6179a700 1 --2- 172.21.15.93:0/197152075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 0x563805919c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.030+0000 7f4b6179a700 1 -- 172.21.15.93:0/197152075 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563805827e80 con 0x563805910db0 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.030+0000 7f4b6179a700 1 --2- 172.21.15.93:0/197152075 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805910db0 0x563805920eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c000c00 tx=0x7f4b4c0046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e71d7272e0176bcc server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.031+0000 7f4b50ff9700 1 -- 172.21.15.93:0/197152075 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00e550 con 0x563805910db0 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.031+0000 7f4b50ff9700 1 -- 172.21.15.93:0/197152075 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b4c016070 con 0x563805910db0 2023-11-16T16:52:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.031+0000 7f4b50ff9700 1 -- 172.21.15.93:0/197152075 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c01dc00 con 0x563805910db0 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.032+0000 7f4b77aceb80 1 -- 172.21.15.93:0/197152075 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805910db0 msgr2=0x563805920eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.032+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/197152075 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805910db0 0x563805920eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c000c00 tx=0x7f4b4c0046d0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.032+0000 7f4b77aceb80 1 -- 172.21.15.93:0/197152075 shutdown_connections 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.032+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/197152075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805911b10 0x563805919c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.032+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/197152075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x563805919720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.032+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/197152075 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805910db0 0x563805920eb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.032+0000 7f4b77aceb80 1 -- 172.21.15.93:0/197152075 >> 172.21.15.93:0/197152075 conn(0x563805905f50 msgr2=0x56380592c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.032+0000 7f4b77aceb80 1 -- 172.21.15.93:0/197152075 shutdown_connections 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.032+0000 7f4b77aceb80 1 -- 172.21.15.93:0/197152075 wait complete. 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.032+0000 7f4b77aceb80 1 -- start start 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.032+0000 7f4b77aceb80 1 Processor -- start 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.033+0000 7f4b77aceb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x56380591d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.033+0000 7f4b77aceb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805910db0 0x56380591d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.033+0000 7f4b77aceb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805911b10 0x5638059202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.033+0000 7f4b77aceb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805920810 con 0x563805910db0 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.033+0000 7f4b77aceb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805920980 con 0x563805911b10 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.033+0000 7f4b77aceb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805920b40 con 0x56380576c6b0 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.033+0000 7f4b77aceb80 10 client.0 _ll_register_callbacks cb 0x7f4b4c0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.033+0000 7f4b61f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x56380591d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.033+0000 7f4b61f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x56380591d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:56646/0 (socket says 172.21.15.93:56646) 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.033+0000 7f4b61f9b700 1 -- 172.21.15.93:0/2901994426 learned_addr learned my addr 172.21.15.93:0/2901994426 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.033+0000 7f4b6279c700 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805911b10 0x5638059202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.033+0000 7f4b6179a700 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805910db0 0x56380591d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b61f9b700 1 -- 172.21.15.93:0/2901994426 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805911b10 msgr2=0x5638059202d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b61f9b700 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805911b10 0x5638059202d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b61f9b700 1 -- 172.21.15.93:0/2901994426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805910db0 msgr2=0x56380591d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b61f9b700 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805910db0 0x56380591d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b61f9b700 1 -- 172.21.15.93:0/2901994426 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563805828130 con 0x56380576c6b0 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b61f9b700 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x56380591d040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4b5809b1b0 tx=0x7f4b58005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c9072848c895f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b457fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b457fa700 1 -- 172.21.15.93:0/2901994426 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b580ce0e0 con 0x56380576c6b0 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b457fa700 1 -- 172.21.15.93:0/2901994426 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f4b5809f3a0 con 0x56380576c6b0 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b77aceb80 1 -- 172.21.15.93:0/2901994426 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b580cbe90 con 0x56380576c6b0 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b457fa700 1 -- 172.21.15.93:0/2901994426 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b5809f150 con 0x56380576c6b0 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b77aceb80 10 client.25612 fetch_fsmap learned FSMap version 26 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.034+0000 7f4b77aceb80 1 -- 172.21.15.93:0/2901994426 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563805678c00 con 0x56380576c6b0 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.035+0000 7f4b457fa700 1 -- 172.21.15.93:0/2901994426 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b580cbe90 con 0x56380576c6b0 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.035+0000 7f4b457fa700 1 -- 172.21.15.93:0/2901994426 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 26) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4b5809a8f0 con 0x56380576c6b0 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.035+0000 7f4b77aceb80 10 client.25612 fetch_fsmap finished waiting for FSMap version 26 2023-11-16T16:52:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.035+0000 7f4b77aceb80 10 client.25612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:52:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.035+0000 7f4b77aceb80 20 client.25612 populate_metadata read hostname 'smithi093' 2023-11-16T16:52:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.035+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56380597a2b0 0x56380597c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.035+0000 7f4b77aceb80 4 client.25612 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:52:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.035+0000 7f4b77aceb80 1 -- 172.21.15.93:0/2901994426 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563805830600 con 0x56380597a2b0 2023-11-16T16:52:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.036+0000 7f4b6179a700 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56380597a2b0 0x56380597c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.036+0000 7f4b6179a700 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56380597a2b0 0x56380597c6c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734861c0dcf3c349 server_cookie=5bf5e91deb612199 in_seq=0 out_seq=0 2023-11-16T16:52:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:16.036+0000 7f4b457fa700 10 client.25612 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:16.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:16 smithi125 ceph-mon[131076]: pgmap v454: 129 pgs: 129 active+clean; 80 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 1.15k op/s 2023-11-16T16:52:16.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:16 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:52:16.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:16 smithi093 ceph-mon[97863]: pgmap v454: 129 pgs: 129 active+clean; 80 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 1.15k op/s 2023-11-16T16:52:16.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:16 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:52:16.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:16 smithi159 ceph-mon[131528]: pgmap v454: 129 pgs: 129 active+clean; 80 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 1.15k op/s 2023-11-16T16:52:16.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:16 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:52:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:52:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:52:17.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:17 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:52:17.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:52:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:52:17.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:17 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:52:18.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:52:18.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:52:18.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:17 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:52:18.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:18 smithi093 ceph-mon[97863]: pgmap v455: 129 pgs: 129 active+clean; 82 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 682 op/s 2023-11-16T16:52:18.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:18 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:52:18.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:18 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:52:18.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:18 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:52:18.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:18 smithi093 ceph-mon[97863]: Service mds.all has no related daemon already placed. 2023-11-16T16:52:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:18 smithi159 ceph-mon[131528]: pgmap v455: 129 pgs: 129 active+clean; 82 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 682 op/s 2023-11-16T16:52:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:18 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:52:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:18 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:52:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:18 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:52:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:18 smithi159 ceph-mon[131528]: Service mds.all has no related daemon already placed. 2023-11-16T16:52:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.028+0000 7f4b457fa700 1 -- 172.21.15.93:0/2901994426 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563805830600 con 0x56380597a2b0 2023-11-16T16:52:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.028+0000 7f4b457fa700 10 client.25612 handle_command_reply: tid=0 2023-11-16T16:52:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.028+0000 7f4b77aceb80 1 client.25612 shutdown 2023-11-16T16:52:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.029+0000 7f4b77aceb80 1 -- 172.21.15.93:0/2901994426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 msgr2=0x56380591d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.029+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x56380591d040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4b5809b1b0 tx=0x7f4b58005fd0 comp rx=0 tx=0).stop 2023-11-16T16:52:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.029+0000 7f4b77aceb80 1 -- 172.21.15.93:0/2901994426 shutdown_connections 2023-11-16T16:52:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.029+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56380597a2b0 0x56380597c6c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.029+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563805911b10 0x5638059202d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.029+0000 7f4b457fa700 0 client.25612 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.029+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380576c6b0 0x56380591d040 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.029+0000 7f4b77aceb80 1 --2- 172.21.15.93:0/2901994426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563805910db0 0x56380591d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.029+0000 7f4b77aceb80 1 -- 172.21.15.93:0/2901994426 >> 172.21.15.93:0/2901994426 conn(0x563805905f50 msgr2=0x563805919720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.030+0000 7f4b77aceb80 1 -- 172.21.15.93:0/2901994426 shutdown_connections 2023-11-16T16:52:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.030+0000 7f4b77aceb80 1 -- 172.21.15.93:0/2901994426 wait complete. 2023-11-16T16:52:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.030+0000 7f4b77aceb80 20 client.25612 trim_cache size 0 max 16384 2023-11-16T16:52:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:52:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:52:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:52:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:52:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.031+0000 7f4b641ff700 1 -- 172.21.15.93:0/2521969105 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4b480a11d0 msgr2=0x7f4b480a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.031+0000 7f4b641ff700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4b480a11d0 0x7f4b480a3680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009d70 tx=0x7f4b4c009260 comp rx=0 tx=0).stop 2023-11-16T16:52:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.031+0000 7f4b641ff700 1 -- 172.21.15.93:0/2521969105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b5c100240 msgr2=0x7f4b5c19c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.031+0000 7f4b641ff700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b5c100240 0x7f4b5c19c600 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4b580098e0 tx=0x7f4b580057b0 comp rx=0 tx=0).stop 2023-11-16T16:52:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.031+0000 7f4b641ff700 1 -- 172.21.15.93:0/2521969105 shutdown_connections 2023-11-16T16:52:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.031+0000 7f4b641ff700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f4b480a11d0 0x7f4b480a3680 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.031+0000 7f4b641ff700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b5c100240 0x7f4b5c19c600 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.031+0000 7f4b641ff700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b5c101dc0 0x7f4b5c1a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.031+0000 7f4b641ff700 1 --2- 172.21.15.93:0/2521969105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b5c101430 0x7f4b5c19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.031+0000 7f4b641ff700 1 -- 172.21.15.93:0/2521969105 >> 172.21.15.93:0/2521969105 conn(0x7f4b5c0fb840 msgr2=0x7f4b5c106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.032+0000 7f4b641ff700 1 -- 172.21.15.93:0/2521969105 shutdown_connections 2023-11-16T16:52:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:19.032+0000 7f4b641ff700 1 -- 172.21.15.93:0/2521969105 wait complete. 2023-11-16T16:52:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:18 smithi125 ceph-mon[131076]: pgmap v455: 129 pgs: 129 active+clean; 82 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 682 op/s 2023-11-16T16:52:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:18 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:52:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:18 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:52:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:18 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:52:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:18 smithi125 ceph-mon[131076]: Service mds.all has no related daemon already placed. 2023-11-16T16:52:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:20 smithi093 ceph-mon[97863]: pgmap v456: 129 pgs: 129 active+clean; 88 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 346 op/s 2023-11-16T16:52:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:20 smithi159 ceph-mon[131528]: pgmap v456: 129 pgs: 129 active+clean; 88 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 346 op/s 2023-11-16T16:52:21.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:20 smithi125 ceph-mon[131076]: pgmap v456: 129 pgs: 129 active+clean; 88 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 346 op/s 2023-11-16T16:52:22.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:22 smithi093 ceph-mon[97863]: pgmap v457: 129 pgs: 129 active+clean; 91 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 318 op/s 2023-11-16T16:52:22.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:22 smithi159 ceph-mon[131528]: pgmap v457: 129 pgs: 129 active+clean; 91 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 318 op/s 2023-11-16T16:52:23.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:22 smithi125 ceph-mon[131076]: pgmap v457: 129 pgs: 129 active+clean; 91 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 318 op/s 2023-11-16T16:52:24.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:24 smithi093 ceph-mon[97863]: pgmap v458: 129 pgs: 129 active+clean; 103 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 429 op/s 2023-11-16T16:52:24.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:24 smithi159 ceph-mon[131528]: pgmap v458: 129 pgs: 129 active+clean; 103 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 429 op/s 2023-11-16T16:52:25.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:24 smithi125 ceph-mon[131076]: pgmap v458: 129 pgs: 129 active+clean; 103 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 429 op/s 2023-11-16T16:52:26.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:26 smithi093 ceph-mon[97863]: pgmap v459: 129 pgs: 129 active+clean; 107 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 414 op/s 2023-11-16T16:52:26.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:26 smithi159 ceph-mon[131528]: pgmap v459: 129 pgs: 129 active+clean; 107 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 414 op/s 2023-11-16T16:52:27.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:26 smithi125 ceph-mon[131076]: pgmap v459: 129 pgs: 129 active+clean; 107 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 414 op/s 2023-11-16T16:52:28.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:28 smithi093 ceph-mon[97863]: pgmap v460: 129 pgs: 129 active+clean; 111 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 393 op/s 2023-11-16T16:52:28.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:28 smithi159 ceph-mon[131528]: pgmap v460: 129 pgs: 129 active+clean; 111 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 393 op/s 2023-11-16T16:52:29.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:28 smithi125 ceph-mon[131076]: pgmap v460: 129 pgs: 129 active+clean; 111 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 393 op/s 2023-11-16T16:52:30.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:30 smithi093 ceph-mon[97863]: pgmap v461: 129 pgs: 129 active+clean; 121 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 395 op/s 2023-11-16T16:52:30.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:30 smithi159 ceph-mon[131528]: pgmap v461: 129 pgs: 129 active+clean; 121 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 395 op/s 2023-11-16T16:52:31.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:30 smithi125 ceph-mon[131076]: pgmap v461: 129 pgs: 129 active+clean; 121 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 395 op/s 2023-11-16T16:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.184+0000 7f05c83ff700 1 -- 172.21.15.93:0/105974224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05c01002c0 msgr2=0x7f05c01006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.184+0000 7f05c83ff700 1 --2- 172.21.15.93:0/105974224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05c01002c0 0x7f05c01006d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0098f0 tx=0x7f05bc009c00 comp rx=0 tx=0).stop 2023-11-16T16:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.184+0000 7f05c83ff700 1 -- 172.21.15.93:0/105974224 shutdown_connections 2023-11-16T16:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.184+0000 7f05c83ff700 1 --2- 172.21.15.93:0/105974224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05c01002c0 0x7f05c01006d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.184+0000 7f05c83ff700 1 --2- 172.21.15.93:0/105974224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c0101e40 0x7f05c0108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.184+0000 7f05c83ff700 1 --2- 172.21.15.93:0/105974224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05c01014b0 0x7f05c0101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.184+0000 7f05c83ff700 1 -- 172.21.15.93:0/105974224 >> 172.21.15.93:0/105974224 conn(0x7f05c00fb8e0 msgr2=0x7f05c00fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.184+0000 7f05c83ff700 1 -- 172.21.15.93:0/105974224 shutdown_connections 2023-11-16T16:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.184+0000 7f05c83ff700 1 -- 172.21.15.93:0/105974224 wait complete. 2023-11-16T16:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.186+0000 7f05c83ff700 1 Processor -- start 2023-11-16T16:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.186+0000 7f05c83ff700 1 -- start start 2023-11-16T16:52:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.186+0000 7f05c83ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05c01002c0 0x7f05c00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c83ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05c01014b0 0x7f05c0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c83ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c0101e40 0x7f05c0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c83ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c0073bc0 con 0x7f05c01014b0 2023-11-16T16:52:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c83ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c0073d30 con 0x7f05c01002c0 2023-11-16T16:52:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c83ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c01a0150 con 0x7f05c0101e40 2023-11-16T16:52:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c619b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05c01002c0 0x7f05c00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c619b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05c01002c0 0x7f05c00748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:59404/0 (socket says 172.21.15.93:59404) 2023-11-16T16:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c619b700 1 -- 172.21.15.93:0/2359305165 learned_addr learned my addr 172.21.15.93:0/2359305165 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c599a700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05c01014b0 0x7f05c0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c699c700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c0101e40 0x7f05c0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c619b700 1 -- 172.21.15.93:0/2359305165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c0101e40 msgr2=0x7f05c0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c619b700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c0101e40 0x7f05c0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c619b700 1 -- 172.21.15.93:0/2359305165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05c01014b0 msgr2=0x7f05c0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c619b700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05c01014b0 0x7f05c0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c619b700 1 -- 172.21.15.93:0/2359305165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b00095f0 con 0x7f05c01002c0 2023-11-16T16:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c599a700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05c01014b0 0x7f05c0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c699c700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c0101e40 0x7f05c0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.187+0000 7f05c619b700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05c01002c0 0x7f05c00748a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05bc005e90 tx=0x7f05bc00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.188+0000 7f05b77fe700 1 -- 172.21.15.93:0/2359305165 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc01c070 con 0x7f05c01002c0 2023-11-16T16:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.188+0000 7f05c83ff700 1 -- 172.21.15.93:0/2359305165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05bc009520 con 0x7f05c01002c0 2023-11-16T16:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.188+0000 7f05b77fe700 1 -- 172.21.15.93:0/2359305165 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05bc00f7f0 con 0x7f05c01002c0 2023-11-16T16:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.188+0000 7f05b77fe700 1 -- 172.21.15.93:0/2359305165 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc003ba0 con 0x7f05c01002c0 2023-11-16T16:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.188+0000 7f05c83ff700 1 -- 172.21.15.93:0/2359305165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c010bc00 con 0x7f05c01002c0 2023-11-16T16:52:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.192+0000 7f05b77fe700 1 -- 172.21.15.93:0/2359305165 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f05bc020030 con 0x7f05c01002c0 2023-11-16T16:52:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.192+0000 7f05dbca5b80 1 Processor -- start 2023-11-16T16:52:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.192+0000 7f05dbca5b80 1 -- start start 2023-11-16T16:52:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.193+0000 7f05dbca5b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c2846b0 0x56211c428870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.193+0000 7f05dbca5b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.193+0000 7f05dbca5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c429b10 0x56211c429f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.193+0000 7f05dbca5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211c32e000 con 0x56211c429b10 2023-11-16T16:52:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.193+0000 7f05dbca5b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211c380840 con 0x56211c2846b0 2023-11-16T16:52:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.193+0000 7f05dbca5b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211c379cd0 con 0x56211c428db0 2023-11-16T16:52:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.193+0000 7f05c699c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c2846b0 0x56211c428870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.193+0000 7f05c699c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c2846b0 0x56211c428870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:59410/0 (socket says 172.21.15.93:59410) 2023-11-16T16:52:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.193+0000 7f05c699c700 1 -- 172.21.15.93:0/2440861186 learned_addr learned my addr 172.21.15.93:0/2440861186 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.193+0000 7f05c599a700 1 --2- 172.21.15.93:0/2440861186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c429b10 0x56211c429f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.193+0000 7f05c619b700 1 --2- 172.21.15.93:0/2440861186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.194+0000 7f05c599a700 1 -- 172.21.15.93:0/2440861186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 msgr2=0x56211c4295d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.194+0000 7f05c599a700 1 --2- 172.21.15.93:0/2440861186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4295d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.194+0000 7f05c599a700 1 -- 172.21.15.93:0/2440861186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c2846b0 msgr2=0x56211c428870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.194+0000 7f05c599a700 1 --2- 172.21.15.93:0/2440861186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c2846b0 0x56211c428870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.194+0000 7f05c599a700 1 -- 172.21.15.93:0/2440861186 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56211c1fb5d0 con 0x56211c429b10 2023-11-16T16:52:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.194+0000 7f05b77fe700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f05ac0a11d0 0x7f05ac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.194+0000 7f05c599a700 1 --2- 172.21.15.93:0/2440861186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c429b10 0x56211c429f80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009f30 tx=0x7f05b00116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c2d5382b6ef5af server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.194+0000 7f05b77fe700 1 -- 172.21.15.93:0/2359305165 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f05bc0cdd40 con 0x7f05c01002c0 2023-11-16T16:52:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.194+0000 7f05c699c700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f05ac0a11d0 0x7f05ac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.195+0000 7f05b4bfe700 1 -- 172.21.15.93:0/2440861186 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0011ee0 con 0x56211c429b10 2023-11-16T16:52:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.195+0000 7f05b4bfe700 1 -- 172.21.15.93:0/2440861186 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05b0004cf0 con 0x56211c429b10 2023-11-16T16:52:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.195+0000 7f05b4bfe700 1 -- 172.21.15.93:0/2440861186 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b00126b0 con 0x56211c429b10 2023-11-16T16:52:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.196+0000 7f05dbca5b80 1 -- 172.21.15.93:0/2440861186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c429b10 msgr2=0x56211c429f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.196+0000 7f05c699c700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f05ac0a11d0 0x7f05ac0a3680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009fd0 tx=0x7f05b800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.196+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/2440861186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c429b10 0x56211c429f80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009f30 tx=0x7f05b00116c0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.196+0000 7f05dbca5b80 1 -- 172.21.15.93:0/2440861186 shutdown_connections 2023-11-16T16:52:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.196+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/2440861186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c429b10 0x56211c429f80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.196+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/2440861186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c2846b0 0x56211c428870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.196+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/2440861186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4295d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.196+0000 7f05dbca5b80 1 -- 172.21.15.93:0/2440861186 >> 172.21.15.93:0/2440861186 conn(0x56211c41df50 msgr2=0x56211c423100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.196+0000 7f05dbca5b80 1 -- 172.21.15.93:0/2440861186 shutdown_connections 2023-11-16T16:52:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.196+0000 7f05dbca5b80 1 -- 172.21.15.93:0/2440861186 wait complete. 2023-11-16T16:52:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.196+0000 7f05dbca5b80 1 -- start start 2023-11-16T16:52:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.197+0000 7f05dbca5b80 1 Processor -- start 2023-11-16T16:52:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.197+0000 7f05dbca5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c431c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.197+0000 7f05dbca5b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4321b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.197+0000 7f05dbca5b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c429b10 0x56211c4391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.197+0000 7f05dbca5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211c31b880 con 0x56211c2846b0 2023-11-16T16:52:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.197+0000 7f05dbca5b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211c31b520 con 0x56211c429b10 2023-11-16T16:52:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.197+0000 7f05dbca5b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211c1fd8c0 con 0x56211c428db0 2023-11-16T16:52:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.197+0000 7f05dbca5b80 10 client.0 _ll_register_callbacks cb 0x56211c339720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:52:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.197+0000 7f05c599a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4321b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05c599a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4321b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:60452/0 (socket says 172.21.15.93:60452) 2023-11-16T16:52:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05c599a700 1 -- 172.21.15.93:0/1270545118 learned_addr learned my addr 172.21.15.93:0/1270545118 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05c619b700 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c431c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05c699c700 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c429b10 0x56211c4391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05c599a700 1 -- 172.21.15.93:0/1270545118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c429b10 msgr2=0x56211c4391f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05c599a700 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c429b10 0x56211c4391f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05c599a700 1 -- 172.21.15.93:0/1270545118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 msgr2=0x56211c431c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05c599a700 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c431c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05c599a700 1 -- 172.21.15.93:0/1270545118 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b0009b90 con 0x56211c428db0 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05c699c700 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c429b10 0x56211c4391f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05c599a700 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4321b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05b0012010 tx=0x7f05b0005430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d329edcb7ee9a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.198+0000 7f05a9ffb700 1 -- 172.21.15.93:0/1270545118 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0004750 con 0x56211c428db0 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05a9ffb700 1 -- 172.21.15.93:0/1270545118 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f05b00048b0 con 0x56211c428db0 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05dbca5b80 1 -- 172.21.15.93:0/1270545118 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56211c41b710 con 0x56211c428db0 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05a9ffb700 1 -- 172.21.15.93:0/1270545118 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05b0004a70 con 0x56211c428db0 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05dbca5b80 10 client.25624 fetch_fsmap learned FSMap version 26 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05dbca5b80 1 -- 172.21.15.93:0/1270545118 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56211c340130 con 0x56211c428db0 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05a9ffb700 1 -- 172.21.15.93:0/1270545118 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56211c41b710 con 0x56211c428db0 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05a9ffb700 1 -- 172.21.15.93:0/1270545118 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 26) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f05b0003c60 con 0x56211c428db0 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05dbca5b80 10 client.25624 fetch_fsmap finished waiting for FSMap version 26 2023-11-16T16:52:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05dbca5b80 10 client.25624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:52:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05dbca5b80 20 client.25624 populate_metadata read hostname 'smithi093' 2023-11-16T16:52:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56211c43da00 0x56211c43fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05dbca5b80 4 client.25624 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:52:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.199+0000 7f05dbca5b80 1 -- 172.21.15.93:0/1270545118 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56211c33c040 con 0x56211c43da00 2023-11-16T16:52:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.200+0000 7f05c619b700 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56211c43da00 0x56211c43fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.200+0000 7f05c619b700 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56211c43da00 0x56211c43fe10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8abca65123a5e044 server_cookie=8ca49e0ee479b925 in_seq=0 out_seq=0 2023-11-16T16:52:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.200+0000 7f05a9ffb700 10 client.25624 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05a9ffb700 1 -- 172.21.15.93:0/1270545118 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56211c33c040 con 0x56211c43da00 2023-11-16T16:52:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05a9ffb700 10 client.25624 handle_command_reply: tid=0 2023-11-16T16:52:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05dbca5b80 1 client.25624 shutdown 2023-11-16T16:52:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05dbca5b80 1 -- 172.21.15.93:0/1270545118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 msgr2=0x56211c4321b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4321b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05b0012010 tx=0x7f05b0005430 comp rx=0 tx=0).stop 2023-11-16T16:52:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05dbca5b80 1 -- 172.21.15.93:0/1270545118 shutdown_connections 2023-11-16T16:52:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56211c43da00 0x56211c43fe10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c431c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c429b10 0x56211c4391f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05a9ffb700 0 client.25624 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/1270545118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4321b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05dbca5b80 1 -- 172.21.15.93:0/1270545118 >> 172.21.15.93:0/1270545118 conn(0x56211c41df50 msgr2=0x56211c4227e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05dbca5b80 1 -- 172.21.15.93:0/1270545118 shutdown_connections 2023-11-16T16:52:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.201+0000 7f05dbca5b80 1 -- 172.21.15.93:0/1270545118 wait complete. 2023-11-16T16:52:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.202+0000 7f05dbca5b80 20 client.25624 trim_cache size 0 max 16384 2023-11-16T16:52:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05dbca5b80 1 Processor -- start 2023-11-16T16:52:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05dbca5b80 1 -- start start 2023-11-16T16:52:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05dbca5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c429ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05dbca5b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4391b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05dbca5b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c429b10 0x56211c42c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05dbca5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211c4328f0 con 0x56211c2846b0 2023-11-16T16:52:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05dbca5b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211c42fb60 con 0x56211c429b10 2023-11-16T16:52:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05dbca5b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211c42d450 con 0x56211c428db0 2023-11-16T16:52:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05c619b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c429ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05c599a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05c599a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4391b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:60462/0 (socket says 172.21.15.93:60462) 2023-11-16T16:52:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05c599a700 1 -- 172.21.15.93:0/3637114232 learned_addr learned my addr 172.21.15.93:0/3637114232 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.219+0000 7f05c699c700 1 --2- 172.21.15.93:0/3637114232 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c429b10 0x56211c42c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.220+0000 7f05c599a700 1 -- 172.21.15.93:0/3637114232 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c429b10 msgr2=0x56211c42c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.220+0000 7f05c599a700 1 --2- 172.21.15.93:0/3637114232 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c429b10 0x56211c42c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.220+0000 7f05c599a700 1 -- 172.21.15.93:0/3637114232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 msgr2=0x56211c429ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.220+0000 7f05c599a700 1 --2- 172.21.15.93:0/3637114232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c429ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.220+0000 7f05c599a700 1 -- 172.21.15.93:0/3637114232 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56211c33fe80 con 0x56211c428db0 2023-11-16T16:52:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.220+0000 7f05c619b700 1 --2- 172.21.15.93:0/3637114232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c429ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:52:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.220+0000 7f05c599a700 1 --2- 172.21.15.93:0/3637114232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4391b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009f30 tx=0x7f05b0005390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba6faa65887bfe44 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.220+0000 7f05b53ff700 1 -- 172.21.15.93:0/3637114232 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0009310 con 0x56211c428db0 2023-11-16T16:52:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.220+0000 7f05b53ff700 1 -- 172.21.15.93:0/3637114232 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05b0012790 con 0x56211c428db0 2023-11-16T16:52:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.220+0000 7f05b53ff700 1 -- 172.21.15.93:0/3637114232 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0004220 con 0x56211c428db0 2023-11-16T16:52:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.221+0000 7f05dbca5b80 1 -- 172.21.15.93:0/3637114232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 msgr2=0x56211c4391b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.221+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/3637114232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4391b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009f30 tx=0x7f05b0005390 comp rx=0 tx=0).stop 2023-11-16T16:52:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.221+0000 7f05dbca5b80 1 -- 172.21.15.93:0/3637114232 shutdown_connections 2023-11-16T16:52:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.221+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/3637114232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c429ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.221+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/3637114232 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c429b10 0x56211c42c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.221+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/3637114232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c428db0 0x56211c4391b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.221+0000 7f05dbca5b80 1 -- 172.21.15.93:0/3637114232 >> 172.21.15.93:0/3637114232 conn(0x56211c41df50 msgr2=0x56211c444770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.221+0000 7f05dbca5b80 1 -- 172.21.15.93:0/3637114232 shutdown_connections 2023-11-16T16:52:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.221+0000 7f05dbca5b80 1 -- 172.21.15.93:0/3637114232 wait complete. 2023-11-16T16:52:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.222+0000 7f05dbca5b80 1 -- start start 2023-11-16T16:52:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.222+0000 7f05dbca5b80 1 Processor -- start 2023-11-16T16:52:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.222+0000 7f05dbca5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c435250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.222+0000 7f05dbca5b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c428db0 0x56211c435790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.222+0000 7f05dbca5b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c429b10 0x56211c4387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.222+0000 7f05dbca5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211c438cf0 con 0x56211c2846b0 2023-11-16T16:52:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.222+0000 7f05dbca5b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211c438e60 con 0x56211c428db0 2023-11-16T16:52:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.222+0000 7f05dbca5b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211c439020 con 0x56211c429b10 2023-11-16T16:52:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.222+0000 7f05dbca5b80 10 client.0 _ll_register_callbacks cb 0x7f05b8009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:52:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c599a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c428db0 0x56211c435790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c599a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c428db0 0x56211c435790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:59436/0 (socket says 172.21.15.93:59436) 2023-11-16T16:52:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c599a700 1 -- 172.21.15.93:0/87651009 learned_addr learned my addr 172.21.15.93:0/87651009 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c699c700 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c429b10 0x56211c4387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c619b700 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c435250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c599a700 1 -- 172.21.15.93:0/87651009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c429b10 msgr2=0x56211c4387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c599a700 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c429b10 0x56211c4387b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c599a700 1 -- 172.21.15.93:0/87651009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 msgr2=0x56211c435250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c599a700 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c435250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c599a700 1 -- 172.21.15.93:0/87651009 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc0d1040 con 0x56211c428db0 2023-11-16T16:52:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c619b700 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c435250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:52:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c699c700 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c429b10 0x56211c4387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:52:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05c599a700 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c428db0 0x56211c435790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f05b00124a0 tx=0x7f05b0005660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47b8eaff5bb5bae0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.223+0000 7f05a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:52:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05a9ffb700 1 -- 172.21.15.93:0/87651009 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0009310 con 0x56211c428db0 2023-11-16T16:52:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05a9ffb700 1 -- 172.21.15.93:0/87651009 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f05b0000e40 con 0x56211c428db0 2023-11-16T16:52:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05dbca5b80 1 -- 172.21.15.93:0/87651009 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05b0004cf0 con 0x56211c428db0 2023-11-16T16:52:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05a9ffb700 1 -- 172.21.15.93:0/87651009 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05b0004920 con 0x56211c428db0 2023-11-16T16:52:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05dbca5b80 10 client.25628 fetch_fsmap learned FSMap version 26 2023-11-16T16:52:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05dbca5b80 1 -- 172.21.15.93:0/87651009 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56211c340130 con 0x56211c428db0 2023-11-16T16:52:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05a9ffb700 1 -- 172.21.15.93:0/87651009 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05b0004cf0 con 0x56211c428db0 2023-11-16T16:52:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05a9ffb700 1 -- 172.21.15.93:0/87651009 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 26) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f05b003e070 con 0x56211c428db0 2023-11-16T16:52:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05dbca5b80 10 client.25628 fetch_fsmap finished waiting for FSMap version 26 2023-11-16T16:52:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05dbca5b80 10 client.25628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:52:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05dbca5b80 20 client.25628 populate_metadata read hostname 'smithi093' 2023-11-16T16:52:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56211c492740 0x56211c494b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05dbca5b80 4 client.25628 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:52:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.224+0000 7f05dbca5b80 1 -- 172.21.15.93:0/87651009 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56211c348600 con 0x56211c492740 2023-11-16T16:52:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.225+0000 7f05c619b700 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56211c492740 0x56211c494b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.225+0000 7f05c619b700 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56211c492740 0x56211c494b50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a6826a0cdf3da8 server_cookie=6b3d337fd10cd799 in_seq=0 out_seq=0 2023-11-16T16:52:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:32.225+0000 7f05a9ffb700 10 client.25628 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:32.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:32 smithi093 ceph-mon[97863]: pgmap v462: 129 pgs: 129 active+clean; 131 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 364 op/s 2023-11-16T16:52:32.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:32 smithi159 ceph-mon[131528]: pgmap v462: 129 pgs: 129 active+clean; 131 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 364 op/s 2023-11-16T16:52:33.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:32 smithi125 ceph-mon[131076]: pgmap v462: 129 pgs: 129 active+clean; 131 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 364 op/s 2023-11-16T16:52:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.259+0000 7f05a9ffb700 1 -- 172.21.15.93:0/87651009 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56211c348600 con 0x56211c492740 2023-11-16T16:52:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.259+0000 7f05a9ffb700 10 client.25628 handle_command_reply: tid=0 2023-11-16T16:52:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.259+0000 7f05dbca5b80 1 client.25628 shutdown 2023-11-16T16:52:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.260+0000 7f05dbca5b80 1 -- 172.21.15.93:0/87651009 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c428db0 msgr2=0x56211c435790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.260+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c428db0 0x56211c435790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f05b00124a0 tx=0x7f05b0005660 comp rx=0 tx=0).stop 2023-11-16T16:52:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.260+0000 7f05dbca5b80 1 -- 172.21.15.93:0/87651009 shutdown_connections 2023-11-16T16:52:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.260+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56211c492740 0x56211c494b50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.260+0000 7f05a9ffb700 0 client.25628 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.260+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56211c2846b0 0x56211c435250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.260+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56211c429b10 0x56211c4387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.260+0000 7f05dbca5b80 1 --2- 172.21.15.93:0/87651009 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56211c428db0 0x56211c435790 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.260+0000 7f05dbca5b80 1 -- 172.21.15.93:0/87651009 >> 172.21.15.93:0/87651009 conn(0x56211c41df50 msgr2=0x56211c42fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.261+0000 7f05dbca5b80 1 -- 172.21.15.93:0/87651009 shutdown_connections 2023-11-16T16:52:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.261+0000 7f05dbca5b80 1 -- 172.21.15.93:0/87651009 wait complete. 2023-11-16T16:52:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.261+0000 7f05dbca5b80 20 client.25628 trim_cache size 0 max 16384 2023-11-16T16:52:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:52:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:52:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:52:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:52:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.262+0000 7f05c83ff700 1 -- 172.21.15.93:0/2359305165 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f05ac0a11d0 msgr2=0x7f05ac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.262+0000 7f05c83ff700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f05ac0a11d0 0x7f05ac0a3680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009fd0 tx=0x7f05b800b040 comp rx=0 tx=0).stop 2023-11-16T16:52:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.262+0000 7f05c83ff700 1 -- 172.21.15.93:0/2359305165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05c01002c0 msgr2=0x7f05c00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.262+0000 7f05c83ff700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05c01002c0 0x7f05c00748a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05bc005e90 tx=0x7f05bc00b7e0 comp rx=0 tx=0).stop 2023-11-16T16:52:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.262+0000 7f05c83ff700 1 -- 172.21.15.93:0/2359305165 shutdown_connections 2023-11-16T16:52:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.262+0000 7f05c83ff700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f05ac0a11d0 0x7f05ac0a3680 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.262+0000 7f05c83ff700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c0101e40 0x7f05c0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.262+0000 7f05c83ff700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05c01002c0 0x7f05c00748a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.262+0000 7f05c83ff700 1 --2- 172.21.15.93:0/2359305165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05c01014b0 0x7f05c0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.262+0000 7f05c83ff700 1 -- 172.21.15.93:0/2359305165 >> 172.21.15.93:0/2359305165 conn(0x7f05c00fb8e0 msgr2=0x7f05c0104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.262+0000 7f05c83ff700 1 -- 172.21.15.93:0/2359305165 shutdown_connections 2023-11-16T16:52:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:34.262+0000 7f05c83ff700 1 -- 172.21.15.93:0/2359305165 wait complete. 2023-11-16T16:52:34.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:34 smithi093 ceph-mon[97863]: pgmap v463: 129 pgs: 129 active+clean; 149 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.8 MiB/s wr, 256 op/s 2023-11-16T16:52:34.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:34 smithi159 ceph-mon[131528]: pgmap v463: 129 pgs: 129 active+clean; 149 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.8 MiB/s wr, 256 op/s 2023-11-16T16:52:35.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:34 smithi125 ceph-mon[131076]: pgmap v463: 129 pgs: 129 active+clean; 149 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.8 MiB/s wr, 256 op/s 2023-11-16T16:52:36.437 INFO:tasks.workunit.client.0.smithi093.stdout:created hierarchy, now cleaning up 2023-11-16T16:52:36.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:36 smithi093 ceph-mon[97863]: pgmap v464: 129 pgs: 129 active+clean; 152 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 259 op/s 2023-11-16T16:52:36.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:36 smithi159 ceph-mon[131528]: pgmap v464: 129 pgs: 129 active+clean; 152 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 259 op/s 2023-11-16T16:52:37.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:36 smithi125 ceph-mon[131076]: pgmap v464: 129 pgs: 129 active+clean; 152 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 259 op/s 2023-11-16T16:52:38.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:38 smithi093 ceph-mon[97863]: pgmap v465: 129 pgs: 129 active+clean; 149 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 376 op/s 2023-11-16T16:52:38.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:38 smithi159 ceph-mon[131528]: pgmap v465: 129 pgs: 129 active+clean; 149 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 376 op/s 2023-11-16T16:52:39.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:38 smithi125 ceph-mon[131076]: pgmap v465: 129 pgs: 129 active+clean; 149 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 376 op/s 2023-11-16T16:52:40.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:40 smithi093 ceph-mon[97863]: pgmap v466: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 529 op/s 2023-11-16T16:52:40.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:40 smithi159 ceph-mon[131528]: pgmap v466: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 529 op/s 2023-11-16T16:52:41.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:40 smithi125 ceph-mon[131076]: pgmap v466: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 529 op/s 2023-11-16T16:52:42.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:42 smithi093 ceph-mon[97863]: pgmap v467: 129 pgs: 129 active+clean; 156 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 599 op/s 2023-11-16T16:52:42.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:42 smithi159 ceph-mon[131528]: pgmap v467: 129 pgs: 129 active+clean; 156 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 599 op/s 2023-11-16T16:52:43.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:42 smithi125 ceph-mon[131076]: pgmap v467: 129 pgs: 129 active+clean; 156 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 599 op/s 2023-11-16T16:52:44.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:44 smithi159 ceph-mon[131528]: pgmap v468: 129 pgs: 129 active+clean; 156 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 635 op/s 2023-11-16T16:52:44.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:44 smithi159 ceph-mon[131528]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:52:45.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:44 smithi125 ceph-mon[131076]: pgmap v468: 129 pgs: 129 active+clean; 156 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 635 op/s 2023-11-16T16:52:45.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:44 smithi125 ceph-mon[131076]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:52:45.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:44 smithi093 ceph-mon[97863]: pgmap v468: 129 pgs: 129 active+clean; 156 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 635 op/s 2023-11-16T16:52:45.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:44 smithi093 ceph-mon[97863]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:52:46.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:46 smithi159 ceph-mon[131528]: pgmap v469: 129 pgs: 129 active+clean; 163 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 623 op/s 2023-11-16T16:52:47.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:46 smithi125 ceph-mon[131076]: pgmap v469: 129 pgs: 129 active+clean; 163 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 623 op/s 2023-11-16T16:52:47.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:46 smithi093 ceph-mon[97863]: pgmap v469: 129 pgs: 129 active+clean; 163 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 623 op/s 2023-11-16T16:52:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.566+0000 7f11613ff700 1 -- 172.21.15.93:0/2680330147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c073550 msgr2=0x7f115c0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.566+0000 7f11613ff700 1 --2- 172.21.15.93:0/2680330147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c073550 0x7f115c0739c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1154009f90 tx=0x7f115400b330 comp rx=0 tx=0).stop 2023-11-16T16:52:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.569+0000 7f11613ff700 1 -- 172.21.15.93:0/2680330147 shutdown_connections 2023-11-16T16:52:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.569+0000 7f11613ff700 1 --2- 172.21.15.93:0/2680330147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f115c10c810 0x7f115c10ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.569+0000 7f11613ff700 1 --2- 172.21.15.93:0/2680330147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f115c074b30 0x7f115c072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.569+0000 7f11613ff700 1 --2- 172.21.15.93:0/2680330147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c073550 0x7f115c0739c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.569+0000 7f11613ff700 1 -- 172.21.15.93:0/2680330147 >> 172.21.15.93:0/2680330147 conn(0x7f115c0fb840 msgr2=0x7f115c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.570+0000 7f11613ff700 1 -- 172.21.15.93:0/2680330147 shutdown_connections 2023-11-16T16:52:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.570+0000 7f11613ff700 1 -- 172.21.15.93:0/2680330147 wait complete. 2023-11-16T16:52:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.572+0000 7f11613ff700 1 Processor -- start 2023-11-16T16:52:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.574+0000 7f11613ff700 1 -- start start 2023-11-16T16:52:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.574+0000 7f11613ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f115c073550 0x7f115c19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.574+0000 7f11613ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c074b30 0x7f115c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.574+0000 7f11613ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f115c10c810 0x7f115c1a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.574+0000 7f11613ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c1a4360 con 0x7f115c10c810 2023-11-16T16:52:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.574+0000 7f11613ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c06a510 con 0x7f115c074b30 2023-11-16T16:52:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.574+0000 7f11613ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c06a6a0 con 0x7f115c073550 2023-11-16T16:52:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.576+0000 7f115affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f115c073550 0x7f115c19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.576+0000 7f115a7fc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c074b30 0x7f115c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.576+0000 7f115affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f115c073550 0x7f115c19c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:55012/0 (socket says 172.21.15.93:55012) 2023-11-16T16:52:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.576+0000 7f115affd700 1 -- 172.21.15.93:0/1512277223 learned_addr learned my addr 172.21.15.93:0/1512277223 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.576+0000 7f115b7fe700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f115c10c810 0x7f115c1a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.576+0000 7f115affd700 1 -- 172.21.15.93:0/1512277223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c074b30 msgr2=0x7f115c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.576+0000 7f115affd700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c074b30 0x7f115c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.576+0000 7f115affd700 1 -- 172.21.15.93:0/1512277223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f115c10c810 msgr2=0x7f115c1a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.576+0000 7f115affd700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f115c10c810 0x7f115c1a3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.576+0000 7f115affd700 1 -- 172.21.15.93:0/1512277223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1154009c40 con 0x7f115c073550 2023-11-16T16:52:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.577+0000 7f115affd700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f115c073550 0x7f115c19c7e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f115000e9f0 tx=0x7f115000edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.577+0000 7f1143fff700 1 -- 172.21.15.93:0/1512277223 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115000cbd0 con 0x7f115c073550 2023-11-16T16:52:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.578+0000 7f1143fff700 1 -- 172.21.15.93:0/1512277223 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115000cd30 con 0x7f115c073550 2023-11-16T16:52:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.578+0000 7f1143fff700 1 -- 172.21.15.93:0/1512277223 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150018710 con 0x7f115c073550 2023-11-16T16:52:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.578+0000 7f11613ff700 1 -- 172.21.15.93:0/1512277223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c112220 con 0x7f115c073550 2023-11-16T16:52:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.579+0000 7f11613ff700 1 -- 172.21.15.93:0/1512277223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c112090 con 0x7f115c073550 2023-11-16T16:52:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.584+0000 7f1143fff700 1 -- 172.21.15.93:0/1512277223 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1150010980 con 0x7f115c073550 2023-11-16T16:52:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.588+0000 7f1174d35b80 1 Processor -- start 2023-11-16T16:52:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.588+0000 7f1143fff700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f11440a12a0 0x7f11440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.589+0000 7f1143fff700 1 -- 172.21.15.93:0/1512277223 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f1150014070 con 0x7f115c073550 2023-11-16T16:52:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.589+0000 7f115b7fe700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f11440a12a0 0x7f11440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.590+0000 7f1174d35b80 1 -- start start 2023-11-16T16:52:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.590+0000 7f1174d35b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x563375077870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.590+0000 7f1174d35b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 0x5633750785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.590+0000 7f1174d35b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375078b10 0x563375078f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.590+0000 7f1174d35b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563374f7d000 con 0x563375077db0 2023-11-16T16:52:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.590+0000 7f1174d35b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563374fcf840 con 0x563375078b10 2023-11-16T16:52:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.590+0000 7f1174d35b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563374fc8cd0 con 0x563374ed36b0 2023-11-16T16:52:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.590+0000 7f115affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x563375077870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.590+0000 7f115affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x563375077870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:55022/0 (socket says 172.21.15.93:55022) 2023-11-16T16:52:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.590+0000 7f115affd700 1 -- 172.21.15.93:0/1199583943 learned_addr learned my addr 172.21.15.93:0/1199583943 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.591+0000 7f115affd700 1 -- 172.21.15.93:0/1199583943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375078b10 msgr2=0x563375078f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.591+0000 7f115affd700 1 --2- 172.21.15.93:0/1199583943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375078b10 0x563375078f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.591+0000 7f115affd700 1 -- 172.21.15.93:0/1199583943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 msgr2=0x5633750785d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-16T16:52:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.591+0000 7f115affd700 1 --2- 172.21.15.93:0/1199583943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 0x5633750785d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.591+0000 7f115affd700 1 -- 172.21.15.93:0/1199583943 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563374e4a5d0 con 0x563374ed36b0 2023-11-16T16:52:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.592+0000 7f115affd700 1 --2- 172.21.15.93:0/1199583943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x563375077870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11500a1ec0 tx=0x7f115001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5d6d6169faac16d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.592+0000 7f11417fa700 1 -- 172.21.15.93:0/1199583943 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500b1070 con 0x563374ed36b0 2023-11-16T16:52:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.592+0000 7f11417fa700 1 -- 172.21.15.93:0/1199583943 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115009dfd0 con 0x563374ed36b0 2023-11-16T16:52:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.592+0000 7f11417fa700 1 -- 172.21.15.93:0/1199583943 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500996f0 con 0x563374ed36b0 2023-11-16T16:52:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.594+0000 7f1174d35b80 1 -- 172.21.15.93:0/1199583943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 msgr2=0x563375077870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.594+0000 7f1174d35b80 1 --2- 172.21.15.93:0/1199583943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x563375077870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11500a1ec0 tx=0x7f115001f000 comp rx=0 tx=0).stop 2023-11-16T16:52:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.595+0000 7f115b7fe700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f11440a12a0 0x7f11440a3750 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f114c00b9f0 tx=0x7f114c00bdb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.596+0000 7f1174d35b80 1 -- 172.21.15.93:0/1199583943 shutdown_connections 2023-11-16T16:52:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.596+0000 7f1174d35b80 1 --2- 172.21.15.93:0/1199583943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375078b10 0x563375078f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.596+0000 7f1174d35b80 1 --2- 172.21.15.93:0/1199583943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x563375077870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.596+0000 7f1174d35b80 1 --2- 172.21.15.93:0/1199583943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 0x5633750785d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.596+0000 7f1174d35b80 1 -- 172.21.15.93:0/1199583943 >> 172.21.15.93:0/1199583943 conn(0x56337506cf50 msgr2=0x563375072100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.597+0000 7f1174d35b80 1 -- 172.21.15.93:0/1199583943 shutdown_connections 2023-11-16T16:52:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.597+0000 7f1174d35b80 1 -- 172.21.15.93:0/1199583943 wait complete. 2023-11-16T16:52:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.598+0000 7f1174d35b80 1 -- start start 2023-11-16T16:52:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.599+0000 7f1174d35b80 1 Processor -- start 2023-11-16T16:52:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.599+0000 7f1174d35b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x563375080c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.599+0000 7f1174d35b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 0x5633750811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.599+0000 7f1174d35b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375078b10 0x5633750881f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.601+0000 7f1174d35b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563374f6a880 con 0x563375077db0 2023-11-16T16:52:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.601+0000 7f1174d35b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563374f6a520 con 0x563375078b10 2023-11-16T16:52:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.601+0000 7f1174d35b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563374e4c8c0 con 0x563374ed36b0 2023-11-16T16:52:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.601+0000 7f1174d35b80 10 client.0 _ll_register_callbacks cb 0x563374f88720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:52:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.601+0000 7f115a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 0x5633750811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.601+0000 7f115a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 0x5633750811b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54808/0 (socket says 172.21.15.93:54808) 2023-11-16T16:52:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.601+0000 7f115a7fc700 1 -- 172.21.15.93:0/513877230 learned_addr learned my addr 172.21.15.93:0/513877230 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.601+0000 7f115b7fe700 1 --2- 172.21.15.93:0/513877230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375078b10 0x5633750881f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.603+0000 7f115b7fe700 1 -- 172.21.15.93:0/513877230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 msgr2=0x563375080c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.603+0000 7f115b7fe700 1 --2- 172.21.15.93:0/513877230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x563375080c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.603+0000 7f115b7fe700 1 -- 172.21.15.93:0/513877230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 msgr2=0x5633750811b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.603+0000 7f115b7fe700 1 --2- 172.21.15.93:0/513877230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 0x5633750811b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.603+0000 7f115b7fe700 1 -- 172.21.15.93:0/513877230 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c112090 con 0x563375078b10 2023-11-16T16:52:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.603+0000 7f115b7fe700 1 --2- 172.21.15.93:0/513877230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375078b10 0x5633750881f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f114c0116c0 tx=0x7f114c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b80dbc143efd9a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.603+0000 7f113eff5700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:52:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.604+0000 7f113eff5700 1 -- 172.21.15.93:0/513877230 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c01d820 con 0x563375078b10 2023-11-16T16:52:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.604+0000 7f113eff5700 1 -- 172.21.15.93:0/513877230 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f114c00b5f0 con 0x563375078b10 2023-11-16T16:52:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.604+0000 7f1174d35b80 1 -- 172.21.15.93:0/513877230 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56337506a710 con 0x563375078b10 2023-11-16T16:52:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.604+0000 7f113eff5700 1 -- 172.21.15.93:0/513877230 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f114c01d980 con 0x563375078b10 2023-11-16T16:52:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.605+0000 7f1174d35b80 10 client.25637 fetch_fsmap learned FSMap version 27 2023-11-16T16:52:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.605+0000 7f1174d35b80 1 -- 172.21.15.93:0/513877230 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563374f8f130 con 0x563375078b10 2023-11-16T16:52:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.606+0000 7f113eff5700 1 -- 172.21.15.93:0/513877230 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56337506a710 con 0x563375078b10 2023-11-16T16:52:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.606+0000 7f113eff5700 1 -- 172.21.15.93:0/513877230 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f114c00f410 con 0x563375078b10 2023-11-16T16:52:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.606+0000 7f1174d35b80 10 client.25637 fetch_fsmap finished waiting for FSMap version 27 2023-11-16T16:52:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.606+0000 7f1174d35b80 10 client.25637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:52:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.606+0000 7f1174d35b80 20 client.25637 populate_metadata read hostname 'smithi093' 2023-11-16T16:52:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.606+0000 7f1174d35b80 1 --2- 172.21.15.93:0/513877230 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56337508cb00 0x56337508ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.606+0000 7f1174d35b80 4 client.25637 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:52:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.606+0000 7f1174d35b80 1 -- 172.21.15.93:0/513877230 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563374f8b040 con 0x56337508cb00 2023-11-16T16:52:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.607+0000 7f115affd700 1 --2- 172.21.15.93:0/513877230 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56337508cb00 0x56337508ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.608+0000 7f113eff5700 10 client.25637 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.610+0000 7f115affd700 1 --2- 172.21.15.93:0/513877230 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56337508cb00 0x56337508ef10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b807cc6f2fe0f0b1 server_cookie=6871ca206b54c246 in_seq=0 out_seq=0 2023-11-16T16:52:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.612+0000 7f113eff5700 1 -- 172.21.15.93:0/513877230 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563374f8b040 con 0x56337508cb00 2023-11-16T16:52:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.612+0000 7f113eff5700 10 client.25637 handle_command_reply: tid=0 2023-11-16T16:52:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.613+0000 7f1174d35b80 1 client.25637 shutdown 2023-11-16T16:52:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.614+0000 7f1174d35b80 1 -- 172.21.15.93:0/513877230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375078b10 msgr2=0x5633750881f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.614+0000 7f1174d35b80 1 --2- 172.21.15.93:0/513877230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375078b10 0x5633750881f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f114c0116c0 tx=0x7f114c004750 comp rx=0 tx=0).stop 2023-11-16T16:52:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.614+0000 7f1174d35b80 1 -- 172.21.15.93:0/513877230 shutdown_connections 2023-11-16T16:52:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.614+0000 7f1174d35b80 1 --2- 172.21.15.93:0/513877230 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x56337508cb00 0x56337508ef10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.614+0000 7f1174d35b80 1 --2- 172.21.15.93:0/513877230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375078b10 0x5633750881f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.614+0000 7f1174d35b80 1 --2- 172.21.15.93:0/513877230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x563375080c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.614+0000 7f1174d35b80 1 --2- 172.21.15.93:0/513877230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 0x5633750811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.614+0000 7f1174d35b80 1 -- 172.21.15.93:0/513877230 >> 172.21.15.93:0/513877230 conn(0x56337506cf50 msgr2=0x5633750717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.615+0000 7f113eff5700 0 client.25637 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.615+0000 7f1174d35b80 1 -- 172.21.15.93:0/513877230 shutdown_connections 2023-11-16T16:52:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.615+0000 7f1174d35b80 1 -- 172.21.15.93:0/513877230 wait complete. 2023-11-16T16:52:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.616+0000 7f1174d35b80 20 client.25637 trim_cache size 0 max 16384 2023-11-16T16:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.653+0000 7f1174d35b80 1 Processor -- start 2023-11-16T16:52:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.653+0000 7f1174d35b80 1 -- start start 2023-11-16T16:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.654+0000 7f1174d35b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x56337507eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.654+0000 7f1174d35b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375077db0 0x56337507f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.654+0000 7f1174d35b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375078b10 0x5633750881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.655+0000 7f115affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x56337507eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.655+0000 7f1174d35b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563374f8b040 con 0x563375078b10 2023-11-16T16:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.655+0000 7f1174d35b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563375080120 con 0x563375077db0 2023-11-16T16:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.655+0000 7f1174d35b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563375078450 con 0x563374ed36b0 2023-11-16T16:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.655+0000 7f115affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x56337507eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:55048/0 (socket says 172.21.15.93:55048) 2023-11-16T16:52:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.655+0000 7f115affd700 1 -- 172.21.15.93:0/3731040034 learned_addr learned my addr 172.21.15.93:0/3731040034 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.655+0000 7f115b7fe700 1 --2- 172.21.15.93:0/3731040034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375078b10 0x5633750881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.656+0000 7f115a7fc700 1 --2- 172.21.15.93:0/3731040034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375077db0 0x56337507f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.656+0000 7f115affd700 1 -- 172.21.15.93:0/3731040034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375077db0 msgr2=0x56337507f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.656+0000 7f115affd700 1 --2- 172.21.15.93:0/3731040034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375077db0 0x56337507f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.656+0000 7f115affd700 1 -- 172.21.15.93:0/3731040034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375078b10 msgr2=0x5633750881b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.656+0000 7f115affd700 1 --2- 172.21.15.93:0/3731040034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375078b10 0x5633750881b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.656+0000 7f115affd700 1 -- 172.21.15.93:0/3731040034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633750791b0 con 0x563374ed36b0 2023-11-16T16:52:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.656+0000 7f115b7fe700 1 --2- 172.21.15.93:0/3731040034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375078b10 0x5633750881b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:52:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.656+0000 7f115affd700 1 --2- 172.21.15.93:0/3731040034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x56337507eeb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f115000c5b0 tx=0x7f115009d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b2cf3c6bbbe5e96 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.657+0000 7f1141ffb700 1 -- 172.21.15.93:0/3731040034 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500b1030 con 0x563374ed36b0 2023-11-16T16:52:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.657+0000 7f1141ffb700 1 -- 172.21.15.93:0/3731040034 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11500981a0 con 0x563374ed36b0 2023-11-16T16:52:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.658+0000 7f1141ffb700 1 -- 172.21.15.93:0/3731040034 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500996f0 con 0x563374ed36b0 2023-11-16T16:52:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.658+0000 7f1174d35b80 1 -- 172.21.15.93:0/3731040034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 msgr2=0x56337507eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.658+0000 7f1174d35b80 1 --2- 172.21.15.93:0/3731040034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x56337507eeb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f115000c5b0 tx=0x7f115009d040 comp rx=0 tx=0).stop 2023-11-16T16:52:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.658+0000 7f1174d35b80 1 -- 172.21.15.93:0/3731040034 shutdown_connections 2023-11-16T16:52:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.658+0000 7f1174d35b80 1 --2- 172.21.15.93:0/3731040034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375078b10 0x5633750881b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.658+0000 7f1174d35b80 1 --2- 172.21.15.93:0/3731040034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563374ed36b0 0x56337507eeb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.658+0000 7f1174d35b80 1 --2- 172.21.15.93:0/3731040034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563375077db0 0x56337507f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.658+0000 7f1174d35b80 1 -- 172.21.15.93:0/3731040034 >> 172.21.15.93:0/3731040034 conn(0x56337506cf50 msgr2=0x563375072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.659+0000 7f1174d35b80 1 -- 172.21.15.93:0/3731040034 shutdown_connections 2023-11-16T16:52:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.659+0000 7f1174d35b80 1 -- 172.21.15.93:0/3731040034 wait complete. 2023-11-16T16:52:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.659+0000 7f1174d35b80 1 -- start start 2023-11-16T16:52:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.660+0000 7f1174d35b80 1 Processor -- start 2023-11-16T16:52:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.660+0000 7f1174d35b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563374ed36b0 0x563375083b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.660+0000 7f1174d35b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 0x563375084080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.660+0000 7f1174d35b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563375078b10 0x5633750870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.660+0000 7f1174d35b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633750875e0 con 0x563375077db0 2023-11-16T16:52:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.660+0000 7f1174d35b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563375087750 con 0x563374ed36b0 2023-11-16T16:52:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.660+0000 7f1174d35b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633750878c0 con 0x563375078b10 2023-11-16T16:52:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.660+0000 7f1174d35b80 10 client.0 _ll_register_callbacks cb 0x7f114c00dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:52:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.661+0000 7f115affd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563374ed36b0 0x563375083b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.661+0000 7f115affd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563374ed36b0 0x563375083b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:48750/0 (socket says 172.21.15.93:48750) 2023-11-16T16:52:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.661+0000 7f115affd700 1 -- 172.21.15.93:0/1554564427 learned_addr learned my addr 172.21.15.93:0/1554564427 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:52:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.662+0000 7f115a7fc700 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 0x563375084080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.662+0000 7f115affd700 1 -- 172.21.15.93:0/1554564427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563375078b10 msgr2=0x5633750870a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.662+0000 7f115b7fe700 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563375078b10 0x5633750870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.662+0000 7f115affd700 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563375078b10 0x5633750870a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.662+0000 7f115affd700 1 -- 172.21.15.93:0/1554564427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 msgr2=0x563375084080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:52:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.662+0000 7f115affd700 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 0x563375084080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.662+0000 7f115affd700 1 -- 172.21.15.93:0/1554564427 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563374e4a5d0 con 0x563374ed36b0 2023-11-16T16:52:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.662+0000 7f115b7fe700 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563375078b10 0x5633750870a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:52:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.663+0000 7f115affd700 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563374ed36b0 0x563375083b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f11500caa00 tx=0x7f115000c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7e00d8bcdb55845 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:52:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.663+0000 7f113eff5700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:52:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.663+0000 7f113eff5700 1 -- 172.21.15.93:0/1554564427 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500b1040 con 0x563374ed36b0 2023-11-16T16:52:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.663+0000 7f113eff5700 1 -- 172.21.15.93:0/1554564427 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f11500b2070 con 0x563374ed36b0 2023-11-16T16:52:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.663+0000 7f1174d35b80 1 -- 172.21.15.93:0/1554564427 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563374f97600 con 0x563374ed36b0 2023-11-16T16:52:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.665+0000 7f113eff5700 1 -- 172.21.15.93:0/1554564427 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11500a1f60 con 0x563374ed36b0 2023-11-16T16:52:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.666+0000 7f1174d35b80 10 client.25643 fetch_fsmap learned FSMap version 27 2023-11-16T16:52:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.666+0000 7f1174d35b80 1 -- 172.21.15.93:0/1554564427 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563374df6d80 con 0x563374ed36b0 2023-11-16T16:52:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.666+0000 7f113eff5700 1 -- 172.21.15.93:0/1554564427 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563374f97600 con 0x563374ed36b0 2023-11-16T16:52:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.666+0000 7f113eff5700 1 -- 172.21.15.93:0/1554564427 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f1150010430 con 0x563374ed36b0 2023-11-16T16:52:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.667+0000 7f1174d35b80 10 client.25643 fetch_fsmap finished waiting for FSMap version 27 2023-11-16T16:52:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.667+0000 7f1174d35b80 10 client.25643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:52:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.667+0000 7f1174d35b80 20 client.25643 populate_metadata read hostname 'smithi093' 2023-11-16T16:52:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.667+0000 7f1174d35b80 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5633750f2210 0x5633750f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:52:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.667+0000 7f1174d35b80 4 client.25643 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:52:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.667+0000 7f1174d35b80 1 -- 172.21.15.93:0/1554564427 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563374e17f10 con 0x5633750f2210 2023-11-16T16:52:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.667+0000 7f115a7fc700 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5633750f2210 0x5633750f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:52:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.668+0000 7f115a7fc700 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5633750f2210 0x5633750f4620 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01685abcced1e11 server_cookie=6a3412fb58c5e462 in_seq=0 out_seq=0 2023-11-16T16:52:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:47.668+0000 7f113eff5700 10 client.25643 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:48.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:48 smithi159 ceph-mon[131528]: pgmap v470: 129 pgs: 129 active+clean; 175 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 527 op/s 2023-11-16T16:52:49.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:48 smithi125 ceph-mon[131076]: pgmap v470: 129 pgs: 129 active+clean; 175 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 527 op/s 2023-11-16T16:52:49.153 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:48 smithi093 ceph-mon[97863]: pgmap v470: 129 pgs: 129 active+clean; 175 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 527 op/s 2023-11-16T16:52:49.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:49 smithi159 ceph-mon[131528]: pgmap v471: 129 pgs: 129 active+clean; 183 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 488 op/s 2023-11-16T16:52:49.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:49 smithi159 ceph-mon[131528]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:52:50.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:49 smithi125 ceph-mon[131076]: pgmap v471: 129 pgs: 129 active+clean; 183 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 488 op/s 2023-11-16T16:52:50.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:49 smithi125 ceph-mon[131076]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:52:50.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:49 smithi093 ceph-mon[97863]: pgmap v471: 129 pgs: 129 active+clean; 183 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 488 op/s 2023-11-16T16:52:50.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:49 smithi093 ceph-mon[97863]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:52:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.858+0000 7f113eff5700 1 -- 172.21.15.93:0/1554564427 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563374e17f10 con 0x5633750f2210 2023-11-16T16:52:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.858+0000 7f113eff5700 10 client.25643 handle_command_reply: tid=0 2023-11-16T16:52:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.858+0000 7f1174d35b80 1 client.25643 shutdown 2023-11-16T16:52:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.859+0000 7f1174d35b80 1 -- 172.21.15.93:0/1554564427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563374ed36b0 msgr2=0x563375083b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.859+0000 7f1174d35b80 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563374ed36b0 0x563375083b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f11500caa00 tx=0x7f115000c700 comp rx=0 tx=0).stop 2023-11-16T16:52:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.859+0000 7f1174d35b80 1 -- 172.21.15.93:0/1554564427 shutdown_connections 2023-11-16T16:52:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.859+0000 7f1174d35b80 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5633750f2210 0x5633750f4620 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.859+0000 7f1174d35b80 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563375078b10 0x5633750870a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.859+0000 7f1174d35b80 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563374ed36b0 0x563375083b40 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f11500caa00 tx=0x7f115000c700 comp rx=0 tx=0).stop 2023-11-16T16:52:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.859+0000 7f1174d35b80 1 --2- 172.21.15.93:0/1554564427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563375077db0 0x563375084080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.859+0000 7f1174d35b80 1 -- 172.21.15.93:0/1554564427 >> 172.21.15.93:0/1554564427 conn(0x56337506cf50 msgr2=0x56337507eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.860+0000 7f113eff5700 0 client.25643 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:52:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.860+0000 7f1174d35b80 1 -- 172.21.15.93:0/1554564427 shutdown_connections 2023-11-16T16:52:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.860+0000 7f1174d35b80 1 -- 172.21.15.93:0/1554564427 wait complete. 2023-11-16T16:52:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.861+0000 7f1174d35b80 20 client.25643 trim_cache size 0 max 16384 2023-11-16T16:52:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:52:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:52:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:52:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:52:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.862+0000 7f11613ff700 1 -- 172.21.15.93:0/1512277223 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f11440a12a0 msgr2=0x7f11440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.862+0000 7f11613ff700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f11440a12a0 0x7f11440a3750 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f114c00b9f0 tx=0x7f114c00bdb0 comp rx=0 tx=0).stop 2023-11-16T16:52:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.862+0000 7f11613ff700 1 -- 172.21.15.93:0/1512277223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f115c073550 msgr2=0x7f115c19c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:52:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.862+0000 7f11613ff700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f115c073550 0x7f115c19c7e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f115000e9f0 tx=0x7f115000edb0 comp rx=0 tx=0).stop 2023-11-16T16:52:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.862+0000 7f11613ff700 1 -- 172.21.15.93:0/1512277223 shutdown_connections 2023-11-16T16:52:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.862+0000 7f11613ff700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f11440a12a0 0x7f11440a3750 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.862+0000 7f11613ff700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f115c10c810 0x7f115c1a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.862+0000 7f11613ff700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f115c073550 0x7f115c19c7e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.862+0000 7f11613ff700 1 --2- 172.21.15.93:0/1512277223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c074b30 0x7f115c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:52:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.862+0000 7f11613ff700 1 -- 172.21.15.93:0/1512277223 >> 172.21.15.93:0/1512277223 conn(0x7f115c0fb840 msgr2=0x7f115c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:52:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.862+0000 7f11613ff700 1 -- 172.21.15.93:0/1512277223 shutdown_connections 2023-11-16T16:52:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:52:50.862+0000 7f11613ff700 1 -- 172.21.15.93:0/1512277223 wait complete. 2023-11-16T16:52:52.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:52 smithi159 ceph-mon[131528]: pgmap v472: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 706 op/s 2023-11-16T16:52:52.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:52 smithi125 ceph-mon[131076]: pgmap v472: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 706 op/s 2023-11-16T16:52:52.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:52 smithi093 ceph-mon[97863]: pgmap v472: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 706 op/s 2023-11-16T16:52:54.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:54 smithi159 ceph-mon[131528]: pgmap v473: 129 pgs: 129 active+clean; 194 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 1.15k op/s 2023-11-16T16:52:54.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:54 smithi125 ceph-mon[131076]: pgmap v473: 129 pgs: 129 active+clean; 194 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 1.15k op/s 2023-11-16T16:52:54.902 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:54 smithi093 ceph-mon[97863]: pgmap v473: 129 pgs: 129 active+clean; 194 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 1.15k op/s 2023-11-16T16:52:56.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:56 smithi159 ceph-mon[131528]: pgmap v474: 129 pgs: 129 active+clean; 197 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 1.40k op/s 2023-11-16T16:52:56.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:56 smithi125 ceph-mon[131076]: pgmap v474: 129 pgs: 129 active+clean; 197 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 1.40k op/s 2023-11-16T16:52:56.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:56 smithi093 ceph-mon[97863]: pgmap v474: 129 pgs: 129 active+clean; 197 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 1.40k op/s 2023-11-16T16:52:58.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:52:58 smithi159 ceph-mon[131528]: pgmap v475: 129 pgs: 129 active+clean; 201 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 1.43k op/s 2023-11-16T16:52:58.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:52:58 smithi125 ceph-mon[131076]: pgmap v475: 129 pgs: 129 active+clean; 201 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 1.43k op/s 2023-11-16T16:52:58.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:52:58 smithi093 ceph-mon[97863]: pgmap v475: 129 pgs: 129 active+clean; 201 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 1.43k op/s 2023-11-16T16:53:00.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:00 smithi159 ceph-mon[131528]: pgmap v476: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 1.49k op/s 2023-11-16T16:53:00.830 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:00 smithi125 ceph-mon[131076]: pgmap v476: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 1.49k op/s 2023-11-16T16:53:00.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:00 smithi093 ceph-mon[97863]: pgmap v476: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 1.49k op/s 2023-11-16T16:53:02.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:02 smithi159 ceph-mon[131528]: pgmap v477: 129 pgs: 129 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 1.50k op/s 2023-11-16T16:53:02.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:02 smithi125 ceph-mon[131076]: pgmap v477: 129 pgs: 129 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 1.50k op/s 2023-11-16T16:53:02.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:02 smithi093 ceph-mon[97863]: pgmap v477: 129 pgs: 129 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 1.50k op/s 2023-11-16T16:53:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.159+0000 7f67eedff700 1 -- 172.21.15.93:0/2546612914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e8071ea0 msgr2=0x7f67e80722b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.160+0000 7f67eedff700 1 --2- 172.21.15.93:0/2546612914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e8071ea0 0x7f67e80722b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0074e0 tx=0x7f67dc00af90 comp rx=0 tx=0).stop 2023-11-16T16:53:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.160+0000 7f67eedff700 1 -- 172.21.15.93:0/2546612914 shutdown_connections 2023-11-16T16:53:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.160+0000 7f67eedff700 1 --2- 172.21.15.93:0/2546612914 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67e81087f0 0x7f67e810f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.160+0000 7f67eedff700 1 --2- 172.21.15.93:0/2546612914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e8071ea0 0x7f67e80722b0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0074e0 tx=0x7f67dc00af90 comp rx=0 tx=0).stop 2023-11-16T16:53:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.160+0000 7f67eedff700 1 --2- 172.21.15.93:0/2546612914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67e8107e60 0x7f67e81082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.160+0000 7f67eedff700 1 -- 172.21.15.93:0/2546612914 >> 172.21.15.93:0/2546612914 conn(0x7f67e806d4a0 msgr2=0x7f67e806f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.164+0000 7f67eedff700 1 -- 172.21.15.93:0/2546612914 shutdown_connections 2023-11-16T16:53:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.164+0000 7f67eedff700 1 -- 172.21.15.93:0/2546612914 wait complete. 2023-11-16T16:53:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.165+0000 7f67eedff700 1 Processor -- start 2023-11-16T16:53:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.166+0000 7f67eedff700 1 -- start start 2023-11-16T16:53:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.166+0000 7f67eedff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e8107e60 0x7f67e81a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.166+0000 7f67eedff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67e81087f0 0x7f67e81a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.167+0000 7f67eedff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67e81aa820 0x7f67e81acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.167+0000 7f67eedff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81ad1d0 con 0x7f67e81aa820 2023-11-16T16:53:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.167+0000 7f67eedff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81ad310 con 0x7f67e8107e60 2023-11-16T16:53:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.167+0000 7f67eedff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81ad4a0 con 0x7f67e81087f0 2023-11-16T16:53:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.167+0000 7f67ecb9b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e8107e60 0x7f67e81a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.167+0000 7f67ecb9b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e8107e60 0x7f67e81a5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:56100/0 (socket says 172.21.15.93:56100) 2023-11-16T16:53:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.167+0000 7f67ecb9b700 1 -- 172.21.15.93:0/2361727758 learned_addr learned my addr 172.21.15.93:0/2361727758 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.168+0000 7f67e7fff700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67e81087f0 0x7f67e81a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.168+0000 7f67ed39c700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67e81aa820 0x7f67e81acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.168+0000 7f67ecb9b700 1 -- 172.21.15.93:0/2361727758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67e81087f0 msgr2=0x7f67e81a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.168+0000 7f67ecb9b700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67e81087f0 0x7f67e81a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.168+0000 7f67ecb9b700 1 -- 172.21.15.93:0/2361727758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67e81aa820 msgr2=0x7f67e81acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.169+0000 7f67ecb9b700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67e81aa820 0x7f67e81acc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.170+0000 7f67ecb9b700 1 -- 172.21.15.93:0/2361727758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc0071c0 con 0x7f67e8107e60 2023-11-16T16:53:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.170+0000 7f67ed39c700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67e81aa820 0x7f67e81acc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:53:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.170+0000 7f67ecb9b700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e8107e60 0x7f67e81a5280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00a040 tx=0x7f67dc004300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.171+0000 7f67e5ffb700 1 -- 172.21.15.93:0/2361727758 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc01f070 con 0x7f67e8107e60 2023-11-16T16:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.171+0000 7f67e5ffb700 1 -- 172.21.15.93:0/2361727758 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67dc005500 con 0x7f67e8107e60 2023-11-16T16:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.171+0000 7f67e5ffb700 1 -- 172.21.15.93:0/2361727758 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00f040 con 0x7f67e8107e60 2023-11-16T16:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.171+0000 7f67eedff700 1 -- 172.21.15.93:0/2361727758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e8111e10 con 0x7f67e8107e60 2023-11-16T16:53:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.171+0000 7f67eedff700 1 -- 172.21.15.93:0/2361727758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e8111c80 con 0x7f67e8107e60 2023-11-16T16:53:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.179+0000 7f67e5ffb700 1 -- 172.21.15.93:0/2361727758 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f67dc004c50 con 0x7f67e8107e60 2023-11-16T16:53:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.180+0000 7f680273cb80 1 Processor -- start 2023-11-16T16:53:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.180+0000 7f680273cb80 1 -- start start 2023-11-16T16:53:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.180+0000 7f680273cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da087c86b0 0x55da0896c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.180+0000 7f680273cb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896cdb0 0x55da0896d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.180+0000 7f680273cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da0896db10 0x55da0896df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.180+0000 7f680273cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da08872000 con 0x55da087c86b0 2023-11-16T16:53:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.180+0000 7f680273cb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da088c4840 con 0x55da0896cdb0 2023-11-16T16:53:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.180+0000 7f680273cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da088bdcd0 con 0x55da0896db10 2023-11-16T16:53:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.181+0000 7f67ed39c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da087c86b0 0x55da0896c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.181+0000 7f67ed39c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da087c86b0 0x55da0896c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36374/0 (socket says 172.21.15.93:36374) 2023-11-16T16:53:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.181+0000 7f67ed39c700 1 -- 172.21.15.93:0/4056262904 learned_addr learned my addr 172.21.15.93:0/4056262904 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.181+0000 7f67ecb9b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896cdb0 0x55da0896d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.182+0000 7f67e7fff700 1 --2- 172.21.15.93:0/4056262904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da0896db10 0x55da0896df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.183+0000 7f67ed39c700 1 -- 172.21.15.93:0/4056262904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da0896db10 msgr2=0x55da0896df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.183+0000 7f67ed39c700 1 --2- 172.21.15.93:0/4056262904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da0896db10 0x55da0896df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.183+0000 7f67ed39c700 1 -- 172.21.15.93:0/4056262904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896cdb0 msgr2=0x55da0896d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.184+0000 7f67ed39c700 1 --2- 172.21.15.93:0/4056262904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896cdb0 0x55da0896d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.184+0000 7f67ed39c700 1 -- 172.21.15.93:0/4056262904 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da0873f5d0 con 0x55da087c86b0 2023-11-16T16:53:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.184+0000 7f67ecb9b700 1 --2- 172.21.15.93:0/4056262904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896cdb0 0x55da0896d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:53:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.184+0000 7f67e5ffb700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f67d00a12a0 0x7f67d00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.184+0000 7f67ed39c700 1 --2- 172.21.15.93:0/4056262904 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da087c86b0 0x55da0896c870 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f67d800aa10 tx=0x7f67d800ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e335d2beb86a21c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.185+0000 7f67ecb9b700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f67d00a12a0 0x7f67d00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.186+0000 7f67e5ffb700 1 -- 172.21.15.93:0/2361727758 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f67dc0cc660 con 0x7f67e8107e60 2023-11-16T16:53:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.186+0000 7f67ceffd700 1 -- 172.21.15.93:0/4056262904 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d800ca90 con 0x55da087c86b0 2023-11-16T16:53:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.186+0000 7f67ceffd700 1 -- 172.21.15.93:0/4056262904 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d800cbf0 con 0x55da087c86b0 2023-11-16T16:53:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.187+0000 7f67ceffd700 1 -- 172.21.15.93:0/4056262904 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d80105b0 con 0x55da087c86b0 2023-11-16T16:53:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.188+0000 7f67ecb9b700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f67d00a12a0 0x7f67d00a3750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f67dc096140 tx=0x7f67dc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.188+0000 7f680273cb80 1 -- 172.21.15.93:0/4056262904 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da087c86b0 msgr2=0x55da0896c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.188+0000 7f680273cb80 1 --2- 172.21.15.93:0/4056262904 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da087c86b0 0x55da0896c870 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f67d800aa10 tx=0x7f67d800ad20 comp rx=0 tx=0).stop 2023-11-16T16:53:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.189+0000 7f680273cb80 1 -- 172.21.15.93:0/4056262904 shutdown_connections 2023-11-16T16:53:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.189+0000 7f680273cb80 1 --2- 172.21.15.93:0/4056262904 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da087c86b0 0x55da0896c870 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.189+0000 7f680273cb80 1 --2- 172.21.15.93:0/4056262904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da0896db10 0x55da0896df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.189+0000 7f680273cb80 1 --2- 172.21.15.93:0/4056262904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896cdb0 0x55da0896d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.189+0000 7f680273cb80 1 -- 172.21.15.93:0/4056262904 >> 172.21.15.93:0/4056262904 conn(0x55da08961f50 msgr2=0x55da08967100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.190+0000 7f680273cb80 1 -- 172.21.15.93:0/4056262904 shutdown_connections 2023-11-16T16:53:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.190+0000 7f680273cb80 1 -- 172.21.15.93:0/4056262904 wait complete. 2023-11-16T16:53:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.191+0000 7f680273cb80 1 -- start start 2023-11-16T16:53:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.191+0000 7f680273cb80 1 Processor -- start 2023-11-16T16:53:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.192+0000 7f680273cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da087c86b0 0x55da08975ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.192+0000 7f680273cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896cdb0 0x55da08976000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.192+0000 7f680273cb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896db10 0x55da0897d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.192+0000 7f680273cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da0885f880 con 0x55da0896cdb0 2023-11-16T16:53:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.192+0000 7f680273cb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da0885f520 con 0x55da0896db10 2023-11-16T16:53:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.192+0000 7f680273cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da087418c0 con 0x55da087c86b0 2023-11-16T16:53:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.192+0000 7f680273cb80 10 client.0 _ll_register_callbacks cb 0x55da0887d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:53:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.194+0000 7f67e7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896cdb0 0x55da08976000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.194+0000 7f67ed39c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896db10 0x55da0897d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.194+0000 7f67ed39c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da087c86b0 0x55da08975ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.194+0000 7f67ed39c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da087c86b0 0x55da08975ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:49790/0 (socket says 172.21.15.93:49790) 2023-11-16T16:53:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.194+0000 7f67ed39c700 1 -- 172.21.15.93:0/2615345535 learned_addr learned my addr 172.21.15.93:0/2615345535 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.195+0000 7f67ed39c700 1 -- 172.21.15.93:0/2615345535 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896db10 msgr2=0x55da0897d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.195+0000 7f67ed39c700 1 --2- 172.21.15.93:0/2615345535 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896db10 0x55da0897d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.195+0000 7f67ed39c700 1 -- 172.21.15.93:0/2615345535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896cdb0 msgr2=0x55da08976000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.195+0000 7f67ed39c700 1 --2- 172.21.15.93:0/2615345535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896cdb0 0x55da08976000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.195+0000 7f67ed39c700 1 -- 172.21.15.93:0/2615345535 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d800a640 con 0x55da087c86b0 2023-11-16T16:53:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.199+0000 7f67ed39c700 1 --2- 172.21.15.93:0/2615345535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da087c86b0 0x55da08975ac0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f67d800a9e0 tx=0x7f67d8011be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95143b28f2217950 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.199+0000 7f67c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-11-16T16:53:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.200+0000 7f67c7fff700 1 -- 172.21.15.93:0/2615345535 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d800cea0 con 0x55da087c86b0 2023-11-16T16:53:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.200+0000 7f67c7fff700 1 -- 172.21.15.93:0/2615345535 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f67d80106c0 con 0x55da087c86b0 2023-11-16T16:53:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.200+0000 7f680273cb80 1 -- 172.21.15.93:0/2615345535 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da0895f710 con 0x55da087c86b0 2023-11-16T16:53:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.200+0000 7f67c7fff700 1 -- 172.21.15.93:0/2615345535 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67d80108a0 con 0x55da087c86b0 2023-11-16T16:53:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.200+0000 7f680273cb80 10 client.25651 fetch_fsmap learned FSMap version 28 2023-11-16T16:53:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.200+0000 7f680273cb80 1 -- 172.21.15.93:0/2615345535 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55da08884130 con 0x55da087c86b0 2023-11-16T16:53:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.201+0000 7f67c7fff700 1 -- 172.21.15.93:0/2615345535 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55da0895f710 con 0x55da087c86b0 2023-11-16T16:53:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.201+0000 7f67c7fff700 1 -- 172.21.15.93:0/2615345535 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f67d8029870 con 0x55da087c86b0 2023-11-16T16:53:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.201+0000 7f680273cb80 10 client.25651 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:53:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.201+0000 7f680273cb80 10 client.25651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:53:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.201+0000 7f680273cb80 20 client.25651 populate_metadata read hostname 'smithi093' 2023-11-16T16:53:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.201+0000 7f680273cb80 1 --2- 172.21.15.93:0/2615345535 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55da08992770 0x55da08994b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.201+0000 7f680273cb80 4 client.25651 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:53:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.201+0000 7f680273cb80 1 -- 172.21.15.93:0/2615345535 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da08880040 con 0x55da08992770 2023-11-16T16:53:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.203+0000 7f67e7fff700 1 --2- 172.21.15.93:0/2615345535 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55da08992770 0x55da08994b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.203+0000 7f67e7fff700 1 --2- 172.21.15.93:0/2615345535 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55da08992770 0x55da08994b80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933551e7565585f6 server_cookie=66b83566f494d662 in_seq=0 out_seq=0 2023-11-16T16:53:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.204+0000 7f67c7fff700 10 client.25651 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.205+0000 7f67c7fff700 1 -- 172.21.15.93:0/2615345535 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55da08880040 con 0x55da08992770 2023-11-16T16:53:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.205+0000 7f67c7fff700 10 client.25651 handle_command_reply: tid=0 2023-11-16T16:53:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.205+0000 7f680273cb80 1 client.25651 shutdown 2023-11-16T16:53:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.207+0000 7f680273cb80 1 -- 172.21.15.93:0/2615345535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da087c86b0 msgr2=0x55da08975ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.207+0000 7f680273cb80 1 --2- 172.21.15.93:0/2615345535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da087c86b0 0x55da08975ac0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f67d800a9e0 tx=0x7f67d8011be0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.209+0000 7f680273cb80 1 -- 172.21.15.93:0/2615345535 shutdown_connections 2023-11-16T16:53:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.209+0000 7f680273cb80 1 --2- 172.21.15.93:0/2615345535 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55da08992770 0x55da08994b80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.209+0000 7f680273cb80 1 --2- 172.21.15.93:0/2615345535 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896db10 0x55da0897d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.209+0000 7f680273cb80 1 --2- 172.21.15.93:0/2615345535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da087c86b0 0x55da08975ac0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.209+0000 7f680273cb80 1 --2- 172.21.15.93:0/2615345535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896cdb0 0x55da08976000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.209+0000 7f67c7fff700 0 client.25651 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.209+0000 7f680273cb80 1 -- 172.21.15.93:0/2615345535 >> 172.21.15.93:0/2615345535 conn(0x55da08961f50 msgr2=0x55da089670d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.213+0000 7f680273cb80 1 -- 172.21.15.93:0/2615345535 shutdown_connections 2023-11-16T16:53:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.214+0000 7f680273cb80 1 -- 172.21.15.93:0/2615345535 wait complete. 2023-11-16T16:53:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.215+0000 7f680273cb80 20 client.25651 trim_cache size 0 max 16384 2023-11-16T16:53:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.252+0000 7f680273cb80 1 Processor -- start 2023-11-16T16:53:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.253+0000 7f680273cb80 1 -- start start 2023-11-16T16:53:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.253+0000 7f680273cb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da087c86b0 0x55da089994a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.253+0000 7f680273cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da0896cdb0 0x55da0897d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.254+0000 7f67ed39c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da087c86b0 0x55da089994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.254+0000 7f680273cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896db10 0x55da089707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.254+0000 7f680273cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da08968bb0 con 0x55da0896db10 2023-11-16T16:53:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.254+0000 7f680273cb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da0896d180 con 0x55da087c86b0 2023-11-16T16:53:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.254+0000 7f680273cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da089951e0 con 0x55da0896cdb0 2023-11-16T16:53:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.254+0000 7f67e7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da0896cdb0 0x55da0897d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.254+0000 7f67e7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da0896cdb0 0x55da0897d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:49798/0 (socket says 172.21.15.93:49798) 2023-11-16T16:53:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.254+0000 7f67e7fff700 1 -- 172.21.15.93:0/2868564206 learned_addr learned my addr 172.21.15.93:0/2868564206 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.255+0000 7f67e7fff700 1 -- 172.21.15.93:0/2868564206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da087c86b0 msgr2=0x55da089994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.255+0000 7f67e7fff700 1 --2- 172.21.15.93:0/2868564206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da087c86b0 0x55da089994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.255+0000 7f67e7fff700 1 -- 172.21.15.93:0/2868564206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896db10 msgr2=0x55da089707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.255+0000 7f67e7fff700 1 --2- 172.21.15.93:0/2868564206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896db10 0x55da089707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.255+0000 7f67e7fff700 1 -- 172.21.15.93:0/2868564206 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da08883e80 con 0x55da0896cdb0 2023-11-16T16:53:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.256+0000 7f67e7fff700 1 --2- 172.21.15.93:0/2868564206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da0896cdb0 0x55da0897d000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55da08995420 tx=0x7f67e00096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2927a99f31441279 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.258+0000 7f67cf7fe700 1 -- 172.21.15.93:0/2868564206 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e001f070 con 0x55da0896cdb0 2023-11-16T16:53:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.258+0000 7f67cf7fe700 1 -- 172.21.15.93:0/2868564206 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67e000ad80 con 0x55da0896cdb0 2023-11-16T16:53:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.258+0000 7f67cf7fe700 1 -- 172.21.15.93:0/2868564206 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0010040 con 0x55da0896cdb0 2023-11-16T16:53:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.260+0000 7f680273cb80 1 -- 172.21.15.93:0/2868564206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da0896cdb0 msgr2=0x55da0897d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.260+0000 7f680273cb80 1 --2- 172.21.15.93:0/2868564206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da0896cdb0 0x55da0897d000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55da08995420 tx=0x7f67e00096d0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.261+0000 7f680273cb80 1 -- 172.21.15.93:0/2868564206 shutdown_connections 2023-11-16T16:53:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.261+0000 7f680273cb80 1 --2- 172.21.15.93:0/2868564206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896db10 0x55da089707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.261+0000 7f680273cb80 1 --2- 172.21.15.93:0/2868564206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da087c86b0 0x55da089994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.261+0000 7f680273cb80 1 --2- 172.21.15.93:0/2868564206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da0896cdb0 0x55da0897d000 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.261+0000 7f680273cb80 1 -- 172.21.15.93:0/2868564206 >> 172.21.15.93:0/2868564206 conn(0x55da08961f50 msgr2=0x55da089670d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.261+0000 7f680273cb80 1 -- 172.21.15.93:0/2868564206 shutdown_connections 2023-11-16T16:53:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.261+0000 7f680273cb80 1 -- 172.21.15.93:0/2868564206 wait complete. 2023-11-16T16:53:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.262+0000 7f680273cb80 1 -- start start 2023-11-16T16:53:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.262+0000 7f680273cb80 1 Processor -- start 2023-11-16T16:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.266+0000 7f680273cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da087c86b0 0x55da0897c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.266+0000 7f680273cb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896cdb0 0x55da0897cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.266+0000 7f680273cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896db10 0x55da089792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.266+0000 7f680273cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da08979830 con 0x55da0896db10 2023-11-16T16:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.266+0000 7f680273cb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da089799a0 con 0x55da0896cdb0 2023-11-16T16:53:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.266+0000 7f680273cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da08979b60 con 0x55da087c86b0 2023-11-16T16:53:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.266+0000 7f680273cb80 10 client.0 _ll_register_callbacks cb 0x7f67d800cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:53:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.267+0000 7f67ed39c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da087c86b0 0x55da0897c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.267+0000 7f67ed39c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896db10 0x55da089792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.267+0000 7f67ed39c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896db10 0x55da089792b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36400/0 (socket says 172.21.15.93:36400) 2023-11-16T16:53:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.267+0000 7f67ed39c700 1 -- 172.21.15.93:0/2960400998 learned_addr learned my addr 172.21.15.93:0/2960400998 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.267+0000 7f67e7fff700 1 --2- 172.21.15.93:0/2960400998 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896cdb0 0x55da0897cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.268+0000 7f67e7fff700 1 -- 172.21.15.93:0/2960400998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da087c86b0 msgr2=0x55da0897c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.268+0000 7f67e7fff700 1 --2- 172.21.15.93:0/2960400998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da087c86b0 0x55da0897c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.268+0000 7f67e7fff700 1 -- 172.21.15.93:0/2960400998 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896db10 msgr2=0x55da089792b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.268+0000 7f67e7fff700 1 --2- 172.21.15.93:0/2960400998 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896db10 0x55da089792b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.268+0000 7f67e7fff700 1 -- 172.21.15.93:0/2960400998 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e000a690 con 0x55da0896cdb0 2023-11-16T16:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.269+0000 7f67e7fff700 1 --2- 172.21.15.93:0/2960400998 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896cdb0 0x55da0897cd50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67e0004ba0 tx=0x7f67e0023760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7031a64b56746fa7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.269+0000 7f67c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:53:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.270+0000 7f67c7fff700 1 -- 172.21.15.93:0/2960400998 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0010070 con 0x55da0896cdb0 2023-11-16T16:53:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.271+0000 7f67c7fff700 1 -- 172.21.15.93:0/2960400998 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f67e002c400 con 0x55da0896cdb0 2023-11-16T16:53:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.271+0000 7f680273cb80 1 -- 172.21.15.93:0/2960400998 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67d800cbf0 con 0x55da0896cdb0 2023-11-16T16:53:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.272+0000 7f67c7fff700 1 -- 172.21.15.93:0/2960400998 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67e001d070 con 0x55da0896cdb0 2023-11-16T16:53:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.272+0000 7f680273cb80 10 client.25661 fetch_fsmap learned FSMap version 28 2023-11-16T16:53:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.272+0000 7f680273cb80 1 -- 172.21.15.93:0/2960400998 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55da086d4c00 con 0x55da0896cdb0 2023-11-16T16:53:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.273+0000 7f67c7fff700 1 -- 172.21.15.93:0/2960400998 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d800cbf0 con 0x55da0896cdb0 2023-11-16T16:53:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.273+0000 7f67c7fff700 1 -- 172.21.15.93:0/2960400998 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f67e002a020 con 0x55da0896cdb0 2023-11-16T16:53:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.273+0000 7f680273cb80 10 client.25661 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:53:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.273+0000 7f680273cb80 10 client.25661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:53:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.273+0000 7f680273cb80 20 client.25661 populate_metadata read hostname 'smithi093' 2023-11-16T16:53:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.273+0000 7f680273cb80 1 --2- 172.21.15.93:0/2960400998 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55da089e78c0 0x55da089e9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.273+0000 7f680273cb80 4 client.25661 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:53:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.273+0000 7f680273cb80 1 -- 172.21.15.93:0/2960400998 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55da0888c600 con 0x55da089e78c0 2023-11-16T16:53:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.274+0000 7f67ed39c700 1 --2- 172.21.15.93:0/2960400998 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55da089e78c0 0x55da089e9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.276+0000 7f67c7fff700 10 client.25661 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.276+0000 7f67ed39c700 1 --2- 172.21.15.93:0/2960400998 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55da089e78c0 0x55da089e9cd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e18873a806ef1e server_cookie=4da4cc39462589b7 in_seq=0 out_seq=0 2023-11-16T16:53:04.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:04 smithi159 ceph-mon[131528]: pgmap v478: 129 pgs: 129 active+clean; 221 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 1.24k op/s 2023-11-16T16:53:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.795+0000 7f67c7fff700 1 -- 172.21.15.93:0/2960400998 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55da0888c600 con 0x55da089e78c0 2023-11-16T16:53:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.795+0000 7f67c7fff700 10 client.25661 handle_command_reply: tid=0 2023-11-16T16:53:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.795+0000 7f680273cb80 1 client.25661 shutdown 2023-11-16T16:53:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.796+0000 7f680273cb80 1 -- 172.21.15.93:0/2960400998 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896cdb0 msgr2=0x55da0897cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.796+0000 7f680273cb80 1 --2- 172.21.15.93:0/2960400998 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896cdb0 0x55da0897cd50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67e0004ba0 tx=0x7f67e0023760 comp rx=0 tx=0).stop 2023-11-16T16:53:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.797+0000 7f680273cb80 1 -- 172.21.15.93:0/2960400998 shutdown_connections 2023-11-16T16:53:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.797+0000 7f680273cb80 1 --2- 172.21.15.93:0/2960400998 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55da089e78c0 0x55da089e9cd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.797+0000 7f680273cb80 1 --2- 172.21.15.93:0/2960400998 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55da0896db10 0x55da089792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.797+0000 7f680273cb80 1 --2- 172.21.15.93:0/2960400998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da087c86b0 0x55da0897c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.797+0000 7f680273cb80 1 --2- 172.21.15.93:0/2960400998 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da0896cdb0 0x55da0897cd50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.797+0000 7f680273cb80 1 -- 172.21.15.93:0/2960400998 >> 172.21.15.93:0/2960400998 conn(0x55da08961f50 msgr2=0x55da08975870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.797+0000 7f67c7fff700 0 client.25661 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.797+0000 7f680273cb80 1 -- 172.21.15.93:0/2960400998 shutdown_connections 2023-11-16T16:53:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.797+0000 7f680273cb80 1 -- 172.21.15.93:0/2960400998 wait complete. 2023-11-16T16:53:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.798+0000 7f680273cb80 20 client.25661 trim_cache size 0 max 16384 2023-11-16T16:53:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:53:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:53:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:53:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:53:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.799+0000 7f67eedff700 1 -- 172.21.15.93:0/2361727758 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f67d00a12a0 msgr2=0x7f67d00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.799+0000 7f67eedff700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f67d00a12a0 0x7f67d00a3750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f67dc096140 tx=0x7f67dc000bc0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.799+0000 7f67eedff700 1 -- 172.21.15.93:0/2361727758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e8107e60 msgr2=0x7f67e81a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.799+0000 7f67eedff700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e8107e60 0x7f67e81a5280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00a040 tx=0x7f67dc004300 comp rx=0 tx=0).stop 2023-11-16T16:53:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.799+0000 7f67eedff700 1 -- 172.21.15.93:0/2361727758 shutdown_connections 2023-11-16T16:53:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.799+0000 7f67eedff700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f67d00a12a0 0x7f67d00a3750 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.799+0000 7f67eedff700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67e81aa820 0x7f67e81acc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.799+0000 7f67eedff700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e8107e60 0x7f67e81a5280 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.799+0000 7f67eedff700 1 --2- 172.21.15.93:0/2361727758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67e81087f0 0x7f67e81a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.799+0000 7f67eedff700 1 -- 172.21.15.93:0/2361727758 >> 172.21.15.93:0/2361727758 conn(0x7f67e806d4a0 msgr2=0x7f67e81150d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.799+0000 7f67eedff700 1 -- 172.21.15.93:0/2361727758 shutdown_connections 2023-11-16T16:53:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:04.799+0000 7f67eedff700 1 -- 172.21.15.93:0/2361727758 wait complete. 2023-11-16T16:53:04.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:04 smithi125 ceph-mon[131076]: pgmap v478: 129 pgs: 129 active+clean; 221 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 1.24k op/s 2023-11-16T16:53:04.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:04 smithi093 ceph-mon[97863]: pgmap v478: 129 pgs: 129 active+clean; 221 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 1.24k op/s 2023-11-16T16:53:05.466 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:53:05 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c[157316]: ignoring --setuser ceph since I am not root 2023-11-16T16:53:05.467 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:53:05 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c[157316]: ignoring --setgroup ceph since I am not root 2023-11-16T16:53:05.467 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:53:05 smithi159 ceph-mds[157340]: -- 172.21.15.159:0/2633148696 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562b921cf520 con 0x562b921a5000 2023-11-16T16:53:05.467 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:53:05 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c[157316]: starting mds.c at 2023-11-16T16:53:05.467 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:05 smithi159 ceph-mon[131528]: daemon mds.c restarted 2023-11-16T16:53:05.467 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:05 smithi159 ceph-mon[131528]: osdmap e93: 12 total, 12 up, 12 in 2023-11-16T16:53:05.467 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:05 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.159:6836/3209267796,v1:172.21.15.159:6837/3209267796] up:boot 2023-11-16T16:53:05.467 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:05 smithi159 ceph-mon[131528]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:53:05.467 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:05 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:53:05.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:05 smithi125 ceph-mon[131076]: daemon mds.c restarted 2023-11-16T16:53:05.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:05 smithi125 ceph-mon[131076]: osdmap e93: 12 total, 12 up, 12 in 2023-11-16T16:53:05.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:05 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.159:6836/3209267796,v1:172.21.15.159:6837/3209267796] up:boot 2023-11-16T16:53:05.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:05 smithi125 ceph-mon[131076]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:53:05.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:05 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:53:05.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:05 smithi093 ceph-mon[97863]: daemon mds.c restarted 2023-11-16T16:53:05.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:05 smithi093 ceph-mon[97863]: osdmap e93: 12 total, 12 up, 12 in 2023-11-16T16:53:05.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:05 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.159:6836/3209267796,v1:172.21.15.159:6837/3209267796] up:boot 2023-11-16T16:53:05.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:05 smithi093 ceph-mon[97863]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:53:05.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:05 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:53:06.734 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:06 smithi159 ceph-mon[131528]: pgmap v479: 129 pgs: 129 active+clean; 232 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 839 op/s 2023-11-16T16:53:06.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:06 smithi125 ceph-mon[131076]: pgmap v479: 129 pgs: 129 active+clean; 232 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 839 op/s 2023-11-16T16:53:06.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:06 smithi093 ceph-mon[97863]: pgmap v479: 129 pgs: 129 active+clean; 232 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 839 op/s 2023-11-16T16:53:08.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:08 smithi159 ceph-mon[131528]: pgmap v481: 129 pgs: 129 active+clean; 231 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 715 op/s 2023-11-16T16:53:08.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:08 smithi125 ceph-mon[131076]: pgmap v481: 129 pgs: 129 active+clean; 231 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 715 op/s 2023-11-16T16:53:08.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:08 smithi093 ceph-mon[97863]: pgmap v481: 129 pgs: 129 active+clean; 231 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 715 op/s 2023-11-16T16:53:10.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:10 smithi125 ceph-mon[131076]: pgmap v482: 129 pgs: 129 active+clean; 235 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 782 op/s 2023-11-16T16:53:10.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:10 smithi093 ceph-mon[97863]: pgmap v482: 129 pgs: 129 active+clean; 235 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 782 op/s 2023-11-16T16:53:10.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:10 smithi159 ceph-mon[131528]: pgmap v482: 129 pgs: 129 active+clean; 235 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 782 op/s 2023-11-16T16:53:12.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:12 smithi125 ceph-mon[131076]: pgmap v483: 129 pgs: 129 active+clean; 230 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 837 op/s 2023-11-16T16:53:12.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:12 smithi093 ceph-mon[97863]: pgmap v483: 129 pgs: 129 active+clean; 230 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 837 op/s 2023-11-16T16:53:12.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:12 smithi159 ceph-mon[131528]: pgmap v483: 129 pgs: 129 active+clean; 230 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 837 op/s 2023-11-16T16:53:14.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:14 smithi125 ceph-mon[131076]: pgmap v484: 129 pgs: 129 active+clean; 240 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 842 op/s 2023-11-16T16:53:14.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:14 smithi093 ceph-mon[97863]: pgmap v484: 129 pgs: 129 active+clean; 240 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 842 op/s 2023-11-16T16:53:14.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:14 smithi159 ceph-mon[131528]: pgmap v484: 129 pgs: 129 active+clean; 240 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 842 op/s 2023-11-16T16:53:16.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:16 smithi125 ceph-mon[131076]: pgmap v485: 129 pgs: 129 active+clean; 246 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 820 op/s 2023-11-16T16:53:16.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:16 smithi093 ceph-mon[97863]: pgmap v485: 129 pgs: 129 active+clean; 246 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 820 op/s 2023-11-16T16:53:16.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:16 smithi159 ceph-mon[131528]: pgmap v485: 129 pgs: 129 active+clean; 246 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 820 op/s 2023-11-16T16:53:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.058+0000 7f65157ff700 1 -- 172.21.15.93:0/420105780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6510106d10 msgr2=0x7f6510107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.058+0000 7f65157ff700 1 --2- 172.21.15.93:0/420105780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6510106d10 0x7f6510107120 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009990 tx=0x7f64f8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.059+0000 7f65157ff700 1 -- 172.21.15.93:0/420105780 shutdown_connections 2023-11-16T16:53:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.059+0000 7f65157ff700 1 --2- 172.21.15.93:0/420105780 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6510108400 0x7f651010ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.059+0000 7f65157ff700 1 --2- 172.21.15.93:0/420105780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6510106d10 0x7f6510107120 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.059+0000 7f65157ff700 1 --2- 172.21.15.93:0/420105780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6510107a70 0x7f6510107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.059+0000 7f65157ff700 1 -- 172.21.15.93:0/420105780 >> 172.21.15.93:0/420105780 conn(0x7f65100758b0 msgr2=0x7f6510077d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.059+0000 7f65157ff700 1 -- 172.21.15.93:0/420105780 shutdown_connections 2023-11-16T16:53:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.059+0000 7f65157ff700 1 -- 172.21.15.93:0/420105780 wait complete. 2023-11-16T16:53:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.061+0000 7f65157ff700 1 Processor -- start 2023-11-16T16:53:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.061+0000 7f65157ff700 1 -- start start 2023-11-16T16:53:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.061+0000 7f65157ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6510106d10 0x7f651019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.061+0000 7f65157ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6510107a70 0x7f651019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.061+0000 7f65157ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6510108400 0x7f65101a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.061+0000 7f65157ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65101a40c0 con 0x7f6510107a70 2023-11-16T16:53:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.061+0000 7f65157ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65101a4200 con 0x7f6510108400 2023-11-16T16:53:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.061+0000 7f65157ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65101a4390 con 0x7f6510106d10 2023-11-16T16:53:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.062+0000 7f650effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6510106d10 0x7f651019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.062+0000 7f650f7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6510108400 0x7f65101a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.062+0000 7f650f7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6510108400 0x7f65101a3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:47912/0 (socket says 172.21.15.93:47912) 2023-11-16T16:53:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.062+0000 7f650f7fe700 1 -- 172.21.15.93:0/3229412021 learned_addr learned my addr 172.21.15.93:0/3229412021 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.062+0000 7f650e7fc700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6510107a70 0x7f651019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.063+0000 7f650effd700 1 -- 172.21.15.93:0/3229412021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6510108400 msgr2=0x7f65101a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.063+0000 7f650effd700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6510108400 0x7f65101a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.063+0000 7f650effd700 1 -- 172.21.15.93:0/3229412021 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6510107a70 msgr2=0x7f651019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.063+0000 7f650effd700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6510107a70 0x7f651019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.063+0000 7f650effd700 1 -- 172.21.15.93:0/3229412021 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f80095f0 con 0x7f6510106d10 2023-11-16T16:53:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.063+0000 7f650f7fe700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6510108400 0x7f65101a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:53:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.063+0000 7f650effd700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6510106d10 0x7f651019c540 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f64f80098e0 tx=0x7f64f80057b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.064+0000 7f6507fff700 1 -- 172.21.15.93:0/3229412021 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f801d070 con 0x7f6510106d10 2023-11-16T16:53:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.064+0000 7f6507fff700 1 -- 172.21.15.93:0/3229412021 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64f8022470 con 0x7f6510106d10 2023-11-16T16:53:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.064+0000 7f6507fff700 1 -- 172.21.15.93:0/3229412021 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8017650 con 0x7f6510106d10 2023-11-16T16:53:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.064+0000 7f65157ff700 1 -- 172.21.15.93:0/3229412021 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65101121d0 con 0x7f6510106d10 2023-11-16T16:53:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.064+0000 7f65157ff700 1 -- 172.21.15.93:0/3229412021 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6510112040 con 0x7f6510106d10 2023-11-16T16:53:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.069+0000 7f6507fff700 1 -- 172.21.15.93:0/3229412021 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f64f8022850 con 0x7f6510106d10 2023-11-16T16:53:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.071+0000 7f6507fff700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f64fc0a12a0 0x7f64fc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.071+0000 7f6507fff700 1 -- 172.21.15.93:0/3229412021 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f64f80cdf00 con 0x7f6510106d10 2023-11-16T16:53:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.071+0000 7f65290e7b80 1 Processor -- start 2023-11-16T16:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.071+0000 7f65290e7b80 1 -- start start 2023-11-16T16:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.072+0000 7f65290e7b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 0x557b30def870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.072+0000 7f65290e7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 0x557b30df05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.072+0000 7f65290e7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30df0b10 0x557b30df0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.072+0000 7f65290e7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30cf5000 con 0x557b30defdb0 2023-11-16T16:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.072+0000 7f65290e7b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30d47840 con 0x557b30c4b6b0 2023-11-16T16:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.072+0000 7f65290e7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30d40cd0 con 0x557b30df0b10 2023-11-16T16:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.072+0000 7f650e7fc700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f64fc0a12a0 0x7f64fc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.072+0000 7f650f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30df0b10 0x557b30df0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.072+0000 7f650f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30df0b10 0x557b30df0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:40716/0 (socket says 172.21.15.93:40716) 2023-11-16T16:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.072+0000 7f650f7fe700 1 -- 172.21.15.93:0/891712857 learned_addr learned my addr 172.21.15.93:0/891712857 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.072+0000 7f650effd700 1 --2- 172.21.15.93:0/891712857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 0x557b30def870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.073+0000 7f650effd700 1 -- 172.21.15.93:0/891712857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30df0b10 msgr2=0x557b30df0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.073+0000 7f650effd700 1 --2- 172.21.15.93:0/891712857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30df0b10 0x557b30df0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.073+0000 7f650effd700 1 -- 172.21.15.93:0/891712857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 msgr2=0x557b30df05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.073+0000 7f650effd700 1 --2- 172.21.15.93:0/891712857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 0x557b30df05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.073+0000 7f650effd700 1 -- 172.21.15.93:0/891712857 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b30bc25d0 con 0x557b30c4b6b0 2023-11-16T16:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.073+0000 7f650effd700 1 --2- 172.21.15.93:0/891712857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 0x557b30def870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f64f80cb680 tx=0x7f64f8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1efa3a11000bdd57 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.073+0000 7f65057fa700 1 -- 172.21.15.93:0/891712857 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f809a930 con 0x557b30c4b6b0 2023-11-16T16:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.074+0000 7f65057fa700 1 -- 172.21.15.93:0/891712857 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64f809afd0 con 0x557b30c4b6b0 2023-11-16T16:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.074+0000 7f65057fa700 1 -- 172.21.15.93:0/891712857 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8099490 con 0x557b30c4b6b0 2023-11-16T16:53:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.075+0000 7f65290e7b80 1 -- 172.21.15.93:0/891712857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 msgr2=0x557b30def870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.075+0000 7f650e7fc700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f64fc0a12a0 0x7f64fc0a3750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6500007fd0 tx=0x7f6500007e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.075+0000 7f65290e7b80 1 --2- 172.21.15.93:0/891712857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 0x557b30def870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f64f80cb680 tx=0x7f64f8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.075+0000 7f65290e7b80 1 -- 172.21.15.93:0/891712857 shutdown_connections 2023-11-16T16:53:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.075+0000 7f65290e7b80 1 --2- 172.21.15.93:0/891712857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30df0b10 0x557b30df0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.075+0000 7f65290e7b80 1 --2- 172.21.15.93:0/891712857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 0x557b30def870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.075+0000 7f65290e7b80 1 --2- 172.21.15.93:0/891712857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 0x557b30df05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.075+0000 7f65290e7b80 1 -- 172.21.15.93:0/891712857 >> 172.21.15.93:0/891712857 conn(0x557b30de4f50 msgr2=0x557b30dea100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.075+0000 7f65290e7b80 1 -- 172.21.15.93:0/891712857 shutdown_connections 2023-11-16T16:53:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.075+0000 7f65290e7b80 1 -- 172.21.15.93:0/891712857 wait complete. 2023-11-16T16:53:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.076+0000 7f65290e7b80 1 -- start start 2023-11-16T16:53:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.076+0000 7f65290e7b80 1 Processor -- start 2023-11-16T16:53:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f65290e7b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 0x557b30df8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f65290e7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 0x557b30df8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f65290e7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30df0b10 0x557b30dffe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f65290e7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30ce2880 con 0x557b30defdb0 2023-11-16T16:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f65290e7b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30ce2520 con 0x557b30c4b6b0 2023-11-16T16:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f65290e7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30bc48c0 con 0x557b30df0b10 2023-11-16T16:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f65290e7b80 10 client.0 _ll_register_callbacks cb 0x557b30d00720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f650effd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 0x557b30df8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f650effd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 0x557b30df8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:47928/0 (socket says 172.21.15.93:47928) 2023-11-16T16:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f650effd700 1 -- 172.21.15.93:0/659878881 learned_addr learned my addr 172.21.15.93:0/659878881 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f650e7fc700 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 0x557b30df8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f650f7fe700 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30df0b10 0x557b30dffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f650effd700 1 -- 172.21.15.93:0/659878881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30df0b10 msgr2=0x557b30dffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f650effd700 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30df0b10 0x557b30dffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f650effd700 1 -- 172.21.15.93:0/659878881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 msgr2=0x557b30df8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f650effd700 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 0x557b30df8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.078+0000 7f650effd700 1 -- 172.21.15.93:0/659878881 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6510112040 con 0x557b30c4b6b0 2023-11-16T16:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.079+0000 7f650f7fe700 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30df0b10 0x557b30dffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.079+0000 7f650effd700 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 0x557b30df8910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64f8005cd0 tx=0x7f64f8005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d621aa069fbfe583 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.079+0000 7f64f67fc700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.079+0000 7f64f67fc700 1 -- 172.21.15.93:0/659878881 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8099470 con 0x557b30c4b6b0 2023-11-16T16:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.079+0000 7f64f67fc700 1 -- 172.21.15.93:0/659878881 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f64f80a3390 con 0x557b30c4b6b0 2023-11-16T16:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.079+0000 7f65290e7b80 1 -- 172.21.15.93:0/659878881 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b30de2710 con 0x557b30c4b6b0 2023-11-16T16:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.080+0000 7f64f67fc700 1 -- 172.21.15.93:0/659878881 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64f809a930 con 0x557b30c4b6b0 2023-11-16T16:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.080+0000 7f65290e7b80 10 client.25676 fetch_fsmap learned FSMap version 29 2023-11-16T16:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.080+0000 7f65290e7b80 1 -- 172.21.15.93:0/659878881 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557b30d07130 con 0x557b30c4b6b0 2023-11-16T16:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.080+0000 7f64f67fc700 1 -- 172.21.15.93:0/659878881 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64f80c9070 con 0x557b30c4b6b0 2023-11-16T16:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.080+0000 7f64f67fc700 1 -- 172.21.15.93:0/659878881 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 29) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f64f8096ee0 con 0x557b30c4b6b0 2023-11-16T16:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.081+0000 7f65290e7b80 10 client.25676 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.081+0000 7f65290e7b80 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.081+0000 7f65290e7b80 20 client.25676 populate_metadata read hostname 'smithi093' 2023-11-16T16:53:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.081+0000 7f65290e7b80 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557b30e041f0 0x557b30e06600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.081+0000 7f65290e7b80 4 client.25676 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.081+0000 7f65290e7b80 1 -- 172.21.15.93:0/659878881 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b30d03040 con 0x557b30e041f0 2023-11-16T16:53:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.081+0000 7f650e7fc700 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557b30e041f0 0x557b30e06600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.082+0000 7f650e7fc700 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557b30e041f0 0x557b30e06600 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eed2b8a2dd8920a server_cookie=a2c0c59fb0dc72bd in_seq=0 out_seq=0 2023-11-16T16:53:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.082+0000 7f64f67fc700 10 client.25676 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.083+0000 7f64f67fc700 1 -- 172.21.15.93:0/659878881 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557b30d03040 con 0x557b30e041f0 2023-11-16T16:53:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.083+0000 7f64f67fc700 10 client.25676 handle_command_reply: tid=0 2023-11-16T16:53:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.083+0000 7f65290e7b80 1 client.25676 shutdown 2023-11-16T16:53:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.083+0000 7f65290e7b80 1 -- 172.21.15.93:0/659878881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 msgr2=0x557b30df8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.083+0000 7f65290e7b80 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 0x557b30df8910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64f8005cd0 tx=0x7f64f8005fb0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.083+0000 7f65290e7b80 1 -- 172.21.15.93:0/659878881 shutdown_connections 2023-11-16T16:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.084+0000 7f65290e7b80 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557b30e041f0 0x557b30e06600 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.084+0000 7f64f67fc700 0 client.25676 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.084+0000 7f65290e7b80 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30df0b10 0x557b30dffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.084+0000 7f65290e7b80 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30c4b6b0 0x557b30df8910 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.084+0000 7f65290e7b80 1 --2- 172.21.15.93:0/659878881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 0x557b30df8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.084+0000 7f65290e7b80 1 -- 172.21.15.93:0/659878881 >> 172.21.15.93:0/659878881 conn(0x557b30de4f50 msgr2=0x557b30de97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.084+0000 7f65290e7b80 1 -- 172.21.15.93:0/659878881 shutdown_connections 2023-11-16T16:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.084+0000 7f65290e7b80 1 -- 172.21.15.93:0/659878881 wait complete. 2023-11-16T16:53:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.084+0000 7f65290e7b80 20 client.25676 trim_cache size 0 max 16384 2023-11-16T16:53:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.109+0000 7f65290e7b80 1 Processor -- start 2023-11-16T16:53:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.109+0000 7f65290e7b80 1 -- start start 2023-11-16T16:53:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.110+0000 7f65290e7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30c4b6b0 0x557b30df8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.110+0000 7f65290e7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30defdb0 0x557b30dffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.110+0000 7f65290e7b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 0x557b30df6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.110+0000 7f65290e7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30de4950 con 0x557b30c4b6b0 2023-11-16T16:53:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.110+0000 7f65290e7b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30d03040 con 0x557b30df0b10 2023-11-16T16:53:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.110+0000 7f65290e7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30df7dc0 con 0x557b30defdb0 2023-11-16T16:53:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.110+0000 7f650f7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 0x557b30df6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.110+0000 7f650f7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 0x557b30df6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:47932/0 (socket says 172.21.15.93:47932) 2023-11-16T16:53:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.110+0000 7f650f7fe700 1 -- 172.21.15.93:0/426201933 learned_addr learned my addr 172.21.15.93:0/426201933 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.110+0000 7f650e7fc700 1 --2- 172.21.15.93:0/426201933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30defdb0 0x557b30dffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.110+0000 7f650effd700 1 --2- 172.21.15.93:0/426201933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30c4b6b0 0x557b30df8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.111+0000 7f650f7fe700 1 -- 172.21.15.93:0/426201933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30defdb0 msgr2=0x557b30dffe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.111+0000 7f650f7fe700 1 --2- 172.21.15.93:0/426201933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30defdb0 0x557b30dffe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.111+0000 7f650f7fe700 1 -- 172.21.15.93:0/426201933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30c4b6b0 msgr2=0x557b30df8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.111+0000 7f650f7fe700 1 --2- 172.21.15.93:0/426201933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30c4b6b0 0x557b30df8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.111+0000 7f650f7fe700 1 -- 172.21.15.93:0/426201933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b30d06e80 con 0x557b30df0b10 2023-11-16T16:53:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.111+0000 7f650f7fe700 1 --2- 172.21.15.93:0/426201933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 0x557b30df6cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f65080107b0 tx=0x7f6508010ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d5312dddcd87fa server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.112+0000 7f6505ffb700 1 -- 172.21.15.93:0/426201933 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508004a30 con 0x557b30df0b10 2023-11-16T16:53:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.112+0000 7f6505ffb700 1 -- 172.21.15.93:0/426201933 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6508018040 con 0x557b30df0b10 2023-11-16T16:53:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.112+0000 7f6505ffb700 1 -- 172.21.15.93:0/426201933 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650801c770 con 0x557b30df0b10 2023-11-16T16:53:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.112+0000 7f65290e7b80 1 -- 172.21.15.93:0/426201933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 msgr2=0x557b30df6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.112+0000 7f65290e7b80 1 --2- 172.21.15.93:0/426201933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 0x557b30df6cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f65080107b0 tx=0x7f6508010ac0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.113+0000 7f65290e7b80 1 -- 172.21.15.93:0/426201933 shutdown_connections 2023-11-16T16:53:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.113+0000 7f65290e7b80 1 --2- 172.21.15.93:0/426201933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30c4b6b0 0x557b30df8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.113+0000 7f65290e7b80 1 --2- 172.21.15.93:0/426201933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 0x557b30df6cd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.113+0000 7f65290e7b80 1 --2- 172.21.15.93:0/426201933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30defdb0 0x557b30dffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.113+0000 7f65290e7b80 1 -- 172.21.15.93:0/426201933 >> 172.21.15.93:0/426201933 conn(0x557b30de4f50 msgr2=0x557b30df86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.113+0000 7f65290e7b80 1 -- 172.21.15.93:0/426201933 shutdown_connections 2023-11-16T16:53:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.113+0000 7f65290e7b80 1 -- 172.21.15.93:0/426201933 wait complete. 2023-11-16T16:53:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.113+0000 7f65290e7b80 1 -- start start 2023-11-16T16:53:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.114+0000 7f65290e7b80 1 Processor -- start 2023-11-16T16:53:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.114+0000 7f65290e7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30c4b6b0 0x557b30dfbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.114+0000 7f65290e7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 0x557b30dfc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.114+0000 7f65290e7b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 0x557b30dff470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.114+0000 7f65290e7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30dff9b0 con 0x557b30defdb0 2023-11-16T16:53:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.114+0000 7f65290e7b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30dffb20 con 0x557b30df0b10 2023-11-16T16:53:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.114+0000 7f65290e7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30dffce0 con 0x557b30c4b6b0 2023-11-16T16:53:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.114+0000 7f65290e7b80 10 client.0 _ll_register_callbacks cb 0x7f64f8097310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:53:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.115+0000 7f650f7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 0x557b30dff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.115+0000 7f650effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30c4b6b0 0x557b30dfbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.115+0000 7f650f7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 0x557b30dff470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:47946/0 (socket says 172.21.15.93:47946) 2023-11-16T16:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.115+0000 7f650f7fe700 1 -- 172.21.15.93:0/700372702 learned_addr learned my addr 172.21.15.93:0/700372702 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.115+0000 7f650e7fc700 1 --2- 172.21.15.93:0/700372702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 0x557b30dfc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.115+0000 7f650f7fe700 1 -- 172.21.15.93:0/700372702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30c4b6b0 msgr2=0x557b30dfbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.115+0000 7f650f7fe700 1 --2- 172.21.15.93:0/700372702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30c4b6b0 0x557b30dfbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.116+0000 7f650f7fe700 1 -- 172.21.15.93:0/700372702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 msgr2=0x557b30dfc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.116+0000 7f650f7fe700 1 --2- 172.21.15.93:0/700372702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 0x557b30dfc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.116+0000 7f650f7fe700 1 -- 172.21.15.93:0/700372702 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6508010430 con 0x557b30df0b10 2023-11-16T16:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.116+0000 7f650f7fe700 1 --2- 172.21.15.93:0/700372702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 0x557b30dff470 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6508004bd0 tx=0x7f6508005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593754e49a70d9a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.116+0000 7f64f67fc700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:53:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.116+0000 7f64f67fc700 1 -- 172.21.15.93:0/700372702 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508005630 con 0x557b30df0b10 2023-11-16T16:53:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.116+0000 7f64f67fc700 1 -- 172.21.15.93:0/700372702 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f650801c4f0 con 0x557b30df0b10 2023-11-16T16:53:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.116+0000 7f65290e7b80 1 -- 172.21.15.93:0/700372702 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64f809afd0 con 0x557b30df0b10 2023-11-16T16:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.117+0000 7f64f67fc700 1 -- 172.21.15.93:0/700372702 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f650801c6b0 con 0x557b30df0b10 2023-11-16T16:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.117+0000 7f65290e7b80 10 client.25682 fetch_fsmap learned FSMap version 29 2023-11-16T16:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.117+0000 7f65290e7b80 1 -- 172.21.15.93:0/700372702 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557b30b57c00 con 0x557b30df0b10 2023-11-16T16:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.117+0000 7f64f67fc700 1 -- 172.21.15.93:0/700372702 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64f809afd0 con 0x557b30df0b10 2023-11-16T16:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.118+0000 7f64f67fc700 1 -- 172.21.15.93:0/700372702 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 29) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6508024070 con 0x557b30df0b10 2023-11-16T16:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.118+0000 7f65290e7b80 10 client.25682 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.118+0000 7f65290e7b80 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.118+0000 7f65290e7b80 20 client.25682 populate_metadata read hostname 'smithi093' 2023-11-16T16:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.118+0000 7f65290e7b80 1 --2- 172.21.15.93:0/700372702 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557b30e59350 0x557b30e5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.118+0000 7f65290e7b80 4 client.25682 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.118+0000 7f65290e7b80 1 -- 172.21.15.93:0/700372702 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557b30d0f600 con 0x557b30e59350 2023-11-16T16:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.118+0000 7f650effd700 1 --2- 172.21.15.93:0/700372702 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557b30e59350 0x557b30e5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.119+0000 7f650effd700 1 --2- 172.21.15.93:0/700372702 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557b30e59350 0x557b30e5b760 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0678eb63bf9c8de server_cookie=cd0c012dea16441d in_seq=0 out_seq=0 2023-11-16T16:53:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:18.119+0000 7f64f67fc700 10 client.25682 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:18.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:18 smithi125 ceph-mon[131076]: pgmap v486: 129 pgs: 129 active+clean; 250 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 783 op/s 2023-11-16T16:53:18.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:18 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:53:18.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:18 smithi093 ceph-mon[97863]: pgmap v486: 129 pgs: 129 active+clean; 250 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 783 op/s 2023-11-16T16:53:18.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:18 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:53:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:18 smithi159 ceph-mon[131528]: pgmap v486: 129 pgs: 129 active+clean; 250 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 783 op/s 2023-11-16T16:53:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:18 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:53:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.824+0000 7f64f67fc700 1 -- 172.21.15.93:0/700372702 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557b30d0f600 con 0x557b30e59350 2023-11-16T16:53:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.824+0000 7f64f67fc700 10 client.25682 handle_command_reply: tid=0 2023-11-16T16:53:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.824+0000 7f65290e7b80 1 client.25682 shutdown 2023-11-16T16:53:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.825+0000 7f65290e7b80 1 -- 172.21.15.93:0/700372702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 msgr2=0x557b30dff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.825+0000 7f65290e7b80 1 --2- 172.21.15.93:0/700372702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 0x557b30dff470 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6508004bd0 tx=0x7f6508005250 comp rx=0 tx=0).stop 2023-11-16T16:53:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.825+0000 7f65290e7b80 1 -- 172.21.15.93:0/700372702 shutdown_connections 2023-11-16T16:53:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.825+0000 7f65290e7b80 1 --2- 172.21.15.93:0/700372702 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x557b30e59350 0x557b30e5b760 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.825+0000 7f65290e7b80 1 --2- 172.21.15.93:0/700372702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557b30df0b10 0x557b30dff470 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.825+0000 7f65290e7b80 1 --2- 172.21.15.93:0/700372702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b30c4b6b0 0x557b30dfbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.825+0000 7f65290e7b80 1 --2- 172.21.15.93:0/700372702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557b30defdb0 0x557b30dfc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.825+0000 7f65290e7b80 1 -- 172.21.15.93:0/700372702 >> 172.21.15.93:0/700372702 conn(0x557b30de4f50 msgr2=0x557b30dee550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.826+0000 7f64f67fc700 0 client.25682 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.826+0000 7f65290e7b80 1 -- 172.21.15.93:0/700372702 shutdown_connections 2023-11-16T16:53:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.826+0000 7f65290e7b80 1 -- 172.21.15.93:0/700372702 wait complete. 2023-11-16T16:53:19.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:53:19.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:53:19.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:53:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:53:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:53:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:53:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:53:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.828+0000 7f65290e7b80 20 client.25682 trim_cache size 0 max 16384 2023-11-16T16:53:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.829+0000 7f65157ff700 1 -- 172.21.15.93:0/3229412021 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f64fc0a12a0 msgr2=0x7f64fc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.829+0000 7f65157ff700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f64fc0a12a0 0x7f64fc0a3750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6500007fd0 tx=0x7f6500007e80 comp rx=0 tx=0).stop 2023-11-16T16:53:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.829+0000 7f65157ff700 1 -- 172.21.15.93:0/3229412021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6510106d10 msgr2=0x7f651019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.829+0000 7f65157ff700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6510106d10 0x7f651019c540 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f64f80098e0 tx=0x7f64f80057b0 comp rx=0 tx=0).stop 2023-11-16T16:53:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.831+0000 7f65157ff700 1 -- 172.21.15.93:0/3229412021 shutdown_connections 2023-11-16T16:53:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.831+0000 7f65157ff700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f64fc0a12a0 0x7f64fc0a3750 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.831+0000 7f65157ff700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6510108400 0x7f65101a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.831+0000 7f65157ff700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6510106d10 0x7f651019c540 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.831+0000 7f65157ff700 1 --2- 172.21.15.93:0/3229412021 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6510107a70 0x7f651019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.831+0000 7f65157ff700 1 -- 172.21.15.93:0/3229412021 >> 172.21.15.93:0/3229412021 conn(0x7f65100758b0 msgr2=0x7f651010af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.831+0000 7f65157ff700 1 -- 172.21.15.93:0/3229412021 shutdown_connections 2023-11-16T16:53:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:19.831+0000 7f65157ff700 1 -- 172.21.15.93:0/3229412021 wait complete. 2023-11-16T16:53:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:53:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:53:19.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:53:19.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:53:19.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:53:19.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:53:20.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:20 smithi125 ceph-mon[131076]: pgmap v487: 129 pgs: 129 active+clean; 259 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 741 op/s 2023-11-16T16:53:20.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:20 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:53:20.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:20 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:53:20.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:20 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:53:20.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:20 smithi125 ceph-mon[131076]: Service mds.all has no related daemon already placed. 2023-11-16T16:53:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:20 smithi093 ceph-mon[97863]: pgmap v487: 129 pgs: 129 active+clean; 259 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 741 op/s 2023-11-16T16:53:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:20 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:53:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:20 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:53:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:20 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:53:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:20 smithi093 ceph-mon[97863]: Service mds.all has no related daemon already placed. 2023-11-16T16:53:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:20 smithi159 ceph-mon[131528]: pgmap v487: 129 pgs: 129 active+clean; 259 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 741 op/s 2023-11-16T16:53:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:20 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:53:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:20 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:53:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:20 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:53:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:20 smithi159 ceph-mon[131528]: Service mds.all has no related daemon already placed. 2023-11-16T16:53:22.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:22 smithi125 ceph-mon[131076]: pgmap v488: 129 pgs: 129 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 727 op/s 2023-11-16T16:53:22.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:22 smithi093 ceph-mon[97863]: pgmap v488: 129 pgs: 129 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 727 op/s 2023-11-16T16:53:22.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:22 smithi159 ceph-mon[131528]: pgmap v488: 129 pgs: 129 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 727 op/s 2023-11-16T16:53:24.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:24 smithi125 ceph-mon[131076]: pgmap v489: 129 pgs: 129 active+clean; 257 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 720 op/s 2023-11-16T16:53:24.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:24 smithi093 ceph-mon[97863]: pgmap v489: 129 pgs: 129 active+clean; 257 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 720 op/s 2023-11-16T16:53:24.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:24 smithi159 ceph-mon[131528]: pgmap v489: 129 pgs: 129 active+clean; 257 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 720 op/s 2023-11-16T16:53:26.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:26 smithi125 ceph-mon[131076]: pgmap v490: 129 pgs: 129 active+clean; 257 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 699 op/s 2023-11-16T16:53:26.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:26 smithi093 ceph-mon[97863]: pgmap v490: 129 pgs: 129 active+clean; 257 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 699 op/s 2023-11-16T16:53:26.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:26 smithi159 ceph-mon[131528]: pgmap v490: 129 pgs: 129 active+clean; 257 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 699 op/s 2023-11-16T16:53:28.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:28 smithi125 ceph-mon[131076]: pgmap v491: 129 pgs: 129 active+clean; 262 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 668 op/s 2023-11-16T16:53:28.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:28 smithi093 ceph-mon[97863]: pgmap v491: 129 pgs: 129 active+clean; 262 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 668 op/s 2023-11-16T16:53:28.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:28 smithi159 ceph-mon[131528]: pgmap v491: 129 pgs: 129 active+clean; 262 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 668 op/s 2023-11-16T16:53:30.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:30 smithi125 ceph-mon[131076]: pgmap v492: 129 pgs: 129 active+clean; 274 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 720 op/s 2023-11-16T16:53:30.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:30 smithi093 ceph-mon[97863]: pgmap v492: 129 pgs: 129 active+clean; 274 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 720 op/s 2023-11-16T16:53:30.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:30 smithi159 ceph-mon[131528]: pgmap v492: 129 pgs: 129 active+clean; 274 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 720 op/s 2023-11-16T16:53:32.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:32 smithi125 ceph-mon[131076]: pgmap v493: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 689 op/s 2023-11-16T16:53:32.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:32 smithi093 ceph-mon[97863]: pgmap v493: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 689 op/s 2023-11-16T16:53:32.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:32 smithi159 ceph-mon[131528]: pgmap v493: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 689 op/s 2023-11-16T16:53:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.037+0000 7f58de9ff700 1 -- 172.21.15.93:0/4041903676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d80734f0 msgr2=0x7f58d8073960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.037+0000 7f58de9ff700 1 --2- 172.21.15.93:0/4041903676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d80734f0 0x7f58d8073960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009990 tx=0x7f58cc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.037+0000 7f58de9ff700 1 -- 172.21.15.93:0/4041903676 shutdown_connections 2023-11-16T16:53:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.037+0000 7f58de9ff700 1 --2- 172.21.15.93:0/4041903676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58d8074ad0 0x7f58d8072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.037+0000 7f58de9ff700 1 --2- 172.21.15.93:0/4041903676 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58d8108100 0x7f58d810a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.037+0000 7f58de9ff700 1 --2- 172.21.15.93:0/4041903676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d80734f0 0x7f58d8073960 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.037+0000 7f58de9ff700 1 -- 172.21.15.93:0/4041903676 >> 172.21.15.93:0/4041903676 conn(0x7f58d80fb890 msgr2=0x7f58d80fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.037+0000 7f58de9ff700 1 -- 172.21.15.93:0/4041903676 shutdown_connections 2023-11-16T16:53:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.038+0000 7f58de9ff700 1 -- 172.21.15.93:0/4041903676 wait complete. 2023-11-16T16:53:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.039+0000 7f58de9ff700 1 Processor -- start 2023-11-16T16:53:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.039+0000 7f58de9ff700 1 -- start start 2023-11-16T16:53:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.039+0000 7f58de9ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58d80734f0 0x7f58d8198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.039+0000 7f58de9ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58d8074ad0 0x7f58d81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.039+0000 7f58de9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d8108100 0x7f58d819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.039+0000 7f58de9ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d819fcf0 con 0x7f58d8074ad0 2023-11-16T16:53:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.039+0000 7f58de9ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d819fe30 con 0x7f58d80734f0 2023-11-16T16:53:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.039+0000 7f58de9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d819ffc0 con 0x7f58d8108100 2023-11-16T16:53:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.040+0000 7f58d7fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58d80734f0 0x7f58d8198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.040+0000 7f58d7fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58d80734f0 0x7f58d8198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:51182/0 (socket says 172.21.15.93:51182) 2023-11-16T16:53:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.040+0000 7f58d7fff700 1 -- 172.21.15.93:0/779209948 learned_addr learned my addr 172.21.15.93:0/779209948 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.040+0000 7f58d77fe700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58d8074ad0 0x7f58d81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.040+0000 7f58dcf9c700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d8108100 0x7f58d819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.040+0000 7f58d7fff700 1 -- 172.21.15.93:0/779209948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d8108100 msgr2=0x7f58d819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.040+0000 7f58d7fff700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d8108100 0x7f58d819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.040+0000 7f58d7fff700 1 -- 172.21.15.93:0/779209948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58d8074ad0 msgr2=0x7f58d81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.040+0000 7f58d7fff700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58d8074ad0 0x7f58d81986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.040+0000 7f58d7fff700 1 -- 172.21.15.93:0/779209948 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c8009520 con 0x7f58d80734f0 2023-11-16T16:53:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.040+0000 7f58d7fff700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58d80734f0 0x7f58d8198170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f58c800eb30 tx=0x7f58c800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.041+0000 7f58d57fa700 1 -- 172.21.15.93:0/779209948 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8009880 con 0x7f58d80734f0 2023-11-16T16:53:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.041+0000 7f58d57fa700 1 -- 172.21.15.93:0/779209948 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58c80044e0 con 0x7f58d80734f0 2023-11-16T16:53:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.041+0000 7f58de9ff700 1 -- 172.21.15.93:0/779209948 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58cc0095f0 con 0x7f58d80734f0 2023-11-16T16:53:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.041+0000 7f58d57fa700 1 -- 172.21.15.93:0/779209948 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8003d80 con 0x7f58d80734f0 2023-11-16T16:53:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.041+0000 7f58de9ff700 1 -- 172.21.15.93:0/779209948 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d810de10 con 0x7f58d80734f0 2023-11-16T16:53:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.044+0000 7f58d57fa700 1 -- 172.21.15.93:0/779209948 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f58c8003f20 con 0x7f58d80734f0 2023-11-16T16:53:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.045+0000 7f58f2263b80 1 Processor -- start 2023-11-16T16:53:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.046+0000 7f58f2263b80 1 -- start start 2023-11-16T16:53:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.046+0000 7f58f2263b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5590274a56b0 0x559027649870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.046+0000 7f58f2263b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559027649db0 0x55902764a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.046+0000 7f58f2263b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55902764ab10 0x55902764af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.046+0000 7f58f2263b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902754f000 con 0x559027649db0 2023-11-16T16:53:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.046+0000 7f58f2263b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590275a1840 con 0x5590274a56b0 2023-11-16T16:53:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.046+0000 7f58f2263b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902759acd0 con 0x55902764ab10 2023-11-16T16:53:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.046+0000 7f58dcf9c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5590274a56b0 0x559027649870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.046+0000 7f58dcf9c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5590274a56b0 0x559027649870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:51194/0 (socket says 172.21.15.93:51194) 2023-11-16T16:53:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.047+0000 7f58dcf9c700 1 -- 172.21.15.93:0/1288079219 learned_addr learned my addr 172.21.15.93:0/1288079219 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.047+0000 7f58d57fa700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f58c00a11d0 0x7f58c00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.047+0000 7f58d77fe700 1 --2- 172.21.15.93:0/1288079219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55902764ab10 0x55902764af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.047+0000 7f58d57fa700 1 -- 172.21.15.93:0/779209948 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f58c8014070 con 0x7f58d80734f0 2023-11-16T16:53:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.047+0000 7f58dcf9c700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f58c00a11d0 0x7f58c00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.047+0000 7f58dcf9c700 1 -- 172.21.15.93:0/1288079219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55902764ab10 msgr2=0x55902764af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.047+0000 7f58dcf9c700 1 --2- 172.21.15.93:0/1288079219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55902764ab10 0x55902764af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.047+0000 7f58dcf9c700 1 -- 172.21.15.93:0/1288079219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559027649db0 msgr2=0x55902764a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.047+0000 7f58dcf9c700 1 --2- 172.21.15.93:0/1288079219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559027649db0 0x55902764a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.047+0000 7f58dcf9c700 1 -- 172.21.15.93:0/1288079219 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55902741c5d0 con 0x5590274a56b0 2023-11-16T16:53:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.047+0000 7f58dcf9c700 1 --2- 172.21.15.93:0/1288079219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5590274a56b0 0x559027649870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f58d00137a0 tx=0x7f58d0013ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0ad62a1cbd7b6e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.048+0000 7f58be7fc700 1 -- 172.21.15.93:0/1288079219 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d00155b0 con 0x5590274a56b0 2023-11-16T16:53:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.048+0000 7f58be7fc700 1 -- 172.21.15.93:0/1288079219 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58d00163f0 con 0x5590274a56b0 2023-11-16T16:53:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.048+0000 7f58be7fc700 1 -- 172.21.15.93:0/1288079219 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0021ea0 con 0x5590274a56b0 2023-11-16T16:53:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.049+0000 7f58f2263b80 1 -- 172.21.15.93:0/1288079219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5590274a56b0 msgr2=0x559027649870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.049+0000 7f58dcf9c700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f58c00a11d0 0x7f58c00a3680 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f58d0020000 tx=0x7f58d001a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.049+0000 7f58f2263b80 1 --2- 172.21.15.93:0/1288079219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5590274a56b0 0x559027649870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f58d00137a0 tx=0x7f58d0013ab0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.049+0000 7f58f2263b80 1 -- 172.21.15.93:0/1288079219 shutdown_connections 2023-11-16T16:53:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.049+0000 7f58f2263b80 1 --2- 172.21.15.93:0/1288079219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55902764ab10 0x55902764af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.049+0000 7f58f2263b80 1 --2- 172.21.15.93:0/1288079219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5590274a56b0 0x559027649870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.049+0000 7f58f2263b80 1 --2- 172.21.15.93:0/1288079219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559027649db0 0x55902764a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.049+0000 7f58f2263b80 1 -- 172.21.15.93:0/1288079219 >> 172.21.15.93:0/1288079219 conn(0x55902763ef50 msgr2=0x559027644100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.049+0000 7f58f2263b80 1 -- 172.21.15.93:0/1288079219 shutdown_connections 2023-11-16T16:53:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.050+0000 7f58f2263b80 1 -- 172.21.15.93:0/1288079219 wait complete. 2023-11-16T16:53:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.050+0000 7f58f2263b80 1 -- start start 2023-11-16T16:53:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.050+0000 7f58f2263b80 1 Processor -- start 2023-11-16T16:53:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.051+0000 7f58f2263b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590274a56b0 0x55902765b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.051+0000 7f58f2263b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559027649db0 0x55902765b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.051+0000 7f58f2263b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55902764ab10 0x559027662a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.051+0000 7f58f2263b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902753c880 con 0x559027649db0 2023-11-16T16:53:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.051+0000 7f58f2263b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902753c520 con 0x55902764ab10 2023-11-16T16:53:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.051+0000 7f58f2263b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902741e8c0 con 0x5590274a56b0 2023-11-16T16:53:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.051+0000 7f58f2263b80 10 client.0 _ll_register_callbacks cb 0x55902755a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:53:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.051+0000 7f58dcf9c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55902764ab10 0x559027662a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.051+0000 7f58dcf9c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55902764ab10 0x559027662a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:51200/0 (socket says 172.21.15.93:51200) 2023-11-16T16:53:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.051+0000 7f58dcf9c700 1 -- 172.21.15.93:0/3809659882 learned_addr learned my addr 172.21.15.93:0/3809659882 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.052+0000 7f58d7fff700 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590274a56b0 0x55902765b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.052+0000 7f58d77fe700 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559027649db0 0x55902765b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.052+0000 7f58dcf9c700 1 -- 172.21.15.93:0/3809659882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590274a56b0 msgr2=0x55902765b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.052+0000 7f58dcf9c700 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590274a56b0 0x55902765b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.052+0000 7f58dcf9c700 1 -- 172.21.15.93:0/3809659882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559027649db0 msgr2=0x55902765b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.052+0000 7f58dcf9c700 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559027649db0 0x55902765b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.052+0000 7f58dcf9c700 1 -- 172.21.15.93:0/3809659882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d0013270 con 0x55902764ab10 2023-11-16T16:53:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.052+0000 7f58dcf9c700 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55902764ab10 0x559027662a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58d0015ee0 tx=0x7f58d0015fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f00ede370008fb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.053+0000 7f58b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:53:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.056+0000 7f58b77fe700 1 -- 172.21.15.93:0/3809659882 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0016ea0 con 0x55902764ab10 2023-11-16T16:53:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.057+0000 7f58b77fe700 1 -- 172.21.15.93:0/3809659882 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f58d0003a00 con 0x55902764ab10 2023-11-16T16:53:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.057+0000 7f58f2263b80 1 -- 172.21.15.93:0/3809659882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55902763c710 con 0x55902764ab10 2023-11-16T16:53:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.057+0000 7f58b77fe700 1 -- 172.21.15.93:0/3809659882 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58d0003bc0 con 0x55902764ab10 2023-11-16T16:53:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.057+0000 7f58f2263b80 10 client.25691 fetch_fsmap learned FSMap version 29 2023-11-16T16:53:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.057+0000 7f58f2263b80 1 -- 172.21.15.93:0/3809659882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559027561130 con 0x55902764ab10 2023-11-16T16:53:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.058+0000 7f58b77fe700 1 -- 172.21.15.93:0/3809659882 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55902763c710 con 0x55902764ab10 2023-11-16T16:53:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.058+0000 7f58b77fe700 1 -- 172.21.15.93:0/3809659882 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 29) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f58d001e640 con 0x55902764ab10 2023-11-16T16:53:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.058+0000 7f58f2263b80 10 client.25691 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:53:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.058+0000 7f58f2263b80 10 client.25691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:53:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.058+0000 7f58f2263b80 20 client.25691 populate_metadata read hostname 'smithi093' 2023-11-16T16:53:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.058+0000 7f58f2263b80 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5590276672d0 0x5590276696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.058+0000 7f58f2263b80 4 client.25691 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:53:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.058+0000 7f58f2263b80 1 -- 172.21.15.93:0/3809659882 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55902755d040 con 0x5590276672d0 2023-11-16T16:53:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.062+0000 7f58d7fff700 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5590276672d0 0x5590276696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.065+0000 7f58d7fff700 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5590276672d0 0x5590276696e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e119e0784f5c03b8 server_cookie=365f852e98c40603 in_seq=0 out_seq=0 2023-11-16T16:53:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.067+0000 7f58b77fe700 10 client.25691 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.067+0000 7f58b77fe700 1 -- 172.21.15.93:0/3809659882 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55902755d040 con 0x5590276672d0 2023-11-16T16:53:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.067+0000 7f58b77fe700 10 client.25691 handle_command_reply: tid=0 2023-11-16T16:53:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.067+0000 7f58f2263b80 1 client.25691 shutdown 2023-11-16T16:53:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.068+0000 7f58f2263b80 1 -- 172.21.15.93:0/3809659882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55902764ab10 msgr2=0x559027662a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.068+0000 7f58f2263b80 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55902764ab10 0x559027662a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58d0015ee0 tx=0x7f58d0015fc0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.073+0000 7f58f2263b80 1 -- 172.21.15.93:0/3809659882 shutdown_connections 2023-11-16T16:53:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.073+0000 7f58f2263b80 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5590276672d0 0x5590276696e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.073+0000 7f58f2263b80 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55902764ab10 0x559027662a30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.073+0000 7f58b77fe700 0 client.25691 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.073+0000 7f58f2263b80 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590274a56b0 0x55902765b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.073+0000 7f58f2263b80 1 --2- 172.21.15.93:0/3809659882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559027649db0 0x55902765b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.073+0000 7f58f2263b80 1 -- 172.21.15.93:0/3809659882 >> 172.21.15.93:0/3809659882 conn(0x55902763ef50 msgr2=0x5590276437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.074+0000 7f58f2263b80 1 -- 172.21.15.93:0/3809659882 shutdown_connections 2023-11-16T16:53:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.074+0000 7f58f2263b80 1 -- 172.21.15.93:0/3809659882 wait complete. 2023-11-16T16:53:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.075+0000 7f58f2263b80 20 client.25691 trim_cache size 0 max 16384 2023-11-16T16:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.119+0000 7f58f2263b80 1 Processor -- start 2023-11-16T16:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.119+0000 7f58f2263b80 1 -- start start 2023-11-16T16:53:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.119+0000 7f58f2263b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590274a56b0 0x55902765a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.120+0000 7f58f2263b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559027649db0 0x5590276629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.120+0000 7f58f2263b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55902764ab10 0x559027650dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.120+0000 7f58f2263b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902765c6a0 con 0x5590274a56b0 2023-11-16T16:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.120+0000 7f58f2263b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902765c500 con 0x55902764ab10 2023-11-16T16:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.120+0000 7f58f2263b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902765b020 con 0x559027649db0 2023-11-16T16:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.120+0000 7f58d77fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559027649db0 0x5590276629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.121+0000 7f58d7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590274a56b0 0x55902765a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.121+0000 7f58d7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590274a56b0 0x55902765a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:52746/0 (socket says 172.21.15.93:52746) 2023-11-16T16:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.121+0000 7f58d7fff700 1 -- 172.21.15.93:0/2749731967 learned_addr learned my addr 172.21.15.93:0/2749731967 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.121+0000 7f58dcf9c700 1 --2- 172.21.15.93:0/2749731967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55902764ab10 0x559027650dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.121+0000 7f58d7fff700 1 -- 172.21.15.93:0/2749731967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559027649db0 msgr2=0x5590276629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.121+0000 7f58d7fff700 1 --2- 172.21.15.93:0/2749731967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559027649db0 0x5590276629f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.121+0000 7f58d7fff700 1 -- 172.21.15.93:0/2749731967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55902764ab10 msgr2=0x559027650dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.121+0000 7f58d7fff700 1 --2- 172.21.15.93:0/2749731967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55902764ab10 0x559027650dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.121+0000 7f58d7fff700 1 -- 172.21.15.93:0/2749731967 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559027560e80 con 0x5590274a56b0 2023-11-16T16:53:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.122+0000 7f58d7fff700 1 --2- 172.21.15.93:0/2749731967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590274a56b0 0x55902765a2c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f58c809a180 tx=0x7f58c8099b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539cf81b7e5726d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.123+0000 7f58beffd700 1 -- 172.21.15.93:0/2749731967 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c809a1f0 con 0x5590274a56b0 2023-11-16T16:53:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.123+0000 7f58beffd700 1 -- 172.21.15.93:0/2749731967 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58c809e220 con 0x5590274a56b0 2023-11-16T16:53:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.123+0000 7f58beffd700 1 -- 172.21.15.93:0/2749731967 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8098660 con 0x5590274a56b0 2023-11-16T16:53:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.126+0000 7f58f2263b80 1 -- 172.21.15.93:0/2749731967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590274a56b0 msgr2=0x55902765a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.126+0000 7f58f2263b80 1 --2- 172.21.15.93:0/2749731967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590274a56b0 0x55902765a2c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f58c809a180 tx=0x7f58c8099b50 comp rx=0 tx=0).stop 2023-11-16T16:53:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.127+0000 7f58f2263b80 1 -- 172.21.15.93:0/2749731967 shutdown_connections 2023-11-16T16:53:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.127+0000 7f58f2263b80 1 --2- 172.21.15.93:0/2749731967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590274a56b0 0x55902765a2c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.127+0000 7f58f2263b80 1 --2- 172.21.15.93:0/2749731967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55902764ab10 0x559027650dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.127+0000 7f58f2263b80 1 --2- 172.21.15.93:0/2749731967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559027649db0 0x5590276629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.127+0000 7f58f2263b80 1 -- 172.21.15.93:0/2749731967 >> 172.21.15.93:0/2749731967 conn(0x55902763ef50 msgr2=0x55902766e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.127+0000 7f58f2263b80 1 -- 172.21.15.93:0/2749731967 shutdown_connections 2023-11-16T16:53:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.127+0000 7f58f2263b80 1 -- 172.21.15.93:0/2749731967 wait complete. 2023-11-16T16:53:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.128+0000 7f58f2263b80 1 -- start start 2023-11-16T16:53:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.129+0000 7f58f2263b80 1 Processor -- start 2023-11-16T16:53:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.129+0000 7f58f2263b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590274a56b0 0x55902765ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.129+0000 7f58f2263b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559027649db0 0x55902765f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.130+0000 7f58f2263b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55902764ab10 0x559027661f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.130+0000 7f58f2263b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590276624a0 con 0x55902764ab10 2023-11-16T16:53:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.130+0000 7f58f2263b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559027662610 con 0x559027649db0 2023-11-16T16:53:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.130+0000 7f58f2263b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590276627d0 con 0x5590274a56b0 2023-11-16T16:53:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.130+0000 7f58f2263b80 10 client.0 _ll_register_callbacks cb 0x7f58c8005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:53:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.130+0000 7f58d7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590274a56b0 0x55902765ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.130+0000 7f58d7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590274a56b0 0x55902765ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:58496/0 (socket says 172.21.15.93:58496) 2023-11-16T16:53:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.130+0000 7f58d7fff700 1 -- 172.21.15.93:0/1161654751 learned_addr learned my addr 172.21.15.93:0/1161654751 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.130+0000 7f58d77fe700 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559027649db0 0x55902765f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.131+0000 7f58d77fe700 1 -- 172.21.15.93:0/1161654751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590274a56b0 msgr2=0x55902765ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.131+0000 7f58d77fe700 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590274a56b0 0x55902765ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.131+0000 7f58dcf9c700 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55902764ab10 0x559027661f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.131+0000 7f58d77fe700 1 -- 172.21.15.93:0/1161654751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55902764ab10 msgr2=0x559027661f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.131+0000 7f58d77fe700 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55902764ab10 0x559027661f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.131+0000 7f58d77fe700 1 -- 172.21.15.93:0/1161654751 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d810de10 con 0x559027649db0 2023-11-16T16:53:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.131+0000 7f58dcf9c700 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55902764ab10 0x559027661f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:53:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.134+0000 7f58d77fe700 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559027649db0 0x55902765f160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00b680 tx=0x7f58cc005320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4727897a4cc87ad9 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.134+0000 7f58b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:53:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.134+0000 7f58b77fe700 1 -- 172.21.15.93:0/1161654751 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc01d040 con 0x559027649db0 2023-11-16T16:53:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.134+0000 7f58b77fe700 1 -- 172.21.15.93:0/1161654751 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f58cc004910 con 0x559027649db0 2023-11-16T16:53:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.136+0000 7f58f2263b80 1 -- 172.21.15.93:0/1161654751 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58d00163f0 con 0x559027649db0 2023-11-16T16:53:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.137+0000 7f58b77fe700 1 -- 172.21.15.93:0/1161654751 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58cc004ad0 con 0x559027649db0 2023-11-16T16:53:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.137+0000 7f58f2263b80 10 client.25697 fetch_fsmap learned FSMap version 29 2023-11-16T16:53:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.137+0000 7f58f2263b80 1 -- 172.21.15.93:0/1161654751 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5590273b1c00 con 0x559027649db0 2023-11-16T16:53:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.137+0000 7f58b77fe700 1 -- 172.21.15.93:0/1161654751 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58d00163f0 con 0x559027649db0 2023-11-16T16:53:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.138+0000 7f58b77fe700 1 -- 172.21.15.93:0/1161654751 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 29) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f58cc023040 con 0x559027649db0 2023-11-16T16:53:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.138+0000 7f58f2263b80 10 client.25697 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:53:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.138+0000 7f58f2263b80 10 client.25697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:53:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.138+0000 7f58f2263b80 20 client.25697 populate_metadata read hostname 'smithi093' 2023-11-16T16:53:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.138+0000 7f58f2263b80 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5590276bc350 0x5590276be760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.138+0000 7f58f2263b80 4 client.25697 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:53:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.138+0000 7f58f2263b80 1 -- 172.21.15.93:0/1161654751 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559027569600 con 0x5590276bc350 2023-11-16T16:53:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.139+0000 7f58d7fff700 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5590276bc350 0x5590276be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.140+0000 7f58d7fff700 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5590276bc350 0x5590276be760 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=430658691f1494b3 server_cookie=f27df91d0ee7a441 in_seq=0 out_seq=0 2023-11-16T16:53:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.140+0000 7f58b77fe700 10 client.25697 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.773+0000 7f58b77fe700 1 -- 172.21.15.93:0/1161654751 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559027569600 con 0x5590276bc350 2023-11-16T16:53:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.774+0000 7f58b77fe700 10 client.25697 handle_command_reply: tid=0 2023-11-16T16:53:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.774+0000 7f58f2263b80 1 client.25697 shutdown 2023-11-16T16:53:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.774+0000 7f58f2263b80 1 -- 172.21.15.93:0/1161654751 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559027649db0 msgr2=0x55902765f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.775+0000 7f58f2263b80 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559027649db0 0x55902765f160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00b680 tx=0x7f58cc005320 comp rx=0 tx=0).stop 2023-11-16T16:53:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.775+0000 7f58f2263b80 1 -- 172.21.15.93:0/1161654751 shutdown_connections 2023-11-16T16:53:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.775+0000 7f58f2263b80 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5590276bc350 0x5590276be760 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.776+0000 7f58f2263b80 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55902764ab10 0x559027661f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.776+0000 7f58f2263b80 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590274a56b0 0x55902765ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.776+0000 7f58f2263b80 1 --2- 172.21.15.93:0/1161654751 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559027649db0 0x55902765f160 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.776+0000 7f58f2263b80 1 -- 172.21.15.93:0/1161654751 >> 172.21.15.93:0/1161654751 conn(0x55902763ef50 msgr2=0x55902764d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.776+0000 7f58b77fe700 0 client.25697 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.776+0000 7f58f2263b80 1 -- 172.21.15.93:0/1161654751 shutdown_connections 2023-11-16T16:53:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.776+0000 7f58f2263b80 1 -- 172.21.15.93:0/1161654751 wait complete. 2023-11-16T16:53:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.777+0000 7f58f2263b80 20 client.25697 trim_cache size 0 max 16384 2023-11-16T16:53:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:53:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:53:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:53:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:53:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.779+0000 7f58de9ff700 1 -- 172.21.15.93:0/779209948 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f58c00a11d0 msgr2=0x7f58c00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.779+0000 7f58de9ff700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f58c00a11d0 0x7f58c00a3680 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f58d0020000 tx=0x7f58d001a000 comp rx=0 tx=0).stop 2023-11-16T16:53:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.779+0000 7f58de9ff700 1 -- 172.21.15.93:0/779209948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58d80734f0 msgr2=0x7f58d8198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.779+0000 7f58de9ff700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58d80734f0 0x7f58d8198170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f58c800eb30 tx=0x7f58c800ee40 comp rx=0 tx=0).stop 2023-11-16T16:53:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.779+0000 7f58de9ff700 1 -- 172.21.15.93:0/779209948 shutdown_connections 2023-11-16T16:53:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.779+0000 7f58de9ff700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f58c00a11d0 0x7f58c00a3680 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.779+0000 7f58de9ff700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d8108100 0x7f58d819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.779+0000 7f58de9ff700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58d80734f0 0x7f58d8198170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.779+0000 7f58de9ff700 1 --2- 172.21.15.93:0/779209948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58d8074ad0 0x7f58d81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.779+0000 7f58de9ff700 1 -- 172.21.15.93:0/779209948 >> 172.21.15.93:0/779209948 conn(0x7f58d80fb890 msgr2=0x7f58d8104090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.780+0000 7f58de9ff700 1 -- 172.21.15.93:0/779209948 shutdown_connections 2023-11-16T16:53:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:33.781+0000 7f58de9ff700 1 -- 172.21.15.93:0/779209948 wait complete. 2023-11-16T16:53:34.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:34 smithi125 ceph-mon[131076]: pgmap v494: 129 pgs: 129 active+clean; 283 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 640 op/s 2023-11-16T16:53:34.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:34 smithi093 ceph-mon[97863]: pgmap v494: 129 pgs: 129 active+clean; 283 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 640 op/s 2023-11-16T16:53:34.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:34 smithi159 ceph-mon[131528]: pgmap v494: 129 pgs: 129 active+clean; 283 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 640 op/s 2023-11-16T16:53:36.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:36 smithi125 ceph-mon[131076]: pgmap v495: 129 pgs: 129 active+clean; 286 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 639 op/s 2023-11-16T16:53:36.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:36 smithi093 ceph-mon[97863]: pgmap v495: 129 pgs: 129 active+clean; 286 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 639 op/s 2023-11-16T16:53:36.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:36 smithi159 ceph-mon[131528]: pgmap v495: 129 pgs: 129 active+clean; 286 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 639 op/s 2023-11-16T16:53:38.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:38 smithi093 ceph-mon[97863]: pgmap v496: 129 pgs: 129 active+clean; 287 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 627 op/s 2023-11-16T16:53:38.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:38 smithi159 ceph-mon[131528]: pgmap v496: 129 pgs: 129 active+clean; 287 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 627 op/s 2023-11-16T16:53:39.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:38 smithi125 ceph-mon[131076]: pgmap v496: 129 pgs: 129 active+clean; 287 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 627 op/s 2023-11-16T16:53:40.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:40 smithi093 ceph-mon[97863]: pgmap v497: 129 pgs: 129 active+clean; 284 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 659 op/s 2023-11-16T16:53:40.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:40 smithi159 ceph-mon[131528]: pgmap v497: 129 pgs: 129 active+clean; 284 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 659 op/s 2023-11-16T16:53:41.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:40 smithi125 ceph-mon[131076]: pgmap v497: 129 pgs: 129 active+clean; 284 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 659 op/s 2023-11-16T16:53:42.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:42 smithi093 ceph-mon[97863]: pgmap v498: 129 pgs: 129 active+clean; 298 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 632 op/s 2023-11-16T16:53:42.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:42 smithi159 ceph-mon[131528]: pgmap v498: 129 pgs: 129 active+clean; 298 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 632 op/s 2023-11-16T16:53:43.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:42 smithi125 ceph-mon[131076]: pgmap v498: 129 pgs: 129 active+clean; 298 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 632 op/s 2023-11-16T16:53:44.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:44 smithi093 ceph-mon[97863]: pgmap v499: 129 pgs: 129 active+clean; 300 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 648 op/s 2023-11-16T16:53:44.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:44 smithi159 ceph-mon[131528]: pgmap v499: 129 pgs: 129 active+clean; 300 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 648 op/s 2023-11-16T16:53:45.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:44 smithi125 ceph-mon[131076]: pgmap v499: 129 pgs: 129 active+clean; 300 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 648 op/s 2023-11-16T16:53:46.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:46 smithi093 ceph-mon[97863]: pgmap v500: 129 pgs: 129 active+clean; 307 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 627 op/s 2023-11-16T16:53:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.959+0000 7fe225fff700 1 -- 172.21.15.93:0/3379209168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2201002c0 msgr2=0x7fe2201006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.959+0000 7fe225fff700 1 --2- 172.21.15.93:0/3379209168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2201002c0 0x7fe2201006d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe210009990 tx=0x7fe210009ca0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.960+0000 7fe225fff700 1 -- 172.21.15.93:0/3379209168 shutdown_connections 2023-11-16T16:53:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.960+0000 7fe225fff700 1 --2- 172.21.15.93:0/3379209168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe220101e40 0x7fe220108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.960+0000 7fe225fff700 1 --2- 172.21.15.93:0/3379209168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2201002c0 0x7fe2201006d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.960+0000 7fe225fff700 1 --2- 172.21.15.93:0/3379209168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2201014b0 0x7fe220101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.960+0000 7fe225fff700 1 -- 172.21.15.93:0/3379209168 >> 172.21.15.93:0/3379209168 conn(0x7fe2200fb8e0 msgr2=0x7fe2200fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.960+0000 7fe225fff700 1 -- 172.21.15.93:0/3379209168 shutdown_connections 2023-11-16T16:53:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.960+0000 7fe225fff700 1 -- 172.21.15.93:0/3379209168 wait complete. 2023-11-16T16:53:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.961+0000 7fe225fff700 1 Processor -- start 2023-11-16T16:53:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.961+0000 7fe225fff700 1 -- start start 2023-11-16T16:53:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.961+0000 7fe225fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2201002c0 0x7fe2201982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.961+0000 7fe225fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2201014b0 0x7fe2201987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.961+0000 7fe225fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe220101e40 0x7fe22019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe225fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22019fe30 con 0x7fe2201014b0 2023-11-16T16:53:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe225fff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22019ff70 con 0x7fe220101e40 2023-11-16T16:53:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe225fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2201a0100 con 0x7fe2201002c0 2023-11-16T16:53:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe21f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2201002c0 0x7fe2201982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe21f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2201002c0 0x7fe2201982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:38836/0 (socket says 172.21.15.93:38836) 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe21f7fe700 1 -- 172.21.15.93:0/517482791 learned_addr learned my addr 172.21.15.93:0/517482791 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe21effd700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2201014b0 0x7fe2201987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe21f7fe700 1 -- 172.21.15.93:0/517482791 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe220101e40 msgr2=0x7fe22019f860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe21f7fe700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe220101e40 0x7fe22019f860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe21f7fe700 1 -- 172.21.15.93:0/517482791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2201014b0 msgr2=0x7fe2201987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe21f7fe700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2201014b0 0x7fe2201987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe21f7fe700 1 -- 172.21.15.93:0/517482791 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100095f0 con 0x7fe2201002c0 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.962+0000 7fe21f7fe700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2201002c0 0x7fe2201982b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe21000f730 tx=0x7fe21000f810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.963+0000 7fe21cff9700 1 -- 172.21.15.93:0/517482791 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21001d070 con 0x7fe2201002c0 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.963+0000 7fe21cff9700 1 -- 172.21.15.93:0/517482791 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe21000fd30 con 0x7fe2201002c0 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.963+0000 7fe225fff700 1 -- 172.21.15.93:0/517482791 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22010bca0 con 0x7fe2201002c0 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.963+0000 7fe21cff9700 1 -- 172.21.15.93:0/517482791 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210005710 con 0x7fe2201002c0 2023-11-16T16:53:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.963+0000 7fe225fff700 1 -- 172.21.15.93:0/517482791 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22010bb10 con 0x7fe2201002c0 2023-11-16T16:53:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.968+0000 7fe21cff9700 1 -- 172.21.15.93:0/517482791 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe2100047e0 con 0x7fe2201002c0 2023-11-16T16:53:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.969+0000 7fe23987fb80 1 Processor -- start 2023-11-16T16:53:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.969+0000 7fe23987fb80 1 -- start start 2023-11-16T16:53:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe23987fb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 0x5606c75cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe23987fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c75cfdb0 0x5606c75d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe23987fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe23987fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c74d5000 con 0x5606c75cfdb0 2023-11-16T16:53:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe23987fb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c7527840 con 0x5606c742b6b0 2023-11-16T16:53:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe23987fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c7520cd0 con 0x5606c75d0b10 2023-11-16T16:53:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe21ffff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 0x5606c75cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe21effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe21ffff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 0x5606c75cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:41594/0 (socket says 172.21.15.93:41594) 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe21effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:38846/0 (socket says 172.21.15.93:38846) 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe21ffff700 1 -- 172.21.15.93:0/2067514143 learned_addr learned my addr 172.21.15.93:0/2067514143 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe21ffff700 1 -- 172.21.15.93:0/2067514143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 msgr2=0x5606c75d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe21ffff700 1 --2- 172.21.15.93:0/2067514143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe21ffff700 1 -- 172.21.15.93:0/2067514143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c75cfdb0 msgr2=0x5606c75d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe21ffff700 1 --2- 172.21.15.93:0/2067514143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c75cfdb0 0x5606c75d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.970+0000 7fe21ffff700 1 -- 172.21.15.93:0/2067514143 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606c73a25d0 con 0x5606c742b6b0 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.971+0000 7fe21ffff700 1 --2- 172.21.15.93:0/2067514143 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 0x5606c75cf870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe218009d00 tx=0x7fe21800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c42bf3ed329ea72 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.971+0000 7fe21cff9700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe2080a1150 0x7fe2080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.971+0000 7fe205ffb700 1 -- 172.21.15.93:0/2067514143 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800e070 con 0x5606c742b6b0 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.971+0000 7fe205ffb700 1 -- 172.21.15.93:0/2067514143 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe218009830 con 0x5606c742b6b0 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.971+0000 7fe205ffb700 1 -- 172.21.15.93:0/2067514143 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218011390 con 0x5606c742b6b0 2023-11-16T16:53:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.971+0000 7fe21f7fe700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe2080a1150 0x7fe2080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.972+0000 7fe23987fb80 1 -- 172.21.15.93:0/2067514143 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 msgr2=0x5606c75cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.972+0000 7fe23987fb80 1 --2- 172.21.15.93:0/2067514143 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 0x5606c75cf870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe218009d00 tx=0x7fe21800b330 comp rx=0 tx=0).stop 2023-11-16T16:53:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.972+0000 7fe21cff9700 1 -- 172.21.15.93:0/517482791 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe2100cdff0 con 0x7fe2201002c0 2023-11-16T16:53:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.972+0000 7fe21f7fe700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe2080a1150 0x7fe2080a3600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe21009b030 tx=0x7fe21009e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.972+0000 7fe23987fb80 1 -- 172.21.15.93:0/2067514143 shutdown_connections 2023-11-16T16:53:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.972+0000 7fe23987fb80 1 --2- 172.21.15.93:0/2067514143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.972+0000 7fe23987fb80 1 --2- 172.21.15.93:0/2067514143 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 0x5606c75cf870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.972+0000 7fe23987fb80 1 --2- 172.21.15.93:0/2067514143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c75cfdb0 0x5606c75d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.972+0000 7fe23987fb80 1 -- 172.21.15.93:0/2067514143 >> 172.21.15.93:0/2067514143 conn(0x5606c75c4f50 msgr2=0x5606c75ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.972+0000 7fe23987fb80 1 -- 172.21.15.93:0/2067514143 shutdown_connections 2023-11-16T16:53:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.972+0000 7fe23987fb80 1 -- 172.21.15.93:0/2067514143 wait complete. 2023-11-16T16:53:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.973+0000 7fe23987fb80 1 -- start start 2023-11-16T16:53:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.973+0000 7fe23987fb80 1 Processor -- start 2023-11-16T16:53:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.974+0000 7fe23987fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c742b6b0 0x5606c75e14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.974+0000 7fe23987fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75cfdb0 0x5606c75e1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.974+0000 7fe23987fb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75d0b10 0x5606c75e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.974+0000 7fe23987fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c74c2880 con 0x5606c742b6b0 2023-11-16T16:53:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.974+0000 7fe23987fb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c74c2520 con 0x5606c75d0b10 2023-11-16T16:53:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.974+0000 7fe23987fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c73a48c0 con 0x5606c75cfdb0 2023-11-16T16:53:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.974+0000 7fe23987fb80 10 client.0 _ll_register_callbacks cb 0x5606c74e0720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:53:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.974+0000 7fe21ffff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75d0b10 0x5606c75e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.974+0000 7fe21effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75cfdb0 0x5606c75e1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.974+0000 7fe21ffff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75d0b10 0x5606c75e8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:41608/0 (socket says 172.21.15.93:41608) 2023-11-16T16:53:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.974+0000 7fe21ffff700 1 -- 172.21.15.93:0/3549988815 learned_addr learned my addr 172.21.15.93:0/3549988815 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.974+0000 7fe21ffff700 1 --2- 172.21.15.93:0/3549988815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c742b6b0 0x5606c75e14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.975+0000 7fe21ffff700 1 -- 172.21.15.93:0/3549988815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75cfdb0 msgr2=0x5606c75e1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.975+0000 7fe21ffff700 1 --2- 172.21.15.93:0/3549988815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75cfdb0 0x5606c75e1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.975+0000 7fe21ffff700 1 -- 172.21.15.93:0/3549988815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75d0b10 msgr2=0x5606c75e8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.975+0000 7fe21ffff700 1 --2- 172.21.15.93:0/3549988815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75d0b10 0x5606c75e8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.975+0000 7fe21ffff700 1 -- 172.21.15.93:0/3549988815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2180099b0 con 0x5606c742b6b0 2023-11-16T16:53:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.976+0000 7fe21ffff700 1 --2- 172.21.15.93:0/3549988815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c742b6b0 0x5606c75e14c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe218009fd0 tx=0x7fe218004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb9fa9759005230 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.976+0000 7fe1feffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:53:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.976+0000 7fe1feffd700 1 -- 172.21.15.93:0/3549988815 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800e040 con 0x5606c742b6b0 2023-11-16T16:53:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.976+0000 7fe1feffd700 1 -- 172.21.15.93:0/3549988815 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe218006e40 con 0x5606c742b6b0 2023-11-16T16:53:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.976+0000 7fe23987fb80 1 -- 172.21.15.93:0/3549988815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606c74e3040 con 0x5606c742b6b0 2023-11-16T16:53:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.977+0000 7fe1feffd700 1 -- 172.21.15.93:0/3549988815 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe21801f940 con 0x5606c742b6b0 2023-11-16T16:53:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.977+0000 7fe23987fb80 10 client.15648 fetch_fsmap learned FSMap version 29 2023-11-16T16:53:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.977+0000 7fe23987fb80 1 -- 172.21.15.93:0/3549988815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5606c74e7130 con 0x5606c742b6b0 2023-11-16T16:53:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.978+0000 7fe1feffd700 1 -- 172.21.15.93:0/3549988815 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe218031080 con 0x5606c742b6b0 2023-11-16T16:53:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.978+0000 7fe1feffd700 1 -- 172.21.15.93:0/3549988815 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 29) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe21801cb40 con 0x5606c742b6b0 2023-11-16T16:53:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.978+0000 7fe23987fb80 10 client.15648 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:53:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.978+0000 7fe23987fb80 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:53:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.978+0000 7fe23987fb80 20 client.15648 populate_metadata read hostname 'smithi093' 2023-11-16T16:53:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.978+0000 7fe23987fb80 1 --2- 172.21.15.93:0/3549988815 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5606c75ed240 0x5606c75ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.978+0000 7fe23987fb80 4 client.15648 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:53:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.978+0000 7fe23987fb80 1 -- 172.21.15.93:0/3549988815 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606c74ef600 con 0x5606c75ed240 2023-11-16T16:53:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.978+0000 7fe21effd700 1 --2- 172.21.15.93:0/3549988815 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5606c75ed240 0x5606c75ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.979+0000 7fe21effd700 1 --2- 172.21.15.93:0/3549988815 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5606c75ed240 0x5606c75ef620 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43ab106998da0701 server_cookie=af0f4ea2eface4e0 in_seq=0 out_seq=0 2023-11-16T16:53:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.979+0000 7fe1feffd700 10 client.15648 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.980+0000 7fe1feffd700 1 -- 172.21.15.93:0/3549988815 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5606c74ef600 con 0x5606c75ed240 2023-11-16T16:53:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.980+0000 7fe1feffd700 10 client.15648 handle_command_reply: tid=0 2023-11-16T16:53:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.981+0000 7fe23987fb80 1 client.15648 shutdown 2023-11-16T16:53:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.981+0000 7fe23987fb80 1 -- 172.21.15.93:0/3549988815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c742b6b0 msgr2=0x5606c75e14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.981+0000 7fe23987fb80 1 --2- 172.21.15.93:0/3549988815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c742b6b0 0x5606c75e14c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe218009fd0 tx=0x7fe218004be0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.981+0000 7fe23987fb80 1 -- 172.21.15.93:0/3549988815 shutdown_connections 2023-11-16T16:53:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.981+0000 7fe23987fb80 1 --2- 172.21.15.93:0/3549988815 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5606c75ed240 0x5606c75ef620 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.981+0000 7fe23987fb80 1 --2- 172.21.15.93:0/3549988815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c742b6b0 0x5606c75e14c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.981+0000 7fe23987fb80 1 --2- 172.21.15.93:0/3549988815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75d0b10 0x5606c75e8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.981+0000 7fe23987fb80 1 --2- 172.21.15.93:0/3549988815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75cfdb0 0x5606c75e1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.981+0000 7fe23987fb80 1 -- 172.21.15.93:0/3549988815 >> 172.21.15.93:0/3549988815 conn(0x5606c75c4f50 msgr2=0x5606c75c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.981+0000 7fe1feffd700 0 client.15648 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.982+0000 7fe23987fb80 1 -- 172.21.15.93:0/3549988815 shutdown_connections 2023-11-16T16:53:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.982+0000 7fe23987fb80 1 -- 172.21.15.93:0/3549988815 wait complete. 2023-11-16T16:53:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.982+0000 7fe23987fb80 20 client.15648 trim_cache size 0 max 16384 2023-11-16T16:53:46.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:46 smithi159 ceph-mon[131528]: pgmap v500: 129 pgs: 129 active+clean; 307 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 627 op/s 2023-11-16T16:53:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.998+0000 7fe23987fb80 1 Processor -- start 2023-11-16T16:53:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.998+0000 7fe23987fb80 1 -- start start 2023-11-16T16:53:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.998+0000 7fe23987fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c742b6b0 0x5606c75d6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.998+0000 7fe23987fb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75cfdb0 0x5606c75e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.998+0000 7fe23987fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.998+0000 7fe23987fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c75e26b0 con 0x5606c742b6b0 2023-11-16T16:53:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.998+0000 7fe23987fb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c75e0970 con 0x5606c75cfdb0 2023-11-16T16:53:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.998+0000 7fe23987fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c75d0420 con 0x5606c75d0b10 2023-11-16T16:53:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.998+0000 7fe21effd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75cfdb0 0x5606c75e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.998+0000 7fe21effd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75cfdb0 0x5606c75e8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:41624/0 (socket says 172.21.15.93:41624) 2023-11-16T16:53:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.998+0000 7fe21effd700 1 -- 172.21.15.93:0/75549908 learned_addr learned my addr 172.21.15.93:0/75549908 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.999+0000 7fe21ffff700 1 --2- 172.21.15.93:0/75549908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c742b6b0 0x5606c75d6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.999+0000 7fe21effd700 1 -- 172.21.15.93:0/75549908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 msgr2=0x5606c75d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.999+0000 7fe21ffff700 1 --2- 172.21.15.93:0/75549908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.999+0000 7fe21effd700 1 --2- 172.21.15.93:0/75549908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75d3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.999+0000 7fe21effd700 1 -- 172.21.15.93:0/75549908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c742b6b0 msgr2=0x5606c75d6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.999+0000 7fe21effd700 1 --2- 172.21.15.93:0/75549908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c742b6b0 0x5606c75d6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.999+0000 7fe21effd700 1 -- 172.21.15.93:0/75549908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606c74e6e80 con 0x5606c75cfdb0 2023-11-16T16:53:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.999+0000 7fe21ffff700 1 --2- 172.21.15.93:0/75549908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75d3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:53:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.999+0000 7fe21effd700 1 --2- 172.21.15.93:0/75549908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75cfdb0 0x5606c75e8a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe214006ea0 tx=0x7fe214008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6267968273f7b4c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.999+0000 7fe2067fc700 1 -- 172.21.15.93:0/75549908 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21401b070 con 0x5606c75cfdb0 2023-11-16T16:53:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.999+0000 7fe2067fc700 1 -- 172.21.15.93:0/75549908 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe21401bc40 con 0x5606c75cfdb0 2023-11-16T16:53:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:46.999+0000 7fe2067fc700 1 -- 172.21.15.93:0/75549908 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214015ad0 con 0x5606c75cfdb0 2023-11-16T16:53:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.000+0000 7fe23987fb80 1 -- 172.21.15.93:0/75549908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75cfdb0 msgr2=0x5606c75e8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.000+0000 7fe23987fb80 1 --2- 172.21.15.93:0/75549908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75cfdb0 0x5606c75e8a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe214006ea0 tx=0x7fe214008040 comp rx=0 tx=0).stop 2023-11-16T16:53:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.000+0000 7fe23987fb80 1 -- 172.21.15.93:0/75549908 shutdown_connections 2023-11-16T16:53:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.000+0000 7fe23987fb80 1 --2- 172.21.15.93:0/75549908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c742b6b0 0x5606c75d6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.000+0000 7fe23987fb80 1 --2- 172.21.15.93:0/75549908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75d3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.000+0000 7fe23987fb80 1 --2- 172.21.15.93:0/75549908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c75cfdb0 0x5606c75e8a00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.000+0000 7fe23987fb80 1 -- 172.21.15.93:0/75549908 >> 172.21.15.93:0/75549908 conn(0x5606c75c4f50 msgr2=0x5606c75e01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.000+0000 7fe23987fb80 1 -- 172.21.15.93:0/75549908 shutdown_connections 2023-11-16T16:53:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.000+0000 7fe23987fb80 1 -- 172.21.15.93:0/75549908 wait complete. 2023-11-16T16:53:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.001+0000 7fe23987fb80 1 -- start start 2023-11-16T16:53:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.001+0000 7fe23987fb80 1 Processor -- start 2023-11-16T16:53:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.001+0000 7fe23987fb80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 0x5606c75e4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.001+0000 7fe23987fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c75cfdb0 0x5606c75e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.001+0000 7fe23987fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.001+0000 7fe23987fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c75e7ff0 con 0x5606c75cfdb0 2023-11-16T16:53:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.001+0000 7fe23987fb80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c75e8160 con 0x5606c742b6b0 2023-11-16T16:53:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.001+0000 7fe23987fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c75e8320 con 0x5606c75d0b10 2023-11-16T16:53:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.002+0000 7fe23987fb80 10 client.0 _ll_register_callbacks cb 0x7fe2140071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:53:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.002+0000 7fe21ffff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 0x5606c75e4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.002+0000 7fe21ffff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 0x5606c75e4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:41640/0 (socket says 172.21.15.93:41640) 2023-11-16T16:53:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.002+0000 7fe21ffff700 1 -- 172.21.15.93:0/2819994405 learned_addr learned my addr 172.21.15.93:0/2819994405 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:53:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.002+0000 7fe21effd700 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c75cfdb0 0x5606c75e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.002+0000 7fe21ffff700 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.002+0000 7fe21ffff700 1 -- 172.21.15.93:0/2819994405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 msgr2=0x5606c75e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.002+0000 7fe21ffff700 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.002+0000 7fe21ffff700 1 -- 172.21.15.93:0/2819994405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c75cfdb0 msgr2=0x5606c75e4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:53:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.002+0000 7fe21ffff700 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c75cfdb0 0x5606c75e4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.002+0000 7fe21ffff700 1 -- 172.21.15.93:0/2819994405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21400e040 con 0x5606c742b6b0 2023-11-16T16:53:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.003+0000 7fe21ffff700 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 0x5606c75e4550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe2180274d0 tx=0x7fe2180098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4325ff285a8e8509 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:53:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.003+0000 7fe1feffd700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:53:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.003+0000 7fe1feffd700 1 -- 172.21.15.93:0/2819994405 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800e070 con 0x5606c742b6b0 2023-11-16T16:53:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.003+0000 7fe23987fb80 1 -- 172.21.15.93:0/2819994405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606c75c2710 con 0x5606c742b6b0 2023-11-16T16:53:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.003+0000 7fe1feffd700 1 -- 172.21.15.93:0/2819994405 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fe218012940 con 0x5606c742b6b0 2023-11-16T16:53:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.003+0000 7fe1feffd700 1 -- 172.21.15.93:0/2819994405 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe218012b20 con 0x5606c742b6b0 2023-11-16T16:53:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.004+0000 7fe23987fb80 10 client.25709 fetch_fsmap learned FSMap version 29 2023-11-16T16:53:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.004+0000 7fe23987fb80 1 -- 172.21.15.93:0/2819994405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5606c7337c00 con 0x5606c742b6b0 2023-11-16T16:53:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.004+0000 7fe1feffd700 1 -- 172.21.15.93:0/2819994405 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5606c75c2710 con 0x5606c742b6b0 2023-11-16T16:53:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.004+0000 7fe1feffd700 1 -- 172.21.15.93:0/2819994405 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 29) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe21801fb10 con 0x5606c742b6b0 2023-11-16T16:53:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.004+0000 7fe23987fb80 10 client.25709 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:53:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.004+0000 7fe23987fb80 10 client.25709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:53:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.004+0000 7fe23987fb80 20 client.25709 populate_metadata read hostname 'smithi093' 2023-11-16T16:53:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.004+0000 7fe23987fb80 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5606c7641b60 0x5606c7643f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:53:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.004+0000 7fe23987fb80 4 client.25709 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:53:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.004+0000 7fe23987fb80 1 -- 172.21.15.93:0/2819994405 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5606c736ff10 con 0x5606c7641b60 2023-11-16T16:53:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.005+0000 7fe21effd700 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5606c7641b60 0x5606c7643f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:53:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.005+0000 7fe1feffd700 10 client.25709 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.005+0000 7fe21effd700 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5606c7641b60 0x5606c7643f70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71662e171120d436 server_cookie=a798ed4bb3a5352 in_seq=0 out_seq=0 2023-11-16T16:53:47.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:46 smithi125 ceph-mon[131076]: pgmap v500: 129 pgs: 129 active+clean; 307 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 627 op/s 2023-11-16T16:53:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.992+0000 7fe1feffd700 1 -- 172.21.15.93:0/2819994405 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5606c736ff10 con 0x5606c7641b60 2023-11-16T16:53:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.992+0000 7fe1feffd700 10 client.25709 handle_command_reply: tid=0 2023-11-16T16:53:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.992+0000 7fe23987fb80 1 client.25709 shutdown 2023-11-16T16:53:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.993+0000 7fe23987fb80 1 -- 172.21.15.93:0/2819994405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 msgr2=0x5606c75e4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.993+0000 7fe23987fb80 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 0x5606c75e4550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe2180274d0 tx=0x7fe2180098d0 comp rx=0 tx=0).stop 2023-11-16T16:53:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.995+0000 7fe23987fb80 1 -- 172.21.15.93:0/2819994405 shutdown_connections 2023-11-16T16:53:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.995+0000 7fe23987fb80 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x5606c7641b60 0x5606c7643f70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.995+0000 7fe23987fb80 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606c75d0b10 0x5606c75e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.995+0000 7fe23987fb80 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5606c742b6b0 0x5606c75e4550 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.995+0000 7fe23987fb80 1 --2- 172.21.15.93:0/2819994405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606c75cfdb0 0x5606c75e4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.995+0000 7fe23987fb80 1 -- 172.21.15.93:0/2819994405 >> 172.21.15.93:0/2819994405 conn(0x5606c75c4f50 msgr2=0x5606c75d6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.995+0000 7fe1feffd700 0 client.25709 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:53:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.995+0000 7fe23987fb80 1 -- 172.21.15.93:0/2819994405 shutdown_connections 2023-11-16T16:53:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.995+0000 7fe23987fb80 1 -- 172.21.15.93:0/2819994405 wait complete. 2023-11-16T16:53:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.996+0000 7fe23987fb80 20 client.25709 trim_cache size 0 max 16384 2023-11-16T16:53:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:53:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:53:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:53:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:53:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.997+0000 7fe225fff700 1 -- 172.21.15.93:0/517482791 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe2080a1150 msgr2=0x7fe2080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.997+0000 7fe225fff700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe2080a1150 0x7fe2080a3600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe21009b030 tx=0x7fe21009e040 comp rx=0 tx=0).stop 2023-11-16T16:53:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.997+0000 7fe225fff700 1 -- 172.21.15.93:0/517482791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2201002c0 msgr2=0x7fe2201982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:53:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.997+0000 7fe225fff700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2201002c0 0x7fe2201982b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe21000f730 tx=0x7fe21000f810 comp rx=0 tx=0).stop 2023-11-16T16:53:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.998+0000 7fe225fff700 1 -- 172.21.15.93:0/517482791 shutdown_connections 2023-11-16T16:53:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.998+0000 7fe225fff700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fe2080a1150 0x7fe2080a3600 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.998+0000 7fe225fff700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe220101e40 0x7fe22019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.998+0000 7fe225fff700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2201002c0 0x7fe2201982b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.998+0000 7fe225fff700 1 --2- 172.21.15.93:0/517482791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2201014b0 0x7fe2201987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:53:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:47.998+0000 7fe225fff700 1 -- 172.21.15.93:0/517482791 >> 172.21.15.93:0/517482791 conn(0x7fe2200fb8e0 msgr2=0x7fe220106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:53:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:48.000+0000 7fe225fff700 1 -- 172.21.15.93:0/517482791 shutdown_connections 2023-11-16T16:53:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:53:48.000+0000 7fe225fff700 1 -- 172.21.15.93:0/517482791 wait complete. 2023-11-16T16:53:48.485 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:53:48 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c[157316]: ignoring --setuser ceph since I am not root 2023-11-16T16:53:48.485 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:53:48 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c[157316]: ignoring --setgroup ceph since I am not root 2023-11-16T16:53:48.485 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:53:48 smithi159 ceph-mds[157340]: -- 172.21.15.159:0/2125597644 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562136d4b520 con 0x562136d20c00 2023-11-16T16:53:48.485 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:53:48 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c[157316]: starting mds.c at 2023-11-16T16:53:48.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:48 smithi093 ceph-mon[97863]: pgmap v501: 129 pgs: 129 active+clean; 308 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 570 op/s 2023-11-16T16:53:48.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:48 smithi093 ceph-mon[97863]: daemon mds.c restarted 2023-11-16T16:53:48.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:48 smithi093 ceph-mon[97863]: osdmap e94: 12 total, 12 up, 12 in 2023-11-16T16:53:48.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:48 smithi159 ceph-mon[131528]: pgmap v501: 129 pgs: 129 active+clean; 308 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 570 op/s 2023-11-16T16:53:48.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:48 smithi159 ceph-mon[131528]: daemon mds.c restarted 2023-11-16T16:53:48.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:48 smithi159 ceph-mon[131528]: osdmap e94: 12 total, 12 up, 12 in 2023-11-16T16:53:49.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:48 smithi125 ceph-mon[131076]: pgmap v501: 129 pgs: 129 active+clean; 308 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 570 op/s 2023-11-16T16:53:49.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:48 smithi125 ceph-mon[131076]: daemon mds.c restarted 2023-11-16T16:53:49.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:48 smithi125 ceph-mon[131076]: osdmap e94: 12 total, 12 up, 12 in 2023-11-16T16:53:49.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:49 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.159:6836/157086341,v1:172.21.15.159:6837/157086341] up:boot 2023-11-16T16:53:49.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:49 smithi093 ceph-mon[97863]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-11-16T16:53:49.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:49 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:53:49.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:49 smithi093 ceph-mon[97863]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:53:49.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:49 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.159:6836/157086341,v1:172.21.15.159:6837/157086341] up:boot 2023-11-16T16:53:49.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:49 smithi159 ceph-mon[131528]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-11-16T16:53:49.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:49 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:53:49.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:49 smithi159 ceph-mon[131528]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:53:50.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:49 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.159:6836/157086341,v1:172.21.15.159:6837/157086341] up:boot 2023-11-16T16:53:50.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:49 smithi125 ceph-mon[131076]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-11-16T16:53:50.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:49 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:53:50.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:49 smithi125 ceph-mon[131076]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:53:50.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:50 smithi093 ceph-mon[97863]: pgmap v503: 129 pgs: 129 active+clean; 316 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 590 op/s 2023-11-16T16:53:50.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:50 smithi159 ceph-mon[131528]: pgmap v503: 129 pgs: 129 active+clean; 316 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 590 op/s 2023-11-16T16:53:51.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:50 smithi125 ceph-mon[131076]: pgmap v503: 129 pgs: 129 active+clean; 316 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 590 op/s 2023-11-16T16:53:52.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:52 smithi093 ceph-mon[97863]: pgmap v504: 129 pgs: 129 active+clean; 309 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 600 op/s 2023-11-16T16:53:52.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:52 smithi159 ceph-mon[131528]: pgmap v504: 129 pgs: 129 active+clean; 309 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 600 op/s 2023-11-16T16:53:53.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:52 smithi125 ceph-mon[131076]: pgmap v504: 129 pgs: 129 active+clean; 309 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 600 op/s 2023-11-16T16:53:54.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:54 smithi093 ceph-mon[97863]: pgmap v505: 129 pgs: 129 active+clean; 309 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 580 op/s 2023-11-16T16:53:54.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:54 smithi159 ceph-mon[131528]: pgmap v505: 129 pgs: 129 active+clean; 309 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 580 op/s 2023-11-16T16:53:55.081 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:54 smithi125 ceph-mon[131076]: pgmap v505: 129 pgs: 129 active+clean; 309 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 580 op/s 2023-11-16T16:53:56.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:56 smithi159 ceph-mon[131528]: pgmap v506: 129 pgs: 129 active+clean; 313 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 587 op/s 2023-11-16T16:53:57.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:56 smithi125 ceph-mon[131076]: pgmap v506: 129 pgs: 129 active+clean; 313 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 587 op/s 2023-11-16T16:53:57.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:56 smithi093 ceph-mon[97863]: pgmap v506: 129 pgs: 129 active+clean; 313 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 587 op/s 2023-11-16T16:53:58.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:53:58 smithi159 ceph-mon[131528]: pgmap v507: 129 pgs: 129 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 598 op/s 2023-11-16T16:53:59.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:53:58 smithi125 ceph-mon[131076]: pgmap v507: 129 pgs: 129 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 598 op/s 2023-11-16T16:53:59.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:53:58 smithi093 ceph-mon[97863]: pgmap v507: 129 pgs: 129 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 598 op/s 2023-11-16T16:54:00.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:00 smithi159 ceph-mon[131528]: pgmap v508: 129 pgs: 129 active+clean; 325 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 632 op/s 2023-11-16T16:54:01.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:00 smithi125 ceph-mon[131076]: pgmap v508: 129 pgs: 129 active+clean; 325 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 632 op/s 2023-11-16T16:54:01.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:00 smithi093 ceph-mon[97863]: pgmap v508: 129 pgs: 129 active+clean; 325 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 632 op/s 2023-11-16T16:54:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.221+0000 7feb83fff700 1 -- 172.21.15.93:0/255793524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb7c1084c0 msgr2=0x7feb7c10ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.221+0000 7feb83fff700 1 --2- 172.21.15.93:0/255793524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb7c1084c0 0x7feb7c10ed60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feb7400b0a0 tx=0x7feb7400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.221+0000 7feb83fff700 1 -- 172.21.15.93:0/255793524 shutdown_connections 2023-11-16T16:54:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.221+0000 7feb83fff700 1 --2- 172.21.15.93:0/255793524 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb7c106c70 0x7feb7c107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.221+0000 7feb83fff700 1 --2- 172.21.15.93:0/255793524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb7c1084c0 0x7feb7c10ed60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.221+0000 7feb83fff700 1 --2- 172.21.15.93:0/255793524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb7c107b10 0x7feb7c107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.221+0000 7feb83fff700 1 -- 172.21.15.93:0/255793524 >> 172.21.15.93:0/255793524 conn(0x7feb7c0758d0 msgr2=0x7feb7c077d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:01.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.223+0000 7feb83fff700 1 -- 172.21.15.93:0/255793524 shutdown_connections 2023-11-16T16:54:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.225+0000 7feb83fff700 1 -- 172.21.15.93:0/255793524 wait complete. 2023-11-16T16:54:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.226+0000 7feb83fff700 1 Processor -- start 2023-11-16T16:54:01.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.226+0000 7feb83fff700 1 -- start start 2023-11-16T16:54:01.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.227+0000 7feb83fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb7c106c70 0x7feb7c07f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.227+0000 7feb83fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb7c107b10 0x7feb7c07f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.227+0000 7feb83fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb7c1084c0 0x7feb7c080810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.227+0000 7feb83fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c080de0 con 0x7feb7c107b10 2023-11-16T16:54:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.227+0000 7feb83fff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c080f50 con 0x7feb7c106c70 2023-11-16T16:54:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.227+0000 7feb83fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c081110 con 0x7feb7c1084c0 2023-11-16T16:54:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.227+0000 7feb8159a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb7c107b10 0x7feb7c07f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.228+0000 7feb8159a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb7c107b10 0x7feb7c07f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43984/0 (socket says 172.21.15.93:43984) 2023-11-16T16:54:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.228+0000 7feb8159a700 1 -- 172.21.15.93:0/1253681696 learned_addr learned my addr 172.21.15.93:0/1253681696 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.228+0000 7feb81d9b700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb7c106c70 0x7feb7c07f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.228+0000 7feb8259c700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb7c1084c0 0x7feb7c080810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.228+0000 7feb8159a700 1 -- 172.21.15.93:0/1253681696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb7c1084c0 msgr2=0x7feb7c080810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.228+0000 7feb8159a700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb7c1084c0 0x7feb7c080810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.228+0000 7feb8159a700 1 -- 172.21.15.93:0/1253681696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb7c106c70 msgr2=0x7feb7c07f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.228+0000 7feb8159a700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb7c106c70 0x7feb7c07f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.228+0000 7feb8159a700 1 -- 172.21.15.93:0/1253681696 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb74009d00 con 0x7feb7c107b10 2023-11-16T16:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.229+0000 7feb8259c700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb7c1084c0 0x7feb7c080810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:54:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.229+0000 7feb8159a700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb7c107b10 0x7feb7c07f7d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00d950 tx=0x7feb6c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.231+0000 7feb72ffd700 1 -- 172.21.15.93:0/1253681696 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c009940 con 0x7feb7c107b10 2023-11-16T16:54:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.231+0000 7feb72ffd700 1 -- 172.21.15.93:0/1253681696 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb6c010460 con 0x7feb7c107b10 2023-11-16T16:54:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.231+0000 7feb72ffd700 1 -- 172.21.15.93:0/1253681696 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c00f620 con 0x7feb7c107b10 2023-11-16T16:54:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.232+0000 7feb83fff700 1 -- 172.21.15.93:0/1253681696 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7c079c50 con 0x7feb7c107b10 2023-11-16T16:54:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.232+0000 7feb83fff700 1 -- 172.21.15.93:0/1253681696 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c079ac0 con 0x7feb7c107b10 2023-11-16T16:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.245+0000 7feb72ffd700 1 -- 172.21.15.93:0/1253681696 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feb6c00f780 con 0x7feb7c107b10 2023-11-16T16:54:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.246+0000 7feb97919b80 1 Processor -- start 2023-11-16T16:54:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.246+0000 7feb97919b80 1 -- start start 2023-11-16T16:54:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.247+0000 7feb97919b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf3366b0 0x564eaf4da870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.247+0000 7feb97919b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf4dadb0 0x564eaf4db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.247+0000 7feb97919b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.247+0000 7feb97919b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaf3e0000 con 0x564eaf4dadb0 2023-11-16T16:54:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.247+0000 7feb97919b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaf432840 con 0x564eaf3366b0 2023-11-16T16:54:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.247+0000 7feb97919b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaf42bcd0 con 0x564eaf4dbb10 2023-11-16T16:54:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.247+0000 7feb8259c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf3366b0 0x564eaf4da870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.247+0000 7feb8259c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf3366b0 0x564eaf4da870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:45440/0 (socket says 172.21.15.93:45440) 2023-11-16T16:54:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.247+0000 7feb8259c700 1 -- 172.21.15.93:0/4261178593 learned_addr learned my addr 172.21.15.93:0/4261178593 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.247+0000 7feb8159a700 1 --2- 172.21.15.93:0/4261178593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.247+0000 7feb81d9b700 1 --2- 172.21.15.93:0/4261178593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf4dadb0 0x564eaf4db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.248+0000 7feb8259c700 1 -- 172.21.15.93:0/4261178593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 msgr2=0x564eaf4dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.248+0000 7feb8259c700 1 --2- 172.21.15.93:0/4261178593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4dbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.248+0000 7feb8259c700 1 -- 172.21.15.93:0/4261178593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf4dadb0 msgr2=0x564eaf4db5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.248+0000 7feb8259c700 1 --2- 172.21.15.93:0/4261178593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf4dadb0 0x564eaf4db5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.248+0000 7feb8259c700 1 -- 172.21.15.93:0/4261178593 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eaf2ad5d0 con 0x564eaf3366b0 2023-11-16T16:54:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.248+0000 7feb8259c700 1 --2- 172.21.15.93:0/4261178593 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf3366b0 0x564eaf4da870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7feb7400b780 tx=0x7feb74009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d921a759fc60dd3a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.248+0000 7feb67fff700 1 -- 172.21.15.93:0/4261178593 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7400e050 con 0x564eaf3366b0 2023-11-16T16:54:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.248+0000 7feb67fff700 1 -- 172.21.15.93:0/4261178593 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb7400bdb0 con 0x564eaf3366b0 2023-11-16T16:54:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.248+0000 7feb67fff700 1 -- 172.21.15.93:0/4261178593 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74005400 con 0x564eaf3366b0 2023-11-16T16:54:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.250+0000 7feb97919b80 1 -- 172.21.15.93:0/4261178593 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf3366b0 msgr2=0x564eaf4da870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.250+0000 7feb97919b80 1 --2- 172.21.15.93:0/4261178593 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf3366b0 0x564eaf4da870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7feb7400b780 tx=0x7feb74009450 comp rx=0 tx=0).stop 2023-11-16T16:54:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.250+0000 7feb97919b80 1 -- 172.21.15.93:0/4261178593 shutdown_connections 2023-11-16T16:54:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.250+0000 7feb97919b80 1 --2- 172.21.15.93:0/4261178593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.250+0000 7feb97919b80 1 --2- 172.21.15.93:0/4261178593 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf3366b0 0x564eaf4da870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.250+0000 7feb97919b80 1 --2- 172.21.15.93:0/4261178593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf4dadb0 0x564eaf4db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.250+0000 7feb97919b80 1 -- 172.21.15.93:0/4261178593 >> 172.21.15.93:0/4261178593 conn(0x564eaf4cff50 msgr2=0x564eaf4d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:01.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.251+0000 7feb72ffd700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7feb680a12a0 0x7feb680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.252+0000 7feb8259c700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7feb680a12a0 0x7feb680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.253+0000 7feb72ffd700 1 -- 172.21.15.93:0/1253681696 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7feb6c0cca60 con 0x7feb7c107b10 2023-11-16T16:54:01.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.253+0000 7feb8259c700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7feb680a12a0 0x7feb680a3750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feb7400b780 tx=0x7feb7400b3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:01.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.253+0000 7feb97919b80 1 -- 172.21.15.93:0/4261178593 shutdown_connections 2023-11-16T16:54:01.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.253+0000 7feb97919b80 1 -- 172.21.15.93:0/4261178593 wait complete. 2023-11-16T16:54:01.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.254+0000 7feb97919b80 1 -- start start 2023-11-16T16:54:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.255+0000 7feb97919b80 1 Processor -- start 2023-11-16T16:54:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.255+0000 7feb97919b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf3366b0 0x564eaf4e3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb81d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf3366b0 0x564eaf4e3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb81d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf3366b0 0x564eaf4e3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43992/0 (socket says 172.21.15.93:43992) 2023-11-16T16:54:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb97919b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dadb0 0x564eaf4e4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb97919b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb97919b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaf3cd880 con 0x564eaf3366b0 2023-11-16T16:54:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb97919b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaf3cd520 con 0x564eaf4dadb0 2023-11-16T16:54:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb97919b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaf2af8c0 con 0x564eaf4dbb10 2023-11-16T16:54:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb8159a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dadb0 0x564eaf4e4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb97919b80 10 client.0 _ll_register_callbacks cb 0x564eaf3eb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:54:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb8159a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dadb0 0x564eaf4e4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:45450/0 (socket says 172.21.15.93:45450) 2023-11-16T16:54:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb8159a700 1 -- 172.21.15.93:0/869958741 learned_addr learned my addr 172.21.15.93:0/869958741 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb8159a700 1 -- 172.21.15.93:0/869958741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 msgr2=0x564eaf4eb1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:54:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb8159a700 1 --2- 172.21.15.93:0/869958741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4eb1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.256+0000 7feb8159a700 1 -- 172.21.15.93:0/869958741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf3366b0 msgr2=0x564eaf4e3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.257+0000 7feb8159a700 1 --2- 172.21.15.93:0/869958741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf3366b0 0x564eaf4e3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.257+0000 7feb8159a700 1 -- 172.21.15.93:0/869958741 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c10f5d0 con 0x564eaf4dadb0 2023-11-16T16:54:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.257+0000 7feb8159a700 1 --2- 172.21.15.93:0/869958741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dadb0 0x564eaf4e4170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feb6c095f40 tx=0x7feb6c099ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c9c706392ea0fd server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.257+0000 7feb657fa700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.257+0000 7feb657fa700 1 -- 172.21.15.93:0/869958741 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c09aea0 con 0x564eaf4dadb0 2023-11-16T16:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.257+0000 7feb657fa700 1 -- 172.21.15.93:0/869958741 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7feb6c099130 con 0x564eaf4dadb0 2023-11-16T16:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.257+0000 7feb97919b80 1 -- 172.21.15.93:0/869958741 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564eaf4cd710 con 0x564eaf4dadb0 2023-11-16T16:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.257+0000 7feb657fa700 1 -- 172.21.15.93:0/869958741 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb6c0a1a00 con 0x564eaf4dadb0 2023-11-16T16:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.257+0000 7feb97919b80 10 client.25727 fetch_fsmap learned FSMap version 31 2023-11-16T16:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.258+0000 7feb97919b80 1 -- 172.21.15.93:0/869958741 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564eaf3f2130 con 0x564eaf4dadb0 2023-11-16T16:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.258+0000 7feb657fa700 1 -- 172.21.15.93:0/869958741 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb6c0b1080 con 0x564eaf4dadb0 2023-11-16T16:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.258+0000 7feb657fa700 1 -- 172.21.15.93:0/869958741 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 31) v1 ==== 7494+0+0 (secure 0 0 0) 0x7feb6c009aa0 con 0x564eaf4dadb0 2023-11-16T16:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.258+0000 7feb97919b80 10 client.25727 fetch_fsmap finished waiting for FSMap version 31 2023-11-16T16:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.258+0000 7feb97919b80 10 client.25727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.259+0000 7feb97919b80 20 client.25727 populate_metadata read hostname 'smithi093' 2023-11-16T16:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.259+0000 7feb97919b80 1 --2- 172.21.15.93:0/869958741 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564eaf4efab0 0x564eaf4f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.259+0000 7feb97919b80 4 client.25727 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.259+0000 7feb97919b80 1 -- 172.21.15.93:0/869958741 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564eaf3ee040 con 0x564eaf4efab0 2023-11-16T16:54:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.286+0000 7feb81d9b700 1 --2- 172.21.15.93:0/869958741 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564eaf4efab0 0x564eaf4f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.287+0000 7feb81d9b700 1 --2- 172.21.15.93:0/869958741 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564eaf4efab0 0x564eaf4f1ec0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58279c754084bb7d server_cookie=8f2e61979f5b1ab4 in_seq=0 out_seq=0 2023-11-16T16:54:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.288+0000 7feb657fa700 10 client.25727 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.293+0000 7feb657fa700 1 -- 172.21.15.93:0/869958741 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564eaf3ee040 con 0x564eaf4efab0 2023-11-16T16:54:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.293+0000 7feb657fa700 10 client.25727 handle_command_reply: tid=0 2023-11-16T16:54:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.293+0000 7feb97919b80 1 client.25727 shutdown 2023-11-16T16:54:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.295+0000 7feb97919b80 1 -- 172.21.15.93:0/869958741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dadb0 msgr2=0x564eaf4e4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.295+0000 7feb97919b80 1 --2- 172.21.15.93:0/869958741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dadb0 0x564eaf4e4170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feb6c095f40 tx=0x7feb6c099ac0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.295+0000 7feb97919b80 1 -- 172.21.15.93:0/869958741 shutdown_connections 2023-11-16T16:54:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.295+0000 7feb97919b80 1 --2- 172.21.15.93:0/869958741 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564eaf4efab0 0x564eaf4f1ec0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.295+0000 7feb97919b80 1 --2- 172.21.15.93:0/869958741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf3366b0 0x564eaf4e3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.295+0000 7feb97919b80 1 --2- 172.21.15.93:0/869958741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4eb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.295+0000 7feb97919b80 1 --2- 172.21.15.93:0/869958741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dadb0 0x564eaf4e4170 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.295+0000 7feb97919b80 1 -- 172.21.15.93:0/869958741 >> 172.21.15.93:0/869958741 conn(0x564eaf4cff50 msgr2=0x564eaf4d47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.296+0000 7feb657fa700 0 client.25727 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.297+0000 7feb97919b80 1 -- 172.21.15.93:0/869958741 shutdown_connections 2023-11-16T16:54:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.297+0000 7feb97919b80 1 -- 172.21.15.93:0/869958741 wait complete. 2023-11-16T16:54:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.298+0000 7feb97919b80 20 client.25727 trim_cache size 0 max 16384 2023-11-16T16:54:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.335+0000 7feb97919b80 1 Processor -- start 2023-11-16T16:54:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.335+0000 7feb97919b80 1 -- start start 2023-11-16T16:54:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.336+0000 7feb97919b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf3366b0 0x564eaf4de960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.336+0000 7feb97919b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dadb0 0x564eaf4deea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.336+0000 7feb97919b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4eb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.336+0000 7feb97919b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaf4e30e0 con 0x564eaf3366b0 2023-11-16T16:54:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.336+0000 7feb97919b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaf4e37a0 con 0x564eaf4dadb0 2023-11-16T16:54:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.336+0000 7feb97919b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaf4ece20 con 0x564eaf4dbb10 2023-11-16T16:54:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.336+0000 7feb8259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4eb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.336+0000 7feb8259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4eb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:51844/0 (socket says 172.21.15.93:51844) 2023-11-16T16:54:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.336+0000 7feb8259c700 1 -- 172.21.15.93:0/2813705440 learned_addr learned my addr 172.21.15.93:0/2813705440 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.337+0000 7feb8259c700 1 -- 172.21.15.93:0/2813705440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dadb0 msgr2=0x564eaf4deea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.337+0000 7feb8259c700 1 --2- 172.21.15.93:0/2813705440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dadb0 0x564eaf4deea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.337+0000 7feb8259c700 1 -- 172.21.15.93:0/2813705440 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf3366b0 msgr2=0x564eaf4de960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.337+0000 7feb8259c700 1 --2- 172.21.15.93:0/2813705440 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf3366b0 0x564eaf4de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.337+0000 7feb8259c700 1 -- 172.21.15.93:0/2813705440 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eaf4d16f0 con 0x564eaf4dbb10 2023-11-16T16:54:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.337+0000 7feb8259c700 1 --2- 172.21.15.93:0/2813705440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4eb0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feb74009610 tx=0x7feb74009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3a88ad8edc89753 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.338+0000 7feb70ff9700 1 -- 172.21.15.93:0/2813705440 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74004d50 con 0x564eaf4dbb10 2023-11-16T16:54:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.338+0000 7feb70ff9700 1 -- 172.21.15.93:0/2813705440 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb740308f0 con 0x564eaf4dbb10 2023-11-16T16:54:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.338+0000 7feb70ff9700 1 -- 172.21.15.93:0/2813705440 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7401b960 con 0x564eaf4dbb10 2023-11-16T16:54:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.341+0000 7feb97919b80 1 -- 172.21.15.93:0/2813705440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 msgr2=0x564eaf4eb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.341+0000 7feb97919b80 1 --2- 172.21.15.93:0/2813705440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4eb0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feb74009610 tx=0x7feb74009450 comp rx=0 tx=0).stop 2023-11-16T16:54:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.341+0000 7feb97919b80 1 -- 172.21.15.93:0/2813705440 shutdown_connections 2023-11-16T16:54:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.341+0000 7feb97919b80 1 --2- 172.21.15.93:0/2813705440 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf3366b0 0x564eaf4de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.341+0000 7feb97919b80 1 --2- 172.21.15.93:0/2813705440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf4dbb10 0x564eaf4eb0d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.341+0000 7feb97919b80 1 --2- 172.21.15.93:0/2813705440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dadb0 0x564eaf4deea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.341+0000 7feb97919b80 1 -- 172.21.15.93:0/2813705440 >> 172.21.15.93:0/2813705440 conn(0x564eaf4cff50 msgr2=0x564eaf4e1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.343+0000 7feb97919b80 1 -- 172.21.15.93:0/2813705440 shutdown_connections 2023-11-16T16:54:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.344+0000 7feb97919b80 1 -- 172.21.15.93:0/2813705440 wait complete. 2023-11-16T16:54:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.345+0000 7feb97919b80 1 -- start start 2023-11-16T16:54:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.345+0000 7feb97919b80 1 Processor -- start 2023-11-16T16:54:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.348+0000 7feb97919b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf3366b0 0x564eaf4e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.349+0000 7feb97919b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf4dadb0 0x564eaf53f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.349+0000 7feb81d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf3366b0 0x564eaf4e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.349+0000 7feb81d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf3366b0 0x564eaf4e90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:51858/0 (socket says 172.21.15.93:51858) 2023-11-16T16:54:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.349+0000 7feb81d9b700 1 -- 172.21.15.93:0/2762312014 learned_addr learned my addr 172.21.15.93:0/2762312014 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.350+0000 7feb97919b80 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dbb10 0x564eaf541e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.350+0000 7feb8159a700 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf4dadb0 0x564eaf53f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.350+0000 7feb97919b80 1 -- 172.21.15.93:0/2762312014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaf4e1660 con 0x564eaf4dadb0 2023-11-16T16:54:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.350+0000 7feb97919b80 1 -- 172.21.15.93:0/2762312014 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaf5423d0 con 0x564eaf4dbb10 2023-11-16T16:54:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.350+0000 7feb97919b80 1 -- 172.21.15.93:0/2762312014 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaf542510 con 0x564eaf3366b0 2023-11-16T16:54:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.350+0000 7feb97919b80 10 client.0 _ll_register_callbacks cb 0x7feb6c099580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:54:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.350+0000 7feb8259c700 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dbb10 0x564eaf541e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.351+0000 7feb8159a700 1 -- 172.21.15.93:0/2762312014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf3366b0 msgr2=0x564eaf4e90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.351+0000 7feb8159a700 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf3366b0 0x564eaf4e90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.351+0000 7feb8159a700 1 -- 172.21.15.93:0/2762312014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dbb10 msgr2=0x564eaf541e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.351+0000 7feb8159a700 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dbb10 0x564eaf541e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.351+0000 7feb8159a700 1 -- 172.21.15.93:0/2762312014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7402b070 con 0x564eaf4dadb0 2023-11-16T16:54:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.356+0000 7feb8159a700 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf4dadb0 0x564eaf53f940 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009e00 tx=0x7feb6c0a1f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf35800fa6396b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.356+0000 7feb657fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:54:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.357+0000 7feb657fa700 1 -- 172.21.15.93:0/2762312014 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c014070 con 0x564eaf4dadb0 2023-11-16T16:54:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.357+0000 7feb657fa700 1 -- 172.21.15.93:0/2762312014 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7feb6c0a1a00 con 0x564eaf4dadb0 2023-11-16T16:54:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.358+0000 7feb97919b80 1 -- 172.21.15.93:0/2762312014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb7400bdb0 con 0x564eaf4dadb0 2023-11-16T16:54:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.359+0000 7feb657fa700 1 -- 172.21.15.93:0/2762312014 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb6c09aea0 con 0x564eaf4dadb0 2023-11-16T16:54:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.359+0000 7feb97919b80 10 client.15672 fetch_fsmap learned FSMap version 31 2023-11-16T16:54:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.359+0000 7feb97919b80 1 -- 172.21.15.93:0/2762312014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564eaf259d80 con 0x564eaf4dadb0 2023-11-16T16:54:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.361+0000 7feb657fa700 1 -- 172.21.15.93:0/2762312014 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb7400bdb0 con 0x564eaf4dadb0 2023-11-16T16:54:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.361+0000 7feb657fa700 1 -- 172.21.15.93:0/2762312014 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 31) v1 ==== 7494+0+0 (secure 0 0 0) 0x7feb6c095830 con 0x564eaf4dadb0 2023-11-16T16:54:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.361+0000 7feb97919b80 10 client.15672 fetch_fsmap finished waiting for FSMap version 31 2023-11-16T16:54:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.361+0000 7feb97919b80 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:54:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.361+0000 7feb97919b80 20 client.15672 populate_metadata read hostname 'smithi093' 2023-11-16T16:54:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.361+0000 7feb97919b80 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564eaf543b60 0x564eaf545f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.361+0000 7feb97919b80 4 client.15672 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:54:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.361+0000 7feb97919b80 1 -- 172.21.15.93:0/2762312014 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564eaf3fa600 con 0x564eaf543b60 2023-11-16T16:54:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.362+0000 7feb81d9b700 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564eaf543b60 0x564eaf545f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.363+0000 7feb81d9b700 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564eaf543b60 0x564eaf545f70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547ef24562acf5dc server_cookie=d9e21f39b15b24c0 in_seq=0 out_seq=0 2023-11-16T16:54:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:01.363+0000 7feb657fa700 10 client.15672 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.218+0000 7feb657fa700 1 -- 172.21.15.93:0/2762312014 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564eaf3fa600 con 0x564eaf543b60 2023-11-16T16:54:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb657fa700 10 client.15672 handle_command_reply: tid=0 2023-11-16T16:54:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb97919b80 1 client.15672 shutdown 2023-11-16T16:54:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb97919b80 1 -- 172.21.15.93:0/2762312014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf4dadb0 msgr2=0x564eaf53f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb97919b80 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf4dadb0 0x564eaf53f940 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009e00 tx=0x7feb6c0a1f50 comp rx=0 tx=0).stop 2023-11-16T16:54:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb97919b80 1 -- 172.21.15.93:0/2762312014 shutdown_connections 2023-11-16T16:54:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb97919b80 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x564eaf543b60 0x564eaf545f70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb97919b80 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564eaf4dbb10 0x564eaf541e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb97919b80 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564eaf3366b0 0x564eaf4e90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb97919b80 1 --2- 172.21.15.93:0/2762312014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eaf4dadb0 0x564eaf53f940 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb97919b80 1 -- 172.21.15.93:0/2762312014 >> 172.21.15.93:0/2762312014 conn(0x564eaf4cff50 msgr2=0x564eaf4e1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb657fa700 0 client.15672 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb97919b80 1 -- 172.21.15.93:0/2762312014 shutdown_connections 2023-11-16T16:54:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb97919b80 1 -- 172.21.15.93:0/2762312014 wait complete. 2023-11-16T16:54:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.219+0000 7feb97919b80 20 client.15672 trim_cache size 0 max 16384 2023-11-16T16:54:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:54:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:54:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:54:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:54:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.220+0000 7feb83fff700 1 -- 172.21.15.93:0/1253681696 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7feb680a12a0 msgr2=0x7feb680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.220+0000 7feb83fff700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7feb680a12a0 0x7feb680a3750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feb7400b780 tx=0x7feb7400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:54:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.220+0000 7feb83fff700 1 -- 172.21.15.93:0/1253681696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb7c107b10 msgr2=0x7feb7c07f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.220+0000 7feb83fff700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb7c107b10 0x7feb7c07f7d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00d950 tx=0x7feb6c00dc60 comp rx=0 tx=0).stop 2023-11-16T16:54:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.220+0000 7feb83fff700 1 -- 172.21.15.93:0/1253681696 shutdown_connections 2023-11-16T16:54:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.220+0000 7feb83fff700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7feb680a12a0 0x7feb680a3750 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.220+0000 7feb83fff700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb7c1084c0 0x7feb7c080810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.220+0000 7feb83fff700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb7c106c70 0x7feb7c07f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.220+0000 7feb83fff700 1 --2- 172.21.15.93:0/1253681696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb7c107b10 0x7feb7c07f7d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.220+0000 7feb83fff700 1 -- 172.21.15.93:0/1253681696 >> 172.21.15.93:0/1253681696 conn(0x7feb7c0758d0 msgr2=0x7feb7c10af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.220+0000 7feb83fff700 1 -- 172.21.15.93:0/1253681696 shutdown_connections 2023-11-16T16:54:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:02.221+0000 7feb83fff700 1 -- 172.21.15.93:0/1253681696 wait complete. 2023-11-16T16:54:02.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:02 smithi159 ceph-mon[131528]: pgmap v509: 129 pgs: 129 active+clean; 330 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 593 op/s 2023-11-16T16:54:03.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:02 smithi125 ceph-mon[131076]: pgmap v509: 129 pgs: 129 active+clean; 330 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 593 op/s 2023-11-16T16:54:03.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:02 smithi093 ceph-mon[97863]: pgmap v509: 129 pgs: 129 active+clean; 330 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 593 op/s 2023-11-16T16:54:04.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:04 smithi159 ceph-mon[131528]: pgmap v510: 129 pgs: 129 active+clean; 334 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 598 op/s 2023-11-16T16:54:05.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:04 smithi125 ceph-mon[131076]: pgmap v510: 129 pgs: 129 active+clean; 334 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 598 op/s 2023-11-16T16:54:05.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:04 smithi093 ceph-mon[97863]: pgmap v510: 129 pgs: 129 active+clean; 334 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 598 op/s 2023-11-16T16:54:06.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:06 smithi159 ceph-mon[131528]: pgmap v511: 129 pgs: 129 active+clean; 339 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 628 op/s 2023-11-16T16:54:07.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:06 smithi125 ceph-mon[131076]: pgmap v511: 129 pgs: 129 active+clean; 339 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 628 op/s 2023-11-16T16:54:07.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:06 smithi093 ceph-mon[97863]: pgmap v511: 129 pgs: 129 active+clean; 339 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 628 op/s 2023-11-16T16:54:08.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:08 smithi159 ceph-mon[131528]: pgmap v512: 129 pgs: 129 active+clean; 335 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 615 op/s 2023-11-16T16:54:09.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:08 smithi125 ceph-mon[131076]: pgmap v512: 129 pgs: 129 active+clean; 335 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 615 op/s 2023-11-16T16:54:09.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:08 smithi093 ceph-mon[97863]: pgmap v512: 129 pgs: 129 active+clean; 335 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 615 op/s 2023-11-16T16:54:09.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:09 smithi159 ceph-mon[131528]: pgmap v513: 129 pgs: 129 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 604 op/s 2023-11-16T16:54:10.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:09 smithi125 ceph-mon[131076]: pgmap v513: 129 pgs: 129 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 604 op/s 2023-11-16T16:54:10.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:09 smithi093 ceph-mon[97863]: pgmap v513: 129 pgs: 129 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 604 op/s 2023-11-16T16:54:12.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:12 smithi159 ceph-mon[131528]: pgmap v514: 129 pgs: 129 active+clean; 345 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 567 op/s 2023-11-16T16:54:12.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:12 smithi125 ceph-mon[131076]: pgmap v514: 129 pgs: 129 active+clean; 345 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 567 op/s 2023-11-16T16:54:12.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:12 smithi093 ceph-mon[97863]: pgmap v514: 129 pgs: 129 active+clean; 345 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 567 op/s 2023-11-16T16:54:14.735 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:14 smithi159 ceph-mon[131528]: pgmap v515: 129 pgs: 129 active+clean; 345 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 567 op/s 2023-11-16T16:54:14.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:14 smithi125 ceph-mon[131076]: pgmap v515: 129 pgs: 129 active+clean; 345 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 567 op/s 2023-11-16T16:54:14.899 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:14 smithi093 ceph-mon[97863]: pgmap v515: 129 pgs: 129 active+clean; 345 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 567 op/s 2023-11-16T16:54:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.497+0000 7fb94f1ff700 1 -- 172.21.15.93:0/4022327843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb948105f00 msgr2=0x7fb948108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.497+0000 7fb94f1ff700 1 --2- 172.21.15.93:0/4022327843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb948105f00 0x7fb948108360 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb94000b0a0 tx=0x7fb94000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.497+0000 7fb94f1ff700 1 -- 172.21.15.93:0/4022327843 shutdown_connections 2023-11-16T16:54:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.497+0000 7fb94f1ff700 1 --2- 172.21.15.93:0/4022327843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb948105f00 0x7fb948108360 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.497+0000 7fb94f1ff700 1 --2- 172.21.15.93:0/4022327843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb948100cc0 0x7fb9481030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.498+0000 7fb94f1ff700 1 --2- 172.21.15.93:0/4022327843 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9481035e0 0x7fb9481059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.498+0000 7fb94f1ff700 1 -- 172.21.15.93:0/4022327843 >> 172.21.15.93:0/4022327843 conn(0x7fb9480fa720 msgr2=0x7fb9480fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.498+0000 7fb94f1ff700 1 -- 172.21.15.93:0/4022327843 shutdown_connections 2023-11-16T16:54:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.499+0000 7fb94f1ff700 1 -- 172.21.15.93:0/4022327843 wait complete. 2023-11-16T16:54:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.501+0000 7fb94f1ff700 1 Processor -- start 2023-11-16T16:54:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.502+0000 7fb94f1ff700 1 -- start start 2023-11-16T16:54:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.502+0000 7fb94f1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb948100cc0 0x7fb948198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.503+0000 7fb94f1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9481035e0 0x7fb948198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.503+0000 7fb94f1ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb948105f00 0x7fb94819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.503+0000 7fb94f1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94819feb0 con 0x7fb9481035e0 2023-11-16T16:54:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.503+0000 7fb94f1ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94819fff0 con 0x7fb948105f00 2023-11-16T16:54:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.503+0000 7fb94f1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9481a0180 con 0x7fb948100cc0 2023-11-16T16:54:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.503+0000 7fb94cf9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb948100cc0 0x7fb948198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.503+0000 7fb94cf9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb948100cc0 0x7fb948198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:49716/0 (socket says 172.21.15.93:49716) 2023-11-16T16:54:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.503+0000 7fb94cf9b700 1 -- 172.21.15.93:0/1587474568 learned_addr learned my addr 172.21.15.93:0/1587474568 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.503+0000 7fb94d79c700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb948105f00 0x7fb94819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.504+0000 7fb94cf9b700 1 -- 172.21.15.93:0/1587474568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb948105f00 msgr2=0x7fb94819f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.504+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb948105f00 0x7fb94819f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.504+0000 7fb94cf9b700 1 -- 172.21.15.93:0/1587474568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9481035e0 msgr2=0x7fb948198870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:54:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.504+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9481035e0 0x7fb948198870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.504+0000 7fb94cf9b700 1 -- 172.21.15.93:0/1587474568 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb940009d00 con 0x7fb948100cc0 2023-11-16T16:54:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.504+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb948100cc0 0x7fb948198330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00d950 tx=0x7fb93c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.506+0000 7fb945ffb700 1 -- 172.21.15.93:0/1587474568 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c009940 con 0x7fb948100cc0 2023-11-16T16:54:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.506+0000 7fb945ffb700 1 -- 172.21.15.93:0/1587474568 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb93c010460 con 0x7fb948100cc0 2023-11-16T16:54:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.506+0000 7fb945ffb700 1 -- 172.21.15.93:0/1587474568 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00f620 con 0x7fb948100cc0 2023-11-16T16:54:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.506+0000 7fb94f1ff700 1 -- 172.21.15.93:0/1587474568 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94810bca0 con 0x7fb948100cc0 2023-11-16T16:54:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.506+0000 7fb94f1ff700 1 -- 172.21.15.93:0/1587474568 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94810bb10 con 0x7fb948100cc0 2023-11-16T16:54:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.513+0000 7fb945ffb700 1 -- 172.21.15.93:0/1587474568 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb93c009aa0 con 0x7fb948100cc0 2023-11-16T16:54:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.515+0000 7fb962a59b80 1 Processor -- start 2023-11-16T16:54:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.516+0000 7fb962a59b80 1 -- start start 2023-11-16T16:54:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.516+0000 7fb962a59b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbce616b0 0x55dbbd00e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.516+0000 7fb962a59b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00e5c0 0x55dbbd00ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.516+0000 7fb962a59b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00f320 0x55dbbd00f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.516+0000 7fb962a59b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbcf0b000 con 0x55dbbce616b0 2023-11-16T16:54:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.516+0000 7fb962a59b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbcf5d840 con 0x55dbbd00f320 2023-11-16T16:54:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.516+0000 7fb962a59b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbcf56cd0 con 0x55dbbd00e5c0 2023-11-16T16:54:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.517+0000 7fb94cf9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00e5c0 0x55dbbd00ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.517+0000 7fb94cf9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00e5c0 0x55dbbd00ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:49732/0 (socket says 172.21.15.93:49732) 2023-11-16T16:54:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.517+0000 7fb94cf9b700 1 -- 172.21.15.93:0/598215135 learned_addr learned my addr 172.21.15.93:0/598215135 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.518+0000 7fb94d79c700 1 --2- 172.21.15.93:0/598215135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbce616b0 0x55dbbd00e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.519+0000 7fb947fff700 1 --2- 172.21.15.93:0/598215135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00f320 0x55dbbd00f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.519+0000 7fb945ffb700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb9380a12d0 0x7fb9380a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.519+0000 7fb94cf9b700 1 -- 172.21.15.93:0/598215135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00f320 msgr2=0x55dbbd00f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.519+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/598215135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00f320 0x55dbbd00f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.519+0000 7fb94cf9b700 1 -- 172.21.15.93:0/598215135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbce616b0 msgr2=0x55dbbd00e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.519+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/598215135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbce616b0 0x55dbbd00e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.519+0000 7fb94cf9b700 1 -- 172.21.15.93:0/598215135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbbcdd85d0 con 0x55dbbd00e5c0 2023-11-16T16:54:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.519+0000 7fb947fff700 1 --2- 172.21.15.93:0/598215135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00f320 0x55dbbd00f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:54:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.519+0000 7fb94d79c700 1 --2- 172.21.15.93:0/598215135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbce616b0 0x55dbbd00e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:54:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.520+0000 7fb945ffb700 1 -- 172.21.15.93:0/1587474568 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fb93c0cca10 con 0x7fb948100cc0 2023-11-16T16:54:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.520+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/598215135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00e5c0 0x55dbbd00ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0ab8e0 tx=0x7fb93c095ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e77d94432d2ee3f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.520+0000 7fb94d79c700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb9380a12d0 0x7fb9380a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.521+0000 7fb932ffd700 1 -- 172.21.15.93:0/598215135 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c099820 con 0x55dbbd00e5c0 2023-11-16T16:54:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.522+0000 7fb932ffd700 1 -- 172.21.15.93:0/598215135 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb93c099590 con 0x55dbbd00e5c0 2023-11-16T16:54:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.522+0000 7fb932ffd700 1 -- 172.21.15.93:0/598215135 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0980b0 con 0x55dbbd00e5c0 2023-11-16T16:54:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.525+0000 7fb94d79c700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb9380a12d0 0x7fb9380a3780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb940001220 tx=0x7fb94000b3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.525+0000 7fb962a59b80 1 -- 172.21.15.93:0/598215135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00e5c0 msgr2=0x55dbbd00ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.525+0000 7fb962a59b80 1 --2- 172.21.15.93:0/598215135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00e5c0 0x55dbbd00ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0ab8e0 tx=0x7fb93c095ec0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.525+0000 7fb962a59b80 1 -- 172.21.15.93:0/598215135 shutdown_connections 2023-11-16T16:54:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.525+0000 7fb962a59b80 1 --2- 172.21.15.93:0/598215135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbce616b0 0x55dbbd00e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.526+0000 7fb962a59b80 1 --2- 172.21.15.93:0/598215135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00f320 0x55dbbd00f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.526+0000 7fb962a59b80 1 --2- 172.21.15.93:0/598215135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00e5c0 0x55dbbd00ede0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.526+0000 7fb962a59b80 1 -- 172.21.15.93:0/598215135 >> 172.21.15.93:0/598215135 conn(0x55dbbcffaf50 msgr2=0x55dbbd000100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.529+0000 7fb962a59b80 1 -- 172.21.15.93:0/598215135 shutdown_connections 2023-11-16T16:54:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.529+0000 7fb962a59b80 1 -- 172.21.15.93:0/598215135 wait complete. 2023-11-16T16:54:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.530+0000 7fb962a59b80 1 -- start start 2023-11-16T16:54:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.530+0000 7fb962a59b80 1 Processor -- start 2023-11-16T16:54:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.531+0000 7fb962a59b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbce616b0 0x55dbbd006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.531+0000 7fb962a59b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00e5c0 0x55dbbd006f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.531+0000 7fb962a59b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbd00f320 0x55dbbd00bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.531+0000 7fb962a59b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbcef8880 con 0x55dbbd00f320 2023-11-16T16:54:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.531+0000 7fb962a59b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbcef8520 con 0x55dbbce616b0 2023-11-16T16:54:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.531+0000 7fb962a59b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbcdda8c0 con 0x55dbbd00e5c0 2023-11-16T16:54:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.531+0000 7fb962a59b80 10 client.0 _ll_register_callbacks cb 0x55dbbcf16720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:54:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.532+0000 7fb94cf9b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbce616b0 0x55dbbd006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.532+0000 7fb94cf9b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbce616b0 0x55dbbd006a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:41252/0 (socket says 172.21.15.93:41252) 2023-11-16T16:54:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.532+0000 7fb94cf9b700 1 -- 172.21.15.93:0/2722079571 learned_addr learned my addr 172.21.15.93:0/2722079571 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.533+0000 7fb94cf9b700 1 -- 172.21.15.93:0/2722079571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00e5c0 msgr2=0x55dbbd006f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:54:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.533+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/2722079571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00e5c0 0x55dbbd006f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.533+0000 7fb94cf9b700 1 -- 172.21.15.93:0/2722079571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbd00f320 msgr2=0x55dbbd00bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.533+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/2722079571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbd00f320 0x55dbbd00bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.533+0000 7fb94cf9b700 1 -- 172.21.15.93:0/2722079571 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94810bb10 con 0x55dbbce616b0 2023-11-16T16:54:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.534+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/2722079571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbce616b0 0x55dbbd006a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb93c095f50 tx=0x7fb93c095e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d0c4c8e30e84ca server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.535+0000 7fb927fff700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:54:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.535+0000 7fb927fff700 1 -- 172.21.15.93:0/2722079571 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c099820 con 0x55dbbce616b0 2023-11-16T16:54:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.535+0000 7fb927fff700 1 -- 172.21.15.93:0/2722079571 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fb93c0ab6d0 con 0x55dbbce616b0 2023-11-16T16:54:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.535+0000 7fb962a59b80 1 -- 172.21.15.93:0/2722079571 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbbcff8710 con 0x55dbbce616b0 2023-11-16T16:54:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.537+0000 7fb927fff700 1 -- 172.21.15.93:0/2722079571 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb93c099a50 con 0x55dbbce616b0 2023-11-16T16:54:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.538+0000 7fb962a59b80 10 client.25739 fetch_fsmap learned FSMap version 31 2023-11-16T16:54:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.538+0000 7fb962a59b80 1 -- 172.21.15.93:0/2722079571 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55dbbcf1d130 con 0x55dbbce616b0 2023-11-16T16:54:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.541+0000 7fb927fff700 1 -- 172.21.15.93:0/2722079571 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dbbcff8710 con 0x55dbbce616b0 2023-11-16T16:54:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.541+0000 7fb927fff700 1 -- 172.21.15.93:0/2722079571 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 31) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fb93c0d4050 con 0x55dbbce616b0 2023-11-16T16:54:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.541+0000 7fb962a59b80 10 client.25739 fetch_fsmap finished waiting for FSMap version 31 2023-11-16T16:54:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.541+0000 7fb962a59b80 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:54:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.543+0000 7fb962a59b80 20 client.25739 populate_metadata read hostname 'smithi093' 2023-11-16T16:54:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.543+0000 7fb962a59b80 1 --2- 172.21.15.93:0/2722079571 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55dbbd02b560 0x55dbbd02d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.543+0000 7fb962a59b80 4 client.25739 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:54:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.543+0000 7fb962a59b80 1 -- 172.21.15.93:0/2722079571 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbbcf19040 con 0x55dbbd02b560 2023-11-16T16:54:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.543+0000 7fb947fff700 1 --2- 172.21.15.93:0/2722079571 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55dbbd02b560 0x55dbbd02d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.544+0000 7fb947fff700 1 --2- 172.21.15.93:0/2722079571 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55dbbd02b560 0x55dbbd02d940 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c660bb70d7eb8bd server_cookie=38560f57d2bb3486 in_seq=0 out_seq=0 2023-11-16T16:54:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.544+0000 7fb927fff700 10 client.25739 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.546+0000 7fb927fff700 1 -- 172.21.15.93:0/2722079571 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dbbcf19040 con 0x55dbbd02b560 2023-11-16T16:54:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.546+0000 7fb927fff700 10 client.25739 handle_command_reply: tid=0 2023-11-16T16:54:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.546+0000 7fb962a59b80 1 client.25739 shutdown 2023-11-16T16:54:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.547+0000 7fb962a59b80 1 -- 172.21.15.93:0/2722079571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbce616b0 msgr2=0x55dbbd006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.547+0000 7fb962a59b80 1 --2- 172.21.15.93:0/2722079571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbce616b0 0x55dbbd006a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb93c095f50 tx=0x7fb93c095e40 comp rx=0 tx=0).stop 2023-11-16T16:54:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.547+0000 7fb962a59b80 1 -- 172.21.15.93:0/2722079571 shutdown_connections 2023-11-16T16:54:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.547+0000 7fb962a59b80 1 --2- 172.21.15.93:0/2722079571 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55dbbd02b560 0x55dbbd02d940 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.547+0000 7fb962a59b80 1 --2- 172.21.15.93:0/2722079571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbd00f320 0x55dbbd00bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.547+0000 7fb962a59b80 1 --2- 172.21.15.93:0/2722079571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbce616b0 0x55dbbd006a20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.547+0000 7fb962a59b80 1 --2- 172.21.15.93:0/2722079571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00e5c0 0x55dbbd006f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.547+0000 7fb962a59b80 1 -- 172.21.15.93:0/2722079571 >> 172.21.15.93:0/2722079571 conn(0x55dbbcffaf50 msgr2=0x55dbbcfff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.547+0000 7fb927fff700 0 client.25739 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.548+0000 7fb962a59b80 1 -- 172.21.15.93:0/2722079571 shutdown_connections 2023-11-16T16:54:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.548+0000 7fb962a59b80 1 -- 172.21.15.93:0/2722079571 wait complete. 2023-11-16T16:54:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.549+0000 7fb962a59b80 20 client.25739 trim_cache size 0 max 16384 2023-11-16T16:54:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.586+0000 7fb962a59b80 1 Processor -- start 2023-11-16T16:54:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.588+0000 7fb962a59b80 1 -- start start 2023-11-16T16:54:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.588+0000 7fb962a59b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbce616b0 0x55dbbd032270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.588+0000 7fb962a59b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00e5c0 0x55dbbd03b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.588+0000 7fb962a59b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbd00f320 0x55dbbd005710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.588+0000 7fb962a59b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbd00dbe0 con 0x55dbbd00f320 2023-11-16T16:54:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.588+0000 7fb962a59b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbd00ebd0 con 0x55dbbd00e5c0 2023-11-16T16:54:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.588+0000 7fb962a59b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbd0154f0 con 0x55dbbce616b0 2023-11-16T16:54:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.589+0000 7fb94cf9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbce616b0 0x55dbbd032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.589+0000 7fb94cf9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbce616b0 0x55dbbd032270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:49752/0 (socket says 172.21.15.93:49752) 2023-11-16T16:54:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.589+0000 7fb94cf9b700 1 -- 172.21.15.93:0/41139055 learned_addr learned my addr 172.21.15.93:0/41139055 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.589+0000 7fb947fff700 1 --2- 172.21.15.93:0/41139055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00e5c0 0x55dbbd03b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.589+0000 7fb94cf9b700 1 -- 172.21.15.93:0/41139055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00e5c0 msgr2=0x55dbbd03b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.589+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/41139055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00e5c0 0x55dbbd03b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.589+0000 7fb94cf9b700 1 -- 172.21.15.93:0/41139055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbd00f320 msgr2=0x55dbbd005710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.589+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/41139055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbd00f320 0x55dbbd005710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.589+0000 7fb94cf9b700 1 -- 172.21.15.93:0/41139055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbbcf1ce80 con 0x55dbbce616b0 2023-11-16T16:54:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.590+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/41139055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbce616b0 0x55dbbd032270 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0999e0 tx=0x7fb93c0ca190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=434db9bfba85fb7d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.590+0000 7fb9337fe700 1 -- 172.21.15.93:0/41139055 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0d21a0 con 0x55dbbce616b0 2023-11-16T16:54:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.591+0000 7fb9337fe700 1 -- 172.21.15.93:0/41139055 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb93c099820 con 0x55dbbce616b0 2023-11-16T16:54:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.592+0000 7fb962a59b80 1 -- 172.21.15.93:0/41139055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbce616b0 msgr2=0x55dbbd032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.593+0000 7fb9337fe700 1 -- 172.21.15.93:0/41139055 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c099a50 con 0x55dbbce616b0 2023-11-16T16:54:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.593+0000 7fb962a59b80 1 --2- 172.21.15.93:0/41139055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbce616b0 0x55dbbd032270 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0999e0 tx=0x7fb93c0ca190 comp rx=0 tx=0).stop 2023-11-16T16:54:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.594+0000 7fb962a59b80 1 -- 172.21.15.93:0/41139055 shutdown_connections 2023-11-16T16:54:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.595+0000 7fb962a59b80 1 --2- 172.21.15.93:0/41139055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbd00f320 0x55dbbd005710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.595+0000 7fb962a59b80 1 --2- 172.21.15.93:0/41139055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbce616b0 0x55dbbd032270 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.595+0000 7fb962a59b80 1 --2- 172.21.15.93:0/41139055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00e5c0 0x55dbbd03b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.595+0000 7fb962a59b80 1 -- 172.21.15.93:0/41139055 >> 172.21.15.93:0/41139055 conn(0x55dbbcffaf50 msgr2=0x55dbbd0000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.595+0000 7fb962a59b80 1 -- 172.21.15.93:0/41139055 shutdown_connections 2023-11-16T16:54:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.595+0000 7fb962a59b80 1 -- 172.21.15.93:0/41139055 wait complete. 2023-11-16T16:54:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.595+0000 7fb962a59b80 1 -- start start 2023-11-16T16:54:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.596+0000 7fb962a59b80 1 Processor -- start 2023-11-16T16:54:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.599+0000 7fb962a59b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbce616b0 0x55dbbd038500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.599+0000 7fb962a59b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00e5c0 0x55dbbd038a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.600+0000 7fb962a59b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00f320 0x55dbbd039a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.600+0000 7fb947fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00e5c0 0x55dbbd038a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.600+0000 7fb947fff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00e5c0 0x55dbbd038a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:41272/0 (socket says 172.21.15.93:41272) 2023-11-16T16:54:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.600+0000 7fb947fff700 1 -- 172.21.15.93:0/707809940 learned_addr learned my addr 172.21.15.93:0/707809940 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.600+0000 7fb962a59b80 1 -- 172.21.15.93:0/707809940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbd039f90 con 0x55dbbce616b0 2023-11-16T16:54:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.600+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbce616b0 0x55dbbd038500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.600+0000 7fb962a59b80 1 -- 172.21.15.93:0/707809940 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbd03a100 con 0x55dbbd00e5c0 2023-11-16T16:54:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.600+0000 7fb962a59b80 1 -- 172.21.15.93:0/707809940 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbd03a2c0 con 0x55dbbd00f320 2023-11-16T16:54:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.600+0000 7fb962a59b80 10 client.0 _ll_register_callbacks cb 0x7fb93c095d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:54:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.600+0000 7fb94d79c700 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00f320 0x55dbbd039a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.601+0000 7fb94cf9b700 1 -- 172.21.15.93:0/707809940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00f320 msgr2=0x55dbbd039a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.601+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00f320 0x55dbbd039a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.601+0000 7fb94cf9b700 1 -- 172.21.15.93:0/707809940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00e5c0 msgr2=0x55dbbd038a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.601+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00e5c0 0x55dbbd038a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.601+0000 7fb94cf9b700 1 -- 172.21.15.93:0/707809940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbbcf1d130 con 0x55dbbce616b0 2023-11-16T16:54:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.601+0000 7fb94cf9b700 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbce616b0 0x55dbbd038500 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0ca4b0 tx=0x7fb93c010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582d8179c66a3c50 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.601+0000 7fb927fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:54:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.602+0000 7fb927fff700 1 -- 172.21.15.93:0/707809940 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0d21a0 con 0x55dbbce616b0 2023-11-16T16:54:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.602+0000 7fb927fff700 1 -- 172.21.15.93:0/707809940 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fb93c099a50 con 0x55dbbce616b0 2023-11-16T16:54:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.602+0000 7fb962a59b80 1 -- 172.21.15.93:0/707809940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb93c099590 con 0x55dbbce616b0 2023-11-16T16:54:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.602+0000 7fb927fff700 1 -- 172.21.15.93:0/707809940 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb93c0993b0 con 0x55dbbce616b0 2023-11-16T16:54:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.602+0000 7fb962a59b80 10 client.15678 fetch_fsmap learned FSMap version 31 2023-11-16T16:54:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.602+0000 7fb962a59b80 1 -- 172.21.15.93:0/707809940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55dbbcd6dc00 con 0x55dbbce616b0 2023-11-16T16:54:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.603+0000 7fb927fff700 1 -- 172.21.15.93:0/707809940 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb93c099590 con 0x55dbbce616b0 2023-11-16T16:54:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.603+0000 7fb927fff700 1 -- 172.21.15.93:0/707809940 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 31) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fb93c0a7070 con 0x55dbbce616b0 2023-11-16T16:54:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.603+0000 7fb962a59b80 10 client.15678 fetch_fsmap finished waiting for FSMap version 31 2023-11-16T16:54:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.603+0000 7fb962a59b80 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:54:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.603+0000 7fb962a59b80 20 client.15678 populate_metadata read hostname 'smithi093' 2023-11-16T16:54:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.603+0000 7fb962a59b80 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55dbbd07fb40 0x55dbbd081f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.603+0000 7fb962a59b80 4 client.15678 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:54:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.603+0000 7fb962a59b80 1 -- 172.21.15.93:0/707809940 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dbbcf25600 con 0x55dbbd07fb40 2023-11-16T16:54:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.603+0000 7fb947fff700 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55dbbd07fb40 0x55dbbd081f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.604+0000 7fb927fff700 10 client.15678 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.604+0000 7fb947fff700 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55dbbd07fb40 0x55dbbd081f20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d651a739bc51c77 server_cookie=92b9040b168f2e39 in_seq=0 out_seq=0 2023-11-16T16:54:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.681+0000 7fb927fff700 1 -- 172.21.15.93:0/707809940 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dbbcf25600 con 0x55dbbd07fb40 2023-11-16T16:54:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.681+0000 7fb927fff700 10 client.15678 handle_command_reply: tid=0 2023-11-16T16:54:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.681+0000 7fb962a59b80 1 client.15678 shutdown 2023-11-16T16:54:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.683+0000 7fb962a59b80 1 -- 172.21.15.93:0/707809940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbce616b0 msgr2=0x55dbbd038500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.683+0000 7fb962a59b80 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbce616b0 0x55dbbd038500 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0ca4b0 tx=0x7fb93c010bb0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.683+0000 7fb962a59b80 1 -- 172.21.15.93:0/707809940 shutdown_connections 2023-11-16T16:54:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.683+0000 7fb962a59b80 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55dbbd07fb40 0x55dbbd081f20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.683+0000 7fb962a59b80 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dbbce616b0 0x55dbbd038500 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.683+0000 7fb962a59b80 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbbd00f320 0x55dbbd039a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.683+0000 7fb962a59b80 1 --2- 172.21.15.93:0/707809940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dbbd00e5c0 0x55dbbd038a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.683+0000 7fb962a59b80 1 -- 172.21.15.93:0/707809940 >> 172.21.15.93:0/707809940 conn(0x55dbbcffaf50 msgr2=0x55dbbd032270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.684+0000 7fb927fff700 0 client.15678 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.684+0000 7fb962a59b80 1 -- 172.21.15.93:0/707809940 shutdown_connections 2023-11-16T16:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.684+0000 7fb962a59b80 1 -- 172.21.15.93:0/707809940 wait complete. 2023-11-16T16:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.686+0000 7fb962a59b80 20 client.15678 trim_cache size 0 max 16384 2023-11-16T16:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:54:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.687+0000 7fb94f1ff700 1 -- 172.21.15.93:0/1587474568 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb9380a12d0 msgr2=0x7fb9380a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.687+0000 7fb94f1ff700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb9380a12d0 0x7fb9380a3780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb940001220 tx=0x7fb94000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.687+0000 7fb94f1ff700 1 -- 172.21.15.93:0/1587474568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb948100cc0 msgr2=0x7fb948198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.687+0000 7fb94f1ff700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb948100cc0 0x7fb948198330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00d950 tx=0x7fb93c00dc60 comp rx=0 tx=0).stop 2023-11-16T16:54:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.688+0000 7fb94f1ff700 1 -- 172.21.15.93:0/1587474568 shutdown_connections 2023-11-16T16:54:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.688+0000 7fb94f1ff700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb9380a12d0 0x7fb9380a3780 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.688+0000 7fb94f1ff700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb948105f00 0x7fb94819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.688+0000 7fb94f1ff700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb948100cc0 0x7fb948198330 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.688+0000 7fb94f1ff700 1 --2- 172.21.15.93:0/1587474568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9481035e0 0x7fb948198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.688+0000 7fb94f1ff700 1 -- 172.21.15.93:0/1587474568 >> 172.21.15.93:0/1587474568 conn(0x7fb9480fa720 msgr2=0x7fb948101940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.689+0000 7fb94f1ff700 1 -- 172.21.15.93:0/1587474568 shutdown_connections 2023-11-16T16:54:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:15.689+0000 7fb94f1ff700 1 -- 172.21.15.93:0/1587474568 wait complete. 2023-11-16T16:54:16.734 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:16 smithi159 ceph-mon[131528]: pgmap v516: 129 pgs: 129 active+clean; 349 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 541 op/s 2023-11-16T16:54:16.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:16 smithi125 ceph-mon[131076]: pgmap v516: 129 pgs: 129 active+clean; 349 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 541 op/s 2023-11-16T16:54:16.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:16 smithi093 ceph-mon[97863]: pgmap v516: 129 pgs: 129 active+clean; 349 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 541 op/s 2023-11-16T16:54:18.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:18 smithi125 ceph-mon[131076]: pgmap v517: 129 pgs: 129 active+clean; 354 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 478 op/s 2023-11-16T16:54:18.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:18 smithi093 ceph-mon[97863]: pgmap v517: 129 pgs: 129 active+clean; 354 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 478 op/s 2023-11-16T16:54:18.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:18 smithi159 ceph-mon[131528]: pgmap v517: 129 pgs: 129 active+clean; 354 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 478 op/s 2023-11-16T16:54:19.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:19 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:54:19.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:19 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:54:19.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:19 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:54:20.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:20 smithi125 ceph-mon[131076]: pgmap v518: 129 pgs: 129 active+clean; 350 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 479 op/s 2023-11-16T16:54:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:20 smithi093 ceph-mon[97863]: pgmap v518: 129 pgs: 129 active+clean; 350 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 479 op/s 2023-11-16T16:54:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:20 smithi159 ceph-mon[131528]: pgmap v518: 129 pgs: 129 active+clean; 350 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 479 op/s 2023-11-16T16:54:21.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:54:21.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:54:21.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:54:21.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:54:21.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:54:21.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:54:21.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:54:21.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:54:21.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:54:22.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:22 smithi125 ceph-mon[131076]: pgmap v519: 129 pgs: 129 active+clean; 357 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 507 op/s 2023-11-16T16:54:22.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:22 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:54:22.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:22 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:54:22.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:22 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:54:22.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:22 smithi125 ceph-mon[131076]: Service mds.all has no related daemon already placed. 2023-11-16T16:54:22.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:22 smithi093 ceph-mon[97863]: pgmap v519: 129 pgs: 129 active+clean; 357 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 507 op/s 2023-11-16T16:54:22.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:22 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:54:22.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:22 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:54:22.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:22 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:54:22.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:22 smithi093 ceph-mon[97863]: Service mds.all has no related daemon already placed. 2023-11-16T16:54:22.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:22 smithi159 ceph-mon[131528]: pgmap v519: 129 pgs: 129 active+clean; 357 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 507 op/s 2023-11-16T16:54:22.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:22 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:54:22.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:22 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:54:22.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:22 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:54:22.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:22 smithi159 ceph-mon[131528]: Service mds.all has no related daemon already placed. 2023-11-16T16:54:24.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:24 smithi125 ceph-mon[131076]: pgmap v520: 129 pgs: 129 active+clean; 358 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 482 op/s 2023-11-16T16:54:24.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:24 smithi093 ceph-mon[97863]: pgmap v520: 129 pgs: 129 active+clean; 358 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 482 op/s 2023-11-16T16:54:24.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:24 smithi159 ceph-mon[131528]: pgmap v520: 129 pgs: 129 active+clean; 358 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 482 op/s 2023-11-16T16:54:26.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:26 smithi125 ceph-mon[131076]: pgmap v521: 129 pgs: 129 active+clean; 358 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 452 op/s 2023-11-16T16:54:26.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:26 smithi093 ceph-mon[97863]: pgmap v521: 129 pgs: 129 active+clean; 358 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 452 op/s 2023-11-16T16:54:26.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:26 smithi159 ceph-mon[131528]: pgmap v521: 129 pgs: 129 active+clean; 358 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 452 op/s 2023-11-16T16:54:28.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:28 smithi125 ceph-mon[131076]: pgmap v522: 129 pgs: 129 active+clean; 363 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 423 op/s 2023-11-16T16:54:28.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:28 smithi093 ceph-mon[97863]: pgmap v522: 129 pgs: 129 active+clean; 363 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 423 op/s 2023-11-16T16:54:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.957+0000 7f1b303ff700 1 -- 172.21.15.93:0/270079809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b28071cc0 msgr2=0x7f1b28072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.957+0000 7f1b303ff700 1 --2- 172.21.15.93:0/270079809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b28071cc0 0x7f1b28072110 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009990 tx=0x7f1b18009ca0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.957+0000 7f1b303ff700 1 -- 172.21.15.93:0/270079809 shutdown_connections 2023-11-16T16:54:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.957+0000 7f1b303ff700 1 --2- 172.21.15.93:0/270079809 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b28072650 0x7f1b28111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.957+0000 7f1b303ff700 1 --2- 172.21.15.93:0/270079809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b28107d60 0x7f1b28108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.957+0000 7f1b303ff700 1 --2- 172.21.15.93:0/270079809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b28071cc0 0x7f1b28072110 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.957+0000 7f1b303ff700 1 -- 172.21.15.93:0/270079809 >> 172.21.15.93:0/270079809 conn(0x7f1b2806d5e0 msgr2=0x7f1b2806fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.958+0000 7f1b303ff700 1 -- 172.21.15.93:0/270079809 shutdown_connections 2023-11-16T16:54:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.958+0000 7f1b303ff700 1 -- 172.21.15.93:0/270079809 wait complete. 2023-11-16T16:54:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.959+0000 7f1b303ff700 1 Processor -- start 2023-11-16T16:54:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.960+0000 7f1b303ff700 1 -- start start 2023-11-16T16:54:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.960+0000 7f1b303ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b28071cc0 0x7f1b281a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.960+0000 7f1b303ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b28072650 0x7f1b281a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.960+0000 7f1b303ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b28107d60 0x7f1b281ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.961+0000 7f1b2d99a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b28072650 0x7f1b281a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.961+0000 7f1b2d99a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b28072650 0x7f1b281a52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:55938/0 (socket says 172.21.15.93:55938) 2023-11-16T16:54:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.961+0000 7f1b2e99c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b28107d60 0x7f1b281ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.961+0000 7f1b2e99c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b28107d60 0x7f1b281ac360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:34168/0 (socket says 172.21.15.93:34168) 2023-11-16T16:54:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.961+0000 7f1b2e99c700 1 -- 172.21.15.93:0/734896209 learned_addr learned my addr 172.21.15.93:0/734896209 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.961+0000 7f1b303ff700 1 -- 172.21.15.93:0/734896209 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b281ac930 con 0x7f1b28071cc0 2023-11-16T16:54:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.961+0000 7f1b303ff700 1 -- 172.21.15.93:0/734896209 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b281aca70 con 0x7f1b28072650 2023-11-16T16:54:28.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.961+0000 7f1b303ff700 1 -- 172.21.15.93:0/734896209 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b281acc00 con 0x7f1b28107d60 2023-11-16T16:54:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.962+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b28071cc0 0x7f1b281a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.962+0000 7f1b2e19b700 1 -- 172.21.15.93:0/734896209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b28107d60 msgr2=0x7f1b281ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.962+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b28107d60 0x7f1b281ac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.962+0000 7f1b2e19b700 1 -- 172.21.15.93:0/734896209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b28072650 msgr2=0x7f1b281a52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.962+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b28072650 0x7f1b281a52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.962+0000 7f1b2e19b700 1 -- 172.21.15.93:0/734896209 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180095f0 con 0x7f1b28071cc0 2023-11-16T16:54:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.963+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b28071cc0 0x7f1b281a4db0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400e9c0 tx=0x7f1b2400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.963+0000 7f1b1f7fe700 1 -- 172.21.15.93:0/734896209 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b24004220 con 0x7f1b28071cc0 2023-11-16T16:54:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.964+0000 7f1b1f7fe700 1 -- 172.21.15.93:0/734896209 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b24004380 con 0x7f1b28071cc0 2023-11-16T16:54:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.964+0000 7f1b303ff700 1 -- 172.21.15.93:0/734896209 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b28116620 con 0x7f1b28071cc0 2023-11-16T16:54:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.964+0000 7f1b303ff700 1 -- 172.21.15.93:0/734896209 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b28116490 con 0x7f1b28071cc0 2023-11-16T16:54:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.964+0000 7f1b1f7fe700 1 -- 172.21.15.93:0/734896209 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b24003be0 con 0x7f1b28071cc0 2023-11-16T16:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.970+0000 7f1b43d83b80 1 Processor -- start 2023-11-16T16:54:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.971+0000 7f1b43d83b80 1 -- start start 2023-11-16T16:54:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.971+0000 7f1b43d83b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e82681f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.971+0000 7f1b43d83b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e82681fdb0 0x55e8268205d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.971+0000 7f1b43d83b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 0x55e826820f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.971+0000 7f1b43d83b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e826725000 con 0x55e82681fdb0 2023-11-16T16:54:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.971+0000 7f1b43d83b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e826777840 con 0x55e82667b6b0 2023-11-16T16:54:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.971+0000 7f1b43d83b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e826770cd0 con 0x55e826820b10 2023-11-16T16:54:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.971+0000 7f1b2e99c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e82681f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.972+0000 7f1b2e99c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e82681f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:55954/0 (socket says 172.21.15.93:55954) 2023-11-16T16:54:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.972+0000 7f1b2e99c700 1 -- 172.21.15.93:0/1089219083 learned_addr learned my addr 172.21.15.93:0/1089219083 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.972+0000 7f1b2d99a700 1 --2- 172.21.15.93:0/1089219083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 0x55e826820f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.972+0000 7f1b2e99c700 1 -- 172.21.15.93:0/1089219083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 msgr2=0x55e826820f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.972+0000 7f1b2e99c700 1 --2- 172.21.15.93:0/1089219083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 0x55e826820f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.972+0000 7f1b2e99c700 1 -- 172.21.15.93:0/1089219083 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e82681fdb0 msgr2=0x55e8268205d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.972+0000 7f1b2e99c700 1 --2- 172.21.15.93:0/1089219083 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e82681fdb0 0x55e8268205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.972+0000 7f1b2e99c700 1 -- 172.21.15.93:0/1089219083 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8265f25d0 con 0x55e82667b6b0 2023-11-16T16:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.973+0000 7f1b2d99a700 1 --2- 172.21.15.93:0/1089219083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 0x55e826820f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.974+0000 7f1b2e99c700 1 --2- 172.21.15.93:0/1089219083 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e82681f870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000e690 tx=0x7f1b20009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ee454cc65778dd server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.975+0000 7f1b1cbfe700 1 -- 172.21.15.93:0/1089219083 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2000e040 con 0x55e82667b6b0 2023-11-16T16:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.975+0000 7f1b1cbfe700 1 -- 172.21.15.93:0/1089219083 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b2000f040 con 0x55e82667b6b0 2023-11-16T16:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.975+0000 7f1b1cbfe700 1 -- 172.21.15.93:0/1089219083 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20004870 con 0x55e82667b6b0 2023-11-16T16:54:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.978+0000 7f1b1f7fe700 1 -- 172.21.15.93:0/734896209 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1b2400c7c0 con 0x7f1b28071cc0 2023-11-16T16:54:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.979+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1089219083 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 msgr2=0x55e82681f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.979+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1089219083 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e82681f870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000e690 tx=0x7f1b20009920 comp rx=0 tx=0).stop 2023-11-16T16:54:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.981+0000 7f1b1f7fe700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1b140a11d0 0x7f1b140a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.981+0000 7f1b1f7fe700 1 -- 172.21.15.93:0/734896209 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f1b24014070 con 0x7f1b28071cc0 2023-11-16T16:54:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.982+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1089219083 shutdown_connections 2023-11-16T16:54:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.982+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1089219083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 0x55e826820f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.982+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1089219083 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e82681f870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.982+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1089219083 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e82681fdb0 0x55e8268205d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.982+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1089219083 >> 172.21.15.93:0/1089219083 conn(0x55e826814f50 msgr2=0x55e82681a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.982+0000 7f1b2e99c700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1b140a11d0 0x7f1b140a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:28.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:28 smithi159 ceph-mon[131528]: pgmap v522: 129 pgs: 129 active+clean; 363 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 423 op/s 2023-11-16T16:54:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.983+0000 7f1b2e99c700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1b140a11d0 0x7f1b140a3680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1b20004310 tx=0x7f1b20009f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.983+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1089219083 shutdown_connections 2023-11-16T16:54:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.983+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1089219083 wait complete. 2023-11-16T16:54:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.985+0000 7f1b43d83b80 1 -- start start 2023-11-16T16:54:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.987+0000 7f1b43d83b80 1 Processor -- start 2023-11-16T16:54:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.987+0000 7f1b43d83b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e826828ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.987+0000 7f1b43d83b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e82681fdb0 0x55e8268291e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.988+0000 7f1b43d83b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 0x55e826830220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.988+0000 7f1b43d83b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e826712880 con 0x55e82681fdb0 2023-11-16T16:54:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.988+0000 7f1b43d83b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e826712520 con 0x55e82667b6b0 2023-11-16T16:54:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.988+0000 7f1b43d83b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8265f48c0 con 0x55e826820b10 2023-11-16T16:54:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.988+0000 7f1b43d83b80 10 client.0 _ll_register_callbacks cb 0x55e826730720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:54:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.988+0000 7f1b2d99a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e82681fdb0 0x55e8268291e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.988+0000 7f1b2d99a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e82681fdb0 0x55e8268291e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44796/0 (socket says 172.21.15.93:44796) 2023-11-16T16:54:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.988+0000 7f1b2d99a700 1 -- 172.21.15.93:0/2408723966 learned_addr learned my addr 172.21.15.93:0/2408723966 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.988+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e826828ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.989+0000 7f1b2e99c700 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 0x55e826830220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.989+0000 7f1b2d99a700 1 -- 172.21.15.93:0/2408723966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 msgr2=0x55e826830220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.989+0000 7f1b2d99a700 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 0x55e826830220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.989+0000 7f1b2d99a700 1 -- 172.21.15.93:0/2408723966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 msgr2=0x55e826828ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.989+0000 7f1b2d99a700 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e826828ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.989+0000 7f1b2d99a700 1 -- 172.21.15.93:0/2408723966 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2000e370 con 0x55e82681fdb0 2023-11-16T16:54:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.990+0000 7f1b2d99a700 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e82681fdb0 0x55e8268291e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1b18015000 tx=0x7f1b18005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2953fa13ede2aa server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.990+0000 7f1b11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:54:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.992+0000 7f1b11ffb700 1 -- 172.21.15.93:0/2408723966 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1801d070 con 0x55e82681fdb0 2023-11-16T16:54:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.992+0000 7f1b11ffb700 1 -- 172.21.15.93:0/2408723966 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f1b18004500 con 0x55e82681fdb0 2023-11-16T16:54:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.992+0000 7f1b43d83b80 1 -- 172.21.15.93:0/2408723966 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e826812710 con 0x55e82681fdb0 2023-11-16T16:54:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.993+0000 7f1b11ffb700 1 -- 172.21.15.93:0/2408723966 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b180046c0 con 0x55e82681fdb0 2023-11-16T16:54:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.993+0000 7f1b43d83b80 10 client.15687 fetch_fsmap learned FSMap version 31 2023-11-16T16:54:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.993+0000 7f1b43d83b80 1 -- 172.21.15.93:0/2408723966 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e826737130 con 0x55e82681fdb0 2023-11-16T16:54:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.995+0000 7f1b11ffb700 1 -- 172.21.15.93:0/2408723966 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e826812710 con 0x55e82681fdb0 2023-11-16T16:54:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.995+0000 7f1b11ffb700 1 -- 172.21.15.93:0/2408723966 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 31) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1b1800f860 con 0x55e82681fdb0 2023-11-16T16:54:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.995+0000 7f1b43d83b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 31 2023-11-16T16:54:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.995+0000 7f1b43d83b80 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:54:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.995+0000 7f1b43d83b80 20 client.15687 populate_metadata read hostname 'smithi093' 2023-11-16T16:54:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.996+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55e826834b80 0x55e826836f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.996+0000 7f1b43d83b80 4 client.15687 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:54:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.996+0000 7f1b43d83b80 1 -- 172.21.15.93:0/2408723966 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e826733040 con 0x55e826834b80 2023-11-16T16:54:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:28.999+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55e826834b80 0x55e826836f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.001+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55e826834b80 0x55e826836f90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ee789612a4499e server_cookie=f2c8fbf4070a7bb4 in_seq=0 out_seq=0 2023-11-16T16:54:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.001+0000 7f1b11ffb700 10 client.15687 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.002+0000 7f1b11ffb700 1 -- 172.21.15.93:0/2408723966 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e826733040 con 0x55e826834b80 2023-11-16T16:54:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.002+0000 7f1b11ffb700 10 client.15687 handle_command_reply: tid=0 2023-11-16T16:54:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.002+0000 7f1b43d83b80 1 client.15687 shutdown 2023-11-16T16:54:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.002+0000 7f1b43d83b80 1 -- 172.21.15.93:0/2408723966 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e82681fdb0 msgr2=0x55e8268291e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.002+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e82681fdb0 0x55e8268291e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1b18015000 tx=0x7f1b18005e80 comp rx=0 tx=0).stop 2023-11-16T16:54:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.003+0000 7f1b43d83b80 1 -- 172.21.15.93:0/2408723966 shutdown_connections 2023-11-16T16:54:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.003+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55e826834b80 0x55e826836f90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.003+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 0x55e826830220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.003+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e826828ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.003+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/2408723966 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e82681fdb0 0x55e8268291e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.003+0000 7f1b43d83b80 1 -- 172.21.15.93:0/2408723966 >> 172.21.15.93:0/2408723966 conn(0x55e826814f50 msgr2=0x55e8268197e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.003+0000 7f1b11ffb700 0 client.15687 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.003+0000 7f1b43d83b80 1 -- 172.21.15.93:0/2408723966 shutdown_connections 2023-11-16T16:54:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.004+0000 7f1b43d83b80 1 -- 172.21.15.93:0/2408723966 wait complete. 2023-11-16T16:54:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.004+0000 7f1b43d83b80 20 client.15687 trim_cache size 0 max 16384 2023-11-16T16:54:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.048+0000 7f1b43d83b80 1 Processor -- start 2023-11-16T16:54:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.048+0000 7f1b43d83b80 1 -- start start 2023-11-16T16:54:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.051+0000 7f1b43d83b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e82667b6b0 0x55e826826e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.051+0000 7f1b43d83b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e826820b10 0x55e826827390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b43d83b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e8268293b0 0x55e826830140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b43d83b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e826829950 con 0x55e8268293b0 2023-11-16T16:54:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b43d83b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e826812250 con 0x55e826820b10 2023-11-16T16:54:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b43d83b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e826820450 con 0x55e82667b6b0 2023-11-16T16:54:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b2e19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e82667b6b0 0x55e826826e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b2e19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e82667b6b0 0x55e826826e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:34200/0 (socket says 172.21.15.93:34200) 2023-11-16T16:54:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b2e19b700 1 -- 172.21.15.93:0/1830246714 learned_addr learned my addr 172.21.15.93:0/1830246714 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b2e99c700 1 --2- 172.21.15.93:0/1830246714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e8268293b0 0x55e826830140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b2e19b700 1 -- 172.21.15.93:0/1830246714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e826820b10 msgr2=0x55e826827390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/1830246714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e826820b10 0x55e826827390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b2e19b700 1 -- 172.21.15.93:0/1830246714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e8268293b0 msgr2=0x55e826830140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/1830246714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e8268293b0 0x55e826830140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.052+0000 7f1b2e19b700 1 -- 172.21.15.93:0/1830246714 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8268166f0 con 0x55e82667b6b0 2023-11-16T16:54:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.053+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/1830246714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e82667b6b0 0x55e826826e50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1b24099e70 tx=0x7f1b240cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=254719c628be6ec5 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.054+0000 7f1b1d3ff700 1 -- 172.21.15.93:0/1830246714 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b240b2070 con 0x55e82667b6b0 2023-11-16T16:54:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.054+0000 7f1b1d3ff700 1 -- 172.21.15.93:0/1830246714 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b240b8310 con 0x55e82667b6b0 2023-11-16T16:54:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.054+0000 7f1b1d3ff700 1 -- 172.21.15.93:0/1830246714 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b240adbe0 con 0x55e82667b6b0 2023-11-16T16:54:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.056+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1830246714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e82667b6b0 msgr2=0x55e826826e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.056+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1830246714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e82667b6b0 0x55e826826e50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1b24099e70 tx=0x7f1b240cc040 comp rx=0 tx=0).stop 2023-11-16T16:54:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.057+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1830246714 shutdown_connections 2023-11-16T16:54:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.057+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1830246714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e8268293b0 0x55e826830140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.057+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1830246714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e82667b6b0 0x55e826826e50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.057+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1830246714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e826820b10 0x55e826827390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.057+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1830246714 >> 172.21.15.93:0/1830246714 conn(0x55e826814f50 msgr2=0x55e826828ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.057+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1830246714 shutdown_connections 2023-11-16T16:54:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.057+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1830246714 wait complete. 2023-11-16T16:54:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.058+0000 7f1b43d83b80 1 -- start start 2023-11-16T16:54:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.059+0000 7f1b43d83b80 1 Processor -- start 2023-11-16T16:54:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.061+0000 7f1b43d83b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e82682fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.061+0000 7f1b43d83b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 0x55e82682ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.061+0000 7f1b43d83b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e8268293b0 0x55e82682c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.061+0000 7f1b2e19b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e82682fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.061+0000 7f1b43d83b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e82682caa0 con 0x55e8268293b0 2023-11-16T16:54:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.061+0000 7f1b43d83b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e82682cc10 con 0x55e82667b6b0 2023-11-16T16:54:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.061+0000 7f1b43d83b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e82682cdd0 con 0x55e826820b10 2023-11-16T16:54:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.061+0000 7f1b2e19b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e82682fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:55986/0 (socket says 172.21.15.93:55986) 2023-11-16T16:54:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.061+0000 7f1b2e19b700 1 -- 172.21.15.93:0/1062443703 learned_addr learned my addr 172.21.15.93:0/1062443703 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.062+0000 7f1b43d83b80 10 client.0 _ll_register_callbacks cb 0x7f1b1800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:54:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.062+0000 7f1b2e19b700 1 -- 172.21.15.93:0/1062443703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 msgr2=0x55e82682ffc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:54:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.062+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/1062443703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 0x55e82682ffc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.062+0000 7f1b2e19b700 1 -- 172.21.15.93:0/1062443703 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e8268293b0 msgr2=0x55e82682c520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-16T16:54:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.062+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/1062443703 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e8268293b0 0x55e82682c520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.062+0000 7f1b2e19b700 1 -- 172.21.15.93:0/1062443703 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b28116490 con 0x55e82667b6b0 2023-11-16T16:54:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.062+0000 7f1b2e19b700 1 --2- 172.21.15.93:0/1062443703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e82682fa80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400c5b0 tx=0x7f1b240b83c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ec50bdd7a0990ce server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.062+0000 7f1b11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-11-16T16:54:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.063+0000 7f1b11ffb700 1 -- 172.21.15.93:0/1062443703 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b240b2050 con 0x55e82667b6b0 2023-11-16T16:54:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.063+0000 7f1b11ffb700 1 -- 172.21.15.93:0/1062443703 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f1b24098020 con 0x55e82667b6b0 2023-11-16T16:54:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.063+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1062443703 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e82673f600 con 0x55e82667b6b0 2023-11-16T16:54:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.064+0000 7f1b11ffb700 1 -- 172.21.15.93:0/1062443703 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b240adbe0 con 0x55e82667b6b0 2023-11-16T16:54:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.064+0000 7f1b43d83b80 10 client.25757 fetch_fsmap learned FSMap version 31 2023-11-16T16:54:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.064+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1062443703 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e82659ed80 con 0x55e82667b6b0 2023-11-16T16:54:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.064+0000 7f1b11ffb700 1 -- 172.21.15.93:0/1062443703 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e82673f600 con 0x55e82667b6b0 2023-11-16T16:54:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.065+0000 7f1b11ffb700 1 -- 172.21.15.93:0/1062443703 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 31) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1b24099460 con 0x55e82667b6b0 2023-11-16T16:54:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.065+0000 7f1b43d83b80 10 client.25757 fetch_fsmap finished waiting for FSMap version 31 2023-11-16T16:54:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.065+0000 7f1b43d83b80 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:54:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.065+0000 7f1b43d83b80 20 client.25757 populate_metadata read hostname 'smithi093' 2023-11-16T16:54:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.065+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1062443703 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55e826889c20 0x55e82688c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.065+0000 7f1b43d83b80 4 client.25757 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:54:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.065+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1062443703 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e8265bff10 con 0x55e826889c20 2023-11-16T16:54:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.066+0000 7f1b2d99a700 1 --2- 172.21.15.93:0/1062443703 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55e826889c20 0x55e82688c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.068+0000 7f1b11ffb700 10 client.25757 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.068+0000 7f1b2d99a700 1 --2- 172.21.15.93:0/1062443703 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55e826889c20 0x55e82688c030 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f96038b6d2d9dbd5 server_cookie=fc76d8a44b3aa227 in_seq=0 out_seq=0 2023-11-16T16:54:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.215+0000 7f1b11ffb700 1 -- 172.21.15.93:0/1062443703 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e8265bff10 con 0x55e826889c20 2023-11-16T16:54:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.215+0000 7f1b11ffb700 10 client.25757 handle_command_reply: tid=0 2023-11-16T16:54:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.215+0000 7f1b43d83b80 1 client.25757 shutdown 2023-11-16T16:54:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.215+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1062443703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 msgr2=0x55e82682fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.215+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1062443703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e82682fa80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400c5b0 tx=0x7f1b240b83c0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.216+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1062443703 shutdown_connections 2023-11-16T16:54:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.216+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1062443703 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x55e826889c20 0x55e82688c030 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.216+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1062443703 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e8268293b0 0x55e82682c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.216+0000 7f1b11ffb700 0 client.25757 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.216+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1062443703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e82667b6b0 0x55e82682fa80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.216+0000 7f1b43d83b80 1 --2- 172.21.15.93:0/1062443703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e826820b10 0x55e82682ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.216+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1062443703 >> 172.21.15.93:0/1062443703 conn(0x55e826814f50 msgr2=0x55e826826e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.216+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1062443703 shutdown_connections 2023-11-16T16:54:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.216+0000 7f1b43d83b80 1 -- 172.21.15.93:0/1062443703 wait complete. 2023-11-16T16:54:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.217+0000 7f1b43d83b80 20 client.25757 trim_cache size 0 max 16384 2023-11-16T16:54:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:54:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:54:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:54:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:54:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.219+0000 7f1b303ff700 1 -- 172.21.15.93:0/734896209 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1b140a11d0 msgr2=0x7f1b140a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.219+0000 7f1b303ff700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1b140a11d0 0x7f1b140a3680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1b20004310 tx=0x7f1b20009f70 comp rx=0 tx=0).stop 2023-11-16T16:54:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.219+0000 7f1b303ff700 1 -- 172.21.15.93:0/734896209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b28071cc0 msgr2=0x7f1b281a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.219+0000 7f1b303ff700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b28071cc0 0x7f1b281a4db0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400e9c0 tx=0x7f1b2400ecd0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.219+0000 7f1b303ff700 1 -- 172.21.15.93:0/734896209 shutdown_connections 2023-11-16T16:54:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.219+0000 7f1b303ff700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1b140a11d0 0x7f1b140a3680 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.219+0000 7f1b303ff700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b28071cc0 0x7f1b281a4db0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.219+0000 7f1b303ff700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b28107d60 0x7f1b281ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.219+0000 7f1b303ff700 1 --2- 172.21.15.93:0/734896209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b28072650 0x7f1b281a52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.219+0000 7f1b303ff700 1 -- 172.21.15.93:0/734896209 >> 172.21.15.93:0/734896209 conn(0x7f1b2806d5e0 msgr2=0x7f1b2810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.220+0000 7f1b303ff700 1 -- 172.21.15.93:0/734896209 shutdown_connections 2023-11-16T16:54:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:29.220+0000 7f1b303ff700 1 -- 172.21.15.93:0/734896209 wait complete. 2023-11-16T16:54:30.755 INFO:tasks.workunit.client.0.smithi093.stdout:cleaned up hierarchy 2023-11-16T16:54:30.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:30 smithi125 ceph-mon[131076]: pgmap v523: 129 pgs: 129 active+clean; 364 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 415 op/s 2023-11-16T16:54:30.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:30 smithi093 ceph-mon[97863]: pgmap v523: 129 pgs: 129 active+clean; 364 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 415 op/s 2023-11-16T16:54:30.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:30 smithi159 ceph-mon[131528]: pgmap v523: 129 pgs: 129 active+clean; 364 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 415 op/s 2023-11-16T16:54:31.271 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:54:31.271 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:54:31.861 INFO:tasks.workunit:Running workunit fs/misc/filelock_deadlock.py... 2023-11-16T16:54:31.861 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/filelock_deadlock.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_deadlock.py 2023-11-16T16:54:32.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:32 smithi125 ceph-mon[131076]: pgmap v524: 129 pgs: 129 active+clean; 365 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 395 op/s 2023-11-16T16:54:32.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:32 smithi093 ceph-mon[97863]: pgmap v524: 129 pgs: 129 active+clean; 365 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 395 op/s 2023-11-16T16:54:32.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:32 smithi159 ceph-mon[131528]: pgmap v524: 129 pgs: 129 active+clean; 365 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 395 op/s 2023-11-16T16:54:34.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:34 smithi125 ceph-mon[131076]: pgmap v525: 129 pgs: 129 active+clean; 368 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1013 KiB/s wr, 344 op/s 2023-11-16T16:54:34.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:34 smithi093 ceph-mon[97863]: pgmap v525: 129 pgs: 129 active+clean; 368 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1013 KiB/s wr, 344 op/s 2023-11-16T16:54:34.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:34 smithi159 ceph-mon[131528]: pgmap v525: 129 pgs: 129 active+clean; 368 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1013 KiB/s wr, 344 op/s 2023-11-16T16:54:36.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:36 smithi125 ceph-mon[131076]: pgmap v526: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 321 op/s 2023-11-16T16:54:36.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:36 smithi093 ceph-mon[97863]: pgmap v526: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 321 op/s 2023-11-16T16:54:36.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:36 smithi159 ceph-mon[131528]: pgmap v526: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 321 op/s 2023-11-16T16:54:38.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:38 smithi125 ceph-mon[131076]: pgmap v527: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 273 op/s 2023-11-16T16:54:38.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:38 smithi093 ceph-mon[97863]: pgmap v527: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 273 op/s 2023-11-16T16:54:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:38 smithi159 ceph-mon[131528]: pgmap v527: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 273 op/s 2023-11-16T16:54:40.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:40 smithi125 ceph-mon[131076]: pgmap v528: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 639 KiB/s wr, 215 op/s 2023-11-16T16:54:40.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:40 smithi093 ceph-mon[97863]: pgmap v528: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 639 KiB/s wr, 215 op/s 2023-11-16T16:54:40.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:40 smithi159 ceph-mon[131528]: pgmap v528: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 639 KiB/s wr, 215 op/s 2023-11-16T16:54:42.019 INFO:tasks.workunit.client.0.smithi093.stdout:ok 2023-11-16T16:54:42.024 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:54:42.025 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:54:42.126 INFO:tasks.workunit:Running workunit fs/misc/filelock_interrupt.py... 2023-11-16T16:54:42.126 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/filelock_interrupt.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_interrupt.py 2023-11-16T16:54:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.391+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/391887951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b98101430 msgr2=0x7f5b98101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.391+0000 7f5b9f5ff700 1 --2- 172.21.15.93:0/391887951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b98101430 0x7f5b98101880 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009990 tx=0x7f5b88009ca0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.391+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/391887951 shutdown_connections 2023-11-16T16:54:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.391+0000 7f5b9f5ff700 1 --2- 172.21.15.93:0/391887951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b98101dc0 0x7f5b98108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.391+0000 7f5b9f5ff700 1 --2- 172.21.15.93:0/391887951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b98100240 0x7f5b98100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.391+0000 7f5b9f5ff700 1 --2- 172.21.15.93:0/391887951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b98101430 0x7f5b98101880 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.391+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/391887951 >> 172.21.15.93:0/391887951 conn(0x7f5b980fb840 msgr2=0x7f5b980fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.391+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/391887951 shutdown_connections 2023-11-16T16:54:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.391+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/391887951 wait complete. 2023-11-16T16:54:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.392+0000 7f5b9f5ff700 1 Processor -- start 2023-11-16T16:54:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.392+0000 7f5b9f5ff700 1 -- start start 2023-11-16T16:54:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9f5ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b98100240 0x7f5b981981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9f5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b98101430 0x7f5b98198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9f5ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b98101dc0 0x7f5b9819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9f5ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9819fd70 con 0x7f5b98100240 2023-11-16T16:54:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9f5ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9819feb0 con 0x7f5b98101dc0 2023-11-16T16:54:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9f5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b981a0040 con 0x7f5b98101430 2023-11-16T16:54:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9d39b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b98100240 0x7f5b981981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9d39b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b98100240 0x7f5b981981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58706/0 (socket says 172.21.15.93:58706) 2023-11-16T16:54:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9d39b700 1 -- 172.21.15.93:0/4131013737 learned_addr learned my addr 172.21.15.93:0/4131013737 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9cb9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b98101430 0x7f5b98198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9db9c700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b98101dc0 0x7f5b9819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9d39b700 1 -- 172.21.15.93:0/4131013737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b98101430 msgr2=0x7f5b98198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b98101430 0x7f5b98198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9d39b700 1 -- 172.21.15.93:0/4131013737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b98101dc0 msgr2=0x7f5b9819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b98101dc0 0x7f5b9819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.393+0000 7f5b9d39b700 1 -- 172.21.15.93:0/4131013737 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b90009c40 con 0x7f5b98100240 2023-11-16T16:54:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.394+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b98100240 0x7f5b981981f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400e970 tx=0x7f5b9400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.394+0000 7f5b8e7fc700 1 -- 172.21.15.93:0/4131013737 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b94009790 con 0x7f5b98100240 2023-11-16T16:54:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.394+0000 7f5b8e7fc700 1 -- 172.21.15.93:0/4131013737 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b9400c7c0 con 0x7f5b98100240 2023-11-16T16:54:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.394+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/4131013737 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b880095f0 con 0x7f5b98100240 2023-11-16T16:54:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.394+0000 7f5b8e7fc700 1 -- 172.21.15.93:0/4131013737 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b94003c20 con 0x7f5b98100240 2023-11-16T16:54:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.394+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/4131013737 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9810bc00 con 0x7f5b98100240 2023-11-16T16:54:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.397+0000 7f5b8e7fc700 1 -- 172.21.15.93:0/4131013737 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5b9400c930 con 0x7f5b98100240 2023-11-16T16:54:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.398+0000 7f5bb2fa4b80 1 Processor -- start 2023-11-16T16:54:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.398+0000 7f5bb2fa4b80 1 -- start start 2023-11-16T16:54:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5b8e7fc700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5b840a1150 0x7f5b840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5bb2fa4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019b636b0 0x563019d07870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5bb2fa4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d07db0 0x563019d085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5bb2fa4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d08b10 0x563019d08f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5bb2fa4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019c0d000 con 0x563019d07db0 2023-11-16T16:54:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5bb2fa4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019c5f840 con 0x563019b636b0 2023-11-16T16:54:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5bb2fa4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019c58cd0 con 0x563019d08b10 2023-11-16T16:54:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5b8e7fc700 1 -- 172.21.15.93:0/4131013737 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f5b94014070 con 0x7f5b98100240 2023-11-16T16:54:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5b9db9c700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5b840a1150 0x7f5b840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5b9d39b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019b636b0 0x563019d07870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5b9cb9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d07db0 0x563019d085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5b9d39b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019b636b0 0x563019d07870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:49292/0 (socket says 172.21.15.93:49292) 2023-11-16T16:54:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5b9d39b700 1 -- 172.21.15.93:0/2574430012 learned_addr learned my addr 172.21.15.93:0/2574430012 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5b9db9c700 1 --2- 172.21.15.93:0/2574430012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d08b10 0x563019d08f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5b9d39b700 1 -- 172.21.15.93:0/2574430012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d08b10 msgr2=0x563019d08f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/2574430012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d08b10 0x563019d08f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5b9d39b700 1 -- 172.21.15.93:0/2574430012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d07db0 msgr2=0x563019d085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.399+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/2574430012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d07db0 0x563019d085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.400+0000 7f5b9d39b700 1 -- 172.21.15.93:0/2574430012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563019ada5d0 con 0x563019b636b0 2023-11-16T16:54:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.400+0000 7f5b9db9c700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5b840a1150 0x7f5b840a3600 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009fd0 tx=0x7f5b9000fde0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.400+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/2574430012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019b636b0 0x563019d07870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b940a1dd0 tx=0x7f5b94099890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27ee176360cb5985 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.400+0000 7f5b837fe700 1 -- 172.21.15.93:0/2574430012 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940ac1d0 con 0x563019b636b0 2023-11-16T16:54:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.400+0000 7f5b837fe700 1 -- 172.21.15.93:0/2574430012 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b940abfa0 con 0x563019b636b0 2023-11-16T16:54:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.400+0000 7f5b837fe700 1 -- 172.21.15.93:0/2574430012 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940ca6b0 con 0x563019b636b0 2023-11-16T16:54:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.401+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/2574430012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019b636b0 msgr2=0x563019d07870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.401+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/2574430012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019b636b0 0x563019d07870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b940a1dd0 tx=0x7f5b94099890 comp rx=0 tx=0).stop 2023-11-16T16:54:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.401+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/2574430012 shutdown_connections 2023-11-16T16:54:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.401+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/2574430012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d08b10 0x563019d08f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.401+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/2574430012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019b636b0 0x563019d07870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.401+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/2574430012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d07db0 0x563019d085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.401+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/2574430012 >> 172.21.15.93:0/2574430012 conn(0x563019cfcf50 msgr2=0x563019d02100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.401+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/2574430012 shutdown_connections 2023-11-16T16:54:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.401+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/2574430012 wait complete. 2023-11-16T16:54:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.401+0000 7f5bb2fa4b80 1 -- start start 2023-11-16T16:54:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.402+0000 7f5bb2fa4b80 1 Processor -- start 2023-11-16T16:54:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.402+0000 7f5bb2fa4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019b636b0 0x563019d10bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.402+0000 7f5bb2fa4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d07db0 0x563019d11130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.402+0000 7f5bb2fa4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019d08b10 0x563019d18170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.402+0000 7f5bb2fa4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019bfa880 con 0x563019d07db0 2023-11-16T16:54:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.402+0000 7f5bb2fa4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019bfa520 con 0x563019d08b10 2023-11-16T16:54:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.402+0000 7f5bb2fa4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019adc8c0 con 0x563019b636b0 2023-11-16T16:54:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.402+0000 7f5bb2fa4b80 10 client.0 _ll_register_callbacks cb 0x563019c18720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:54:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.402+0000 7f5b9cb9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d07db0 0x563019d11130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.402+0000 7f5b9cb9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d07db0 0x563019d11130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58724/0 (socket says 172.21.15.93:58724) 2023-11-16T16:54:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.402+0000 7f5b9cb9a700 1 -- 172.21.15.93:0/3033769680 learned_addr learned my addr 172.21.15.93:0/3033769680 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.403+0000 7f5b9db9c700 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019d08b10 0x563019d18170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.403+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019b636b0 0x563019d10bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.403+0000 7f5b9cb9a700 1 -- 172.21.15.93:0/3033769680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019b636b0 msgr2=0x563019d10bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.403+0000 7f5b9cb9a700 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019b636b0 0x563019d10bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.403+0000 7f5b9cb9a700 1 -- 172.21.15.93:0/3033769680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019d08b10 msgr2=0x563019d18170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.403+0000 7f5b9cb9a700 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019d08b10 0x563019d18170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.403+0000 7f5b9cb9a700 1 -- 172.21.15.93:0/3033769680 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9810bc00 con 0x563019d07db0 2023-11-16T16:54:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.403+0000 7f5b9cb9a700 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d07db0 0x563019d11130 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5b88015000 tx=0x7f5b88009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331b89b95d7073d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.403+0000 7f5b80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:54:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.403+0000 7f5b80ff9700 1 -- 172.21.15.93:0/3033769680 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8801d030 con 0x563019d07db0 2023-11-16T16:54:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.403+0000 7f5b80ff9700 1 -- 172.21.15.93:0/3033769680 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f5b88004500 con 0x563019d07db0 2023-11-16T16:54:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.403+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/3033769680 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563019cfa710 con 0x563019d07db0 2023-11-16T16:54:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.404+0000 7f5b80ff9700 1 -- 172.21.15.93:0/3033769680 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b880046c0 con 0x563019d07db0 2023-11-16T16:54:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.404+0000 7f5bb2fa4b80 10 client.15702 fetch_fsmap learned FSMap version 31 2023-11-16T16:54:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.404+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/3033769680 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563019c1f130 con 0x563019d07db0 2023-11-16T16:54:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.404+0000 7f5b80ff9700 1 -- 172.21.15.93:0/3033769680 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563019cfa710 con 0x563019d07db0 2023-11-16T16:54:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.404+0000 7f5b80ff9700 1 -- 172.21.15.93:0/3033769680 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 31) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5b8800f870 con 0x563019d07db0 2023-11-16T16:54:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.404+0000 7f5bb2fa4b80 10 client.15702 fetch_fsmap finished waiting for FSMap version 31 2023-11-16T16:54:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.404+0000 7f5bb2fa4b80 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:54:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.404+0000 7f5bb2fa4b80 20 client.15702 populate_metadata read hostname 'smithi093' 2023-11-16T16:54:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.404+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x563019d1caa0 0x563019d1eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.404+0000 7f5bb2fa4b80 4 client.15702 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:54:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.404+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/3033769680 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563019c1b040 con 0x563019d1caa0 2023-11-16T16:54:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.405+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x563019d1caa0 0x563019d1eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.405+0000 7f5b80ff9700 10 client.15702 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.405+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x563019d1caa0 0x563019d1eeb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480fa90058032a83 server_cookie=9046bedbe8201f21 in_seq=0 out_seq=0 2023-11-16T16:54:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5b80ff9700 1 -- 172.21.15.93:0/3033769680 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563019c1b040 con 0x563019d1caa0 2023-11-16T16:54:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5b80ff9700 10 client.15702 handle_command_reply: tid=0 2023-11-16T16:54:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5bb2fa4b80 1 client.15702 shutdown 2023-11-16T16:54:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/3033769680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d07db0 msgr2=0x563019d11130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d07db0 0x563019d11130 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5b88015000 tx=0x7f5b88009570 comp rx=0 tx=0).stop 2023-11-16T16:54:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/3033769680 shutdown_connections 2023-11-16T16:54:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x563019d1caa0 0x563019d1eeb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019d08b10 0x563019d18170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019b636b0 0x563019d10bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5b80ff9700 0 client.15702 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/3033769680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d07db0 0x563019d11130 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/3033769680 >> 172.21.15.93:0/3033769680 conn(0x563019cfcf50 msgr2=0x563019d017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/3033769680 shutdown_connections 2023-11-16T16:54:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/3033769680 wait complete. 2023-11-16T16:54:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.406+0000 7f5bb2fa4b80 20 client.15702 trim_cache size 0 max 16384 2023-11-16T16:54:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5bb2fa4b80 1 Processor -- start 2023-11-16T16:54:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5bb2fa4b80 1 -- start start 2023-11-16T16:54:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5bb2fa4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019b636b0 0x563019cfe420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5bb2fa4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d07db0 0x563019d18130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5bb2fa4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019d08b10 0x563019d08ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5bb2fa4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019d09420 con 0x563019b636b0 2023-11-16T16:54:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5bb2fa4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019d0ec10 con 0x563019d08b10 2023-11-16T16:54:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5bb2fa4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019d0c520 con 0x563019d07db0 2023-11-16T16:54:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5b9d39b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019b636b0 0x563019cfe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5b9d39b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019b636b0 0x563019cfe420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58732/0 (socket says 172.21.15.93:58732) 2023-11-16T16:54:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5b9d39b700 1 -- 172.21.15.93:0/2170487943 learned_addr learned my addr 172.21.15.93:0/2170487943 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5b9cb9a700 1 --2- 172.21.15.93:0/2170487943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d07db0 0x563019d18130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.422+0000 7f5b9db9c700 1 --2- 172.21.15.93:0/2170487943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019d08b10 0x563019d08ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.423+0000 7f5b9d39b700 1 -- 172.21.15.93:0/2170487943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d07db0 msgr2=0x563019d18130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.423+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/2170487943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d07db0 0x563019d18130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.423+0000 7f5b9d39b700 1 -- 172.21.15.93:0/2170487943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019d08b10 msgr2=0x563019d08ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.423+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/2170487943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019d08b10 0x563019d08ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.423+0000 7f5b9d39b700 1 -- 172.21.15.93:0/2170487943 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563019c1ee80 con 0x563019b636b0 2023-11-16T16:54:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.423+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/2170487943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019b636b0 0x563019cfe420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5b94096360 tx=0x7f5b94099db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffb066cb2db0b8f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.423+0000 7f5b83fff700 1 -- 172.21.15.93:0/2170487943 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940a2300 con 0x563019b636b0 2023-11-16T16:54:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.423+0000 7f5b83fff700 1 -- 172.21.15.93:0/2170487943 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b9409dee0 con 0x563019b636b0 2023-11-16T16:54:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.423+0000 7f5b83fff700 1 -- 172.21.15.93:0/2170487943 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940adbb0 con 0x563019b636b0 2023-11-16T16:54:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.424+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/2170487943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019b636b0 msgr2=0x563019cfe420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.424+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/2170487943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019b636b0 0x563019cfe420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5b94096360 tx=0x7f5b94099db0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.424+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/2170487943 shutdown_connections 2023-11-16T16:54:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.424+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/2170487943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019b636b0 0x563019cfe420 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.424+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/2170487943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019d08b10 0x563019d08ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.424+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/2170487943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d07db0 0x563019d18130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.424+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/2170487943 >> 172.21.15.93:0/2170487943 conn(0x563019cfcf50 msgr2=0x563019d23810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.424+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/2170487943 shutdown_connections 2023-11-16T16:54:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.424+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/2170487943 wait complete. 2023-11-16T16:54:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.424+0000 7f5bb2fa4b80 1 -- start start 2023-11-16T16:54:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.425+0000 7f5bb2fa4b80 1 Processor -- start 2023-11-16T16:54:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.425+0000 7f5bb2fa4b80 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019b636b0 0x563019d16370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.425+0000 7f5bb2fa4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d07db0 0x563019d168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.425+0000 7f5bb2fa4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d08b10 0x563019d16e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.425+0000 7f5bb2fa4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019d17560 con 0x563019d08b10 2023-11-16T16:54:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.425+0000 7f5bb2fa4b80 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019d176d0 con 0x563019b636b0 2023-11-16T16:54:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.425+0000 7f5bb2fa4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563019d17890 con 0x563019d07db0 2023-11-16T16:54:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.425+0000 7f5bb2fa4b80 10 client.0 _ll_register_callbacks cb 0x7f5b88005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:54:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.425+0000 7f5b9db9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d08b10 0x563019d16e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.425+0000 7f5b9db9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d08b10 0x563019d16e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:58740/0 (socket says 172.21.15.93:58740) 2023-11-16T16:54:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.425+0000 7f5b9db9c700 1 -- 172.21.15.93:0/1071940415 learned_addr learned my addr 172.21.15.93:0/1071940415 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.426+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019b636b0 0x563019d16370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.426+0000 7f5b9cb9a700 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d07db0 0x563019d168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.426+0000 7f5b9db9c700 1 -- 172.21.15.93:0/1071940415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d07db0 msgr2=0x563019d168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.426+0000 7f5b9db9c700 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d07db0 0x563019d168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.426+0000 7f5b9db9c700 1 -- 172.21.15.93:0/1071940415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019b636b0 msgr2=0x563019d16370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:54:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.426+0000 7f5b9db9c700 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019b636b0 0x563019d16370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.426+0000 7f5b9db9c700 1 -- 172.21.15.93:0/1071940415 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563019ada5d0 con 0x563019d08b10 2023-11-16T16:54:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.426+0000 7f5b9db9c700 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d08b10 0x563019d16e50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5b90018dd0 tx=0x7f5b90004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116a26635a5456c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.426+0000 7f5b80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-11-16T16:54:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.426+0000 7f5b80ff9700 1 -- 172.21.15.93:0/1071940415 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9000e040 con 0x563019d08b10 2023-11-16T16:54:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.426+0000 7f5b80ff9700 1 -- 172.21.15.93:0/1071940415 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f5b90003c00 con 0x563019d08b10 2023-11-16T16:54:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.426+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/1071940415 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b940abfa0 con 0x563019d08b10 2023-11-16T16:54:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.427+0000 7f5b80ff9700 1 -- 172.21.15.93:0/1071940415 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b90008e90 con 0x563019d08b10 2023-11-16T16:54:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.427+0000 7f5bb2fa4b80 10 client.15708 fetch_fsmap learned FSMap version 31 2023-11-16T16:54:42.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.427+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/1071940415 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563019a6fc00 con 0x563019d08b10 2023-11-16T16:54:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.427+0000 7f5b80ff9700 1 -- 172.21.15.93:0/1071940415 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b940abfa0 con 0x563019d08b10 2023-11-16T16:54:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.427+0000 7f5b80ff9700 1 -- 172.21.15.93:0/1071940415 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 31) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5b9001a440 con 0x563019d08b10 2023-11-16T16:54:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.427+0000 7f5bb2fa4b80 10 client.15708 fetch_fsmap finished waiting for FSMap version 31 2023-11-16T16:54:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.427+0000 7f5bb2fa4b80 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-16T16:54:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.427+0000 7f5bb2fa4b80 20 client.15708 populate_metadata read hostname 'smithi093' 2023-11-16T16:54:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.427+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x563019d71660 0x563019d73a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.427+0000 7f5bb2fa4b80 4 client.15708 mds_command: new command op to 24566 tid=0[{"prefix": "flush journal"}] 2023-11-16T16:54:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.427+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/1071940415 --> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563019c27600 con 0x563019d71660 2023-11-16T16:54:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.427+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x563019d71660 0x563019d73a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.428+0000 7f5b80ff9700 10 client.15708 ms_handle_connect on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:42.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.428+0000 7f5b9d39b700 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x563019d71660 0x563019d73a70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f3599ef734575fd server_cookie=4a03d1505938054a in_seq=0 out_seq=0 2023-11-16T16:54:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.452+0000 7f5b80ff9700 1 -- 172.21.15.93:0/1071940415 <== mds.0 v2:172.21.15.125:6838/3590027328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563019c27600 con 0x563019d71660 2023-11-16T16:54:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5b80ff9700 10 client.15708 handle_command_reply: tid=0 2023-11-16T16:54:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5bb2fa4b80 1 client.15708 shutdown 2023-11-16T16:54:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/1071940415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d08b10 msgr2=0x563019d16e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d08b10 0x563019d16e50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5b90018dd0 tx=0x7f5b90004b40 comp rx=0 tx=0).stop 2023-11-16T16:54:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/1071940415 shutdown_connections 2023-11-16T16:54:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.125:6838/3590027328,v1:172.21.15.125:6839/3590027328] conn(0x563019d71660 0x563019d73a70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563019d08b10 0x563019d16e50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563019b636b0 0x563019d16370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5bb2fa4b80 1 --2- 172.21.15.93:0/1071940415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563019d07db0 0x563019d168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5b80ff9700 0 client.15708 ms_handle_reset on v2:172.21.15.125:6838/3590027328 2023-11-16T16:54:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/1071940415 >> 172.21.15.93:0/1071940415 conn(0x563019cfcf50 msgr2=0x563019d0b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/1071940415 shutdown_connections 2023-11-16T16:54:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5bb2fa4b80 1 -- 172.21.15.93:0/1071940415 wait complete. 2023-11-16T16:54:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.453+0000 7f5bb2fa4b80 20 client.15708 trim_cache size 0 max 16384 2023-11-16T16:54:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-11-16T16:54:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-11-16T16:54:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-11-16T16:54:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-11-16T16:54:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.454+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/4131013737 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5b840a1150 msgr2=0x7f5b840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.454+0000 7f5b9f5ff700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5b840a1150 0x7f5b840a3600 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009fd0 tx=0x7f5b9000fde0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.454+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/4131013737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b98100240 msgr2=0x7f5b981981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.454+0000 7f5b9f5ff700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b98100240 0x7f5b981981f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400e970 tx=0x7f5b9400ec80 comp rx=0 tx=0).stop 2023-11-16T16:54:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.454+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/4131013737 shutdown_connections 2023-11-16T16:54:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.454+0000 7f5b9f5ff700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f5b840a1150 0x7f5b840a3600 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.454+0000 7f5b9f5ff700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b98100240 0x7f5b981981f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.454+0000 7f5b9f5ff700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b98101dc0 0x7f5b9819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.454+0000 7f5b9f5ff700 1 --2- 172.21.15.93:0/4131013737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b98101430 0x7f5b98198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.454+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/4131013737 >> 172.21.15.93:0/4131013737 conn(0x7f5b980fb840 msgr2=0x7f5b98106d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.454+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/4131013737 shutdown_connections 2023-11-16T16:54:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-11-16T16:54:42.454+0000 7f5b9f5ff700 1 -- 172.21.15.93:0/4131013737 wait complete. 2023-11-16T16:54:42.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:42 smithi125 ceph-mon[131076]: pgmap v529: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 502 KiB/s wr, 166 op/s 2023-11-16T16:54:42.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:42 smithi093 ceph-mon[97863]: pgmap v529: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 502 KiB/s wr, 166 op/s 2023-11-16T16:54:42.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:42 smithi159 ceph-mon[131528]: pgmap v529: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 502 KiB/s wr, 166 op/s 2023-11-16T16:54:44.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:44 smithi125 ceph-mon[131076]: pgmap v530: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 387 KiB/s wr, 108 op/s 2023-11-16T16:54:44.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:44 smithi093 ceph-mon[97863]: pgmap v530: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 387 KiB/s wr, 108 op/s 2023-11-16T16:54:44.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:44 smithi159 ceph-mon[131528]: pgmap v530: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 387 KiB/s wr, 108 op/s 2023-11-16T16:54:46.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:46 smithi125 ceph-mon[131076]: pgmap v531: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 58 op/s 2023-11-16T16:54:46.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:46 smithi093 ceph-mon[97863]: pgmap v531: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 58 op/s 2023-11-16T16:54:46.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:46 smithi159 ceph-mon[131528]: pgmap v531: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 58 op/s 2023-11-16T16:54:48.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:48 smithi125 ceph-mon[131076]: pgmap v532: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 23 op/s 2023-11-16T16:54:48.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:48 smithi093 ceph-mon[97863]: pgmap v532: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 23 op/s 2023-11-16T16:54:48.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:48 smithi159 ceph-mon[131528]: pgmap v532: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 23 op/s 2023-11-16T16:54:50.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:50 smithi125 ceph-mon[131076]: pgmap v533: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2023-11-16T16:54:50.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:50 smithi093 ceph-mon[97863]: pgmap v533: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2023-11-16T16:54:50.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:50 smithi159 ceph-mon[131528]: pgmap v533: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2023-11-16T16:54:52.230 INFO:tasks.workunit.client.0.smithi093.stdout:ok 2023-11-16T16:54:52.235 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:54:52.235 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:54:52.369 INFO:tasks.workunit:Running workunit fs/misc/general_vxattrs.sh... 2023-11-16T16:54:52.369 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/general_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh 2023-11-16T16:54:52.450 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir -p dir 2023-11-16T16:54:52.455 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-16T16:54:52.455 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'ceph.dir.pin="-1"' 2023-11-16T16:54:52.457 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.dir.pin="-1" 2023-11-16T16:54:52.458 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.pin dir 2023-11-16T16:54:52.459 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-16T16:54:52.461 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-11-16T16:54:52.461 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.pin -v 1 dir 2023-11-16T16:54:52.465 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-16T16:54:52.466 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'ceph.dir.pin="1"' 2023-11-16T16:54:52.469 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.dir.pin="1" 2023-11-16T16:54:52.469 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.pin dir 2023-11-16T16:54:52.473 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-16T16:54:52.473 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'ceph.dir.pin="-1"' 2023-11-16T16:54:52.476 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.dir.pin="-1" 2023-11-16T16:54:52.477 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.pin.distributed dir 2023-11-16T16:54:52.477 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-16T16:54:52.480 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-11-16T16:54:52.480 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 dir 2023-11-16T16:54:52.483 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.pin.distributed dir 2023-11-16T16:54:52.487 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.pin.random dir 2023-11-16T16:54:52.487 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-16T16:54:52.490 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-11-16T16:54:52.490 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.pin.random -v 0.01 dir 2023-11-16T16:54:52.493 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.pin.random dir 2023-11-16T16:54:52.496 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.subvolume dir 2023-11-16T16:54:52.497 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-16T16:54:52.499 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-11-16T16:54:52.499 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.subvolume -v 1 dir 2023-11-16T16:54:52.509 INFO:tasks.workunit.client.0.smithi093.stderr:setfattr: dir: Invalid argument 2023-11-16T16:54:52.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:54:52.510 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2023-11-16T16:54:52.510 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-11-16T16:54:52.617 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:52 smithi093 ceph-mon[97863]: pgmap v534: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.1 KiB/s wr, 15 op/s 2023-11-16T16:54:52.829 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:52 smithi125 ceph-mon[131076]: pgmap v534: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.1 KiB/s wr, 15 op/s 2023-11-16T16:54:52.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:52 smithi159 ceph-mon[131528]: pgmap v534: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.1 KiB/s wr, 15 op/s 2023-11-16T16:54:54.127 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi093 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-11-16T16:54:54.190 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=bd6ad00590474c61b9881987bce57f35 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi093 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-11-16T16:54:54.192 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-11-16T16:54:54.230 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi093.front.sepia.ceph.com 2023-11-16T16:54:54.242 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-11-16T16:54:54.263 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:54:54.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.480+0000 7f578d3ff700 1 -- 172.21.15.93:0/3318831296 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57881014a0 msgr2=0x7f57881018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:54.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.480+0000 7f578d3ff700 1 --2- 172.21.15.93:0/3318831296 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57881014a0 0x7f57881018f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f577c009990 tx=0x7f577c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.480+0000 7f578d3ff700 1 -- 172.21.15.93:0/3318831296 shutdown_connections 2023-11-16T16:54:54.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.480+0000 7f578d3ff700 1 --2- 172.21.15.93:0/3318831296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5788101e30 0x7f57881086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.480+0000 7f578d3ff700 1 --2- 172.21.15.93:0/3318831296 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57881002b0 0x7f57881006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.480+0000 7f578d3ff700 1 --2- 172.21.15.93:0/3318831296 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57881014a0 0x7f57881018f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.480+0000 7f578d3ff700 1 -- 172.21.15.93:0/3318831296 >> 172.21.15.93:0/3318831296 conn(0x7f57880fb890 msgr2=0x7f57880fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:54.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.480+0000 7f578d3ff700 1 -- 172.21.15.93:0/3318831296 shutdown_connections 2023-11-16T16:54:54.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.480+0000 7f578d3ff700 1 -- 172.21.15.93:0/3318831296 wait complete. 2023-11-16T16:54:54.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.481+0000 7f578d3ff700 1 Processor -- start 2023-11-16T16:54:54.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f578d3ff700 1 -- start start 2023-11-16T16:54:54.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f578d3ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57881002b0 0x7f5788198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:54.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f578d3ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57881014a0 0x7f5788198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:54.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f578d3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5788101e30 0x7f578819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:54.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f578d3ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578819fda0 con 0x7f57881002b0 2023-11-16T16:54:54.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f578d3ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578819fee0 con 0x7f57881014a0 2023-11-16T16:54:54.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f578d3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57881a0070 con 0x7f5788101e30 2023-11-16T16:54:54.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f5786ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57881002b0 0x7f5788198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:54.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f5786ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57881002b0 0x7f5788198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57124/0 (socket says 172.21.15.93:57124) 2023-11-16T16:54:54.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f5786ffd700 1 -- 172.21.15.93:0/1508861534 learned_addr learned my addr 172.21.15.93:0/1508861534 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:54.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f57867fc700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57881014a0 0x7f5788198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:54.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f57877fe700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5788101e30 0x7f578819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:54.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f5786ffd700 1 -- 172.21.15.93:0/1508861534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5788101e30 msgr2=0x7f578819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:54.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f5786ffd700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5788101e30 0x7f578819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.482+0000 7f5786ffd700 1 -- 172.21.15.93:0/1508861534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57881014a0 msgr2=0x7f5788198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:54.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.483+0000 7f5786ffd700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57881014a0 0x7f5788198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.483+0000 7f5786ffd700 1 -- 172.21.15.93:0/1508861534 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0095f0 con 0x7f57881002b0 2023-11-16T16:54:54.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.483+0000 7f57867fc700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57881014a0 0x7f5788198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:54:54.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.483+0000 7f5786ffd700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57881002b0 0x7f5788198220 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f577800e970 tx=0x7f577800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:54.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.483+0000 7f576ffff700 1 -- 172.21.15.93:0/1508861534 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778009790 con 0x7f57881002b0 2023-11-16T16:54:54.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.483+0000 7f576ffff700 1 -- 172.21.15.93:0/1508861534 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f577800c7e0 con 0x7f57881002b0 2023-11-16T16:54:54.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.483+0000 7f578d3ff700 1 -- 172.21.15.93:0/1508861534 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578810bc60 con 0x7f57881002b0 2023-11-16T16:54:54.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.483+0000 7f576ffff700 1 -- 172.21.15.93:0/1508861534 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778003c70 con 0x7f57881002b0 2023-11-16T16:54:54.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.483+0000 7f578d3ff700 1 -- 172.21.15.93:0/1508861534 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578810bad0 con 0x7f57881002b0 2023-11-16T16:54:54.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.484+0000 7f578d3ff700 1 -- 172.21.15.93:0/1508861534 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57881928f0 con 0x7f57881002b0 2023-11-16T16:54:54.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.486+0000 7f576ffff700 1 -- 172.21.15.93:0/1508861534 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f577800cc50 con 0x7f57881002b0 2023-11-16T16:54:54.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.487+0000 7f576ffff700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f57700a11d0 0x7f57700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:54.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.487+0000 7f576ffff700 1 -- 172.21.15.93:0/1508861534 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f5778014070 con 0x7f57881002b0 2023-11-16T16:54:54.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.487+0000 7f57867fc700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f57700a11d0 0x7f57700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:54.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.488+0000 7f57867fc700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f57700a11d0 0x7f57700a3680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f577c00b2e0 tx=0x7f577c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:54.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.491+0000 7f576ffff700 1 -- 172.21.15.93:0/1508861534 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f57780d3080 con 0x7f57881002b0 2023-11-16T16:54:54.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.693+0000 7f578d3ff700 1 -- 172.21.15.93:0/1508861534 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f578802cfe0 con 0x7f57881002b0 2023-11-16T16:54:54.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.695+0000 7f576ffff700 1 -- 172.21.15.93:0/1508861534 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24795 (secure 0 0 0) 0x7f57780961a0 con 0x7f57881002b0 2023-11-16T16:54:54.696 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:54:54.697 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":94,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:53:48.219838+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":1,"to":2}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:54:54.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.697+0000 7f578d3ff700 1 -- 172.21.15.93:0/1508861534 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f57700a11d0 msgr2=0x7f57700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:54.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.697+0000 7f578d3ff700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f57700a11d0 0x7f57700a3680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f577c00b2e0 tx=0x7f577c005fb0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.697+0000 7f578d3ff700 1 -- 172.21.15.93:0/1508861534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57881002b0 msgr2=0x7f5788198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:54.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.697+0000 7f578d3ff700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57881002b0 0x7f5788198220 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f577800e970 tx=0x7f577800ec80 comp rx=0 tx=0).stop 2023-11-16T16:54:54.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.698+0000 7f578d3ff700 1 -- 172.21.15.93:0/1508861534 shutdown_connections 2023-11-16T16:54:54.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.698+0000 7f578d3ff700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f57700a11d0 0x7f57700a3680 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.698+0000 7f578d3ff700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57881002b0 0x7f5788198220 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.698+0000 7f578d3ff700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5788101e30 0x7f578819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.698+0000 7f578d3ff700 1 --2- 172.21.15.93:0/1508861534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57881014a0 0x7f5788198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.698+0000 7f578d3ff700 1 -- 172.21.15.93:0/1508861534 >> 172.21.15.93:0/1508861534 conn(0x7f57880fb890 msgr2=0x7f5788106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:54.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.698+0000 7f578d3ff700 1 -- 172.21.15.93:0/1508861534 shutdown_connections 2023-11-16T16:54:54.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.698+0000 7f578d3ff700 1 -- 172.21.15.93:0/1508861534 wait complete. 2023-11-16T16:54:54.718 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:42:13.029135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:46:35.301321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:46:36.552545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-16T16:46:39.628380+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-16T16:54:54.719 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:54:54.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:54 smithi093 ceph-mon[97863]: pgmap v535: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.9 KiB/s wr, 14 op/s 2023-11-16T16:54:54.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.920+0000 7f04c35ff700 1 -- 172.21.15.93:0/1152074218 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04bc101e50 msgr2=0x7f04bc108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:54.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.920+0000 7f04c35ff700 1 --2- 172.21.15.93:0/1152074218 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04bc101e50 0x7f04bc108710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f04b400b0a0 tx=0x7f04b400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.920+0000 7f04c35ff700 1 -- 172.21.15.93:0/1152074218 shutdown_connections 2023-11-16T16:54:54.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.920+0000 7f04c35ff700 1 --2- 172.21.15.93:0/1152074218 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04bc101e50 0x7f04bc108710 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.920+0000 7f04c35ff700 1 --2- 172.21.15.93:0/1152074218 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04bc1002d0 0x7f04bc1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.920+0000 7f04c35ff700 1 --2- 172.21.15.93:0/1152074218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04bc1014c0 0x7f04bc101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.920+0000 7f04c35ff700 1 -- 172.21.15.93:0/1152074218 >> 172.21.15.93:0/1152074218 conn(0x7f04bc0fb8b0 msgr2=0x7f04bc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:54.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.920+0000 7f04c35ff700 1 -- 172.21.15.93:0/1152074218 shutdown_connections 2023-11-16T16:54:54.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.920+0000 7f04c35ff700 1 -- 172.21.15.93:0/1152074218 wait complete. 2023-11-16T16:54:54.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.921+0000 7f04c35ff700 1 Processor -- start 2023-11-16T16:54:54.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c35ff700 1 -- start start 2023-11-16T16:54:54.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c35ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04bc1002d0 0x7f04bc198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:54.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c35ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04bc1014c0 0x7f04bc1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:54.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c35ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04bc101e50 0x7f04bc19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:54.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c35ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc19fee0 con 0x7f04bc1014c0 2023-11-16T16:54:54.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c35ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1a0020 con 0x7f04bc1002d0 2023-11-16T16:54:54.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c35ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1a01b0 con 0x7f04bc101e50 2023-11-16T16:54:54.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c0b9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04bc1014c0 0x7f04bc1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:54.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c139b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04bc1002d0 0x7f04bc198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:54.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c139b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04bc1002d0 0x7f04bc198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:38124/0 (socket says 172.21.15.93:38124) 2023-11-16T16:54:54.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c139b700 1 -- 172.21.15.93:0/2423280097 learned_addr learned my addr 172.21.15.93:0/2423280097 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:54.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c1b9c700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04bc101e50 0x7f04bc19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:54.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c0b9a700 1 -- 172.21.15.93:0/2423280097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04bc101e50 msgr2=0x7f04bc19f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:54.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c0b9a700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04bc101e50 0x7f04bc19f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c0b9a700 1 -- 172.21.15.93:0/2423280097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04bc1002d0 msgr2=0x7f04bc198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:54.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c0b9a700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04bc1002d0 0x7f04bc198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:54.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.922+0000 7f04c0b9a700 1 -- 172.21.15.93:0/2423280097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b8009520 con 0x7f04bc1014c0 2023-11-16T16:54:54.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.923+0000 7f04c0b9a700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04bc1014c0 0x7f04bc1988a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00e9f0 tx=0x7f04ac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:54.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.923+0000 7f04b27fc700 1 -- 172.21.15.93:0/2423280097 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00c9c0 con 0x7f04bc1014c0 2023-11-16T16:54:54.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.923+0000 7f04b27fc700 1 -- 172.21.15.93:0/2423280097 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04ac010430 con 0x7f04bc1014c0 2023-11-16T16:54:54.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.923+0000 7f04c35ff700 1 -- 172.21.15.93:0/2423280097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b4009d00 con 0x7f04bc1014c0 2023-11-16T16:54:54.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.923+0000 7f04b27fc700 1 -- 172.21.15.93:0/2423280097 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac018680 con 0x7f04bc1014c0 2023-11-16T16:54:54.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.923+0000 7f04c35ff700 1 -- 172.21.15.93:0/2423280097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04bc10bca0 con 0x7f04bc1014c0 2023-11-16T16:54:54.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.924+0000 7f04c35ff700 1 -- 172.21.15.93:0/2423280097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04bc067b00 con 0x7f04bc1014c0 2023-11-16T16:54:54.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.926+0000 7f04b27fc700 1 -- 172.21.15.93:0/2423280097 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f04ac010860 con 0x7f04bc1014c0 2023-11-16T16:54:54.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.927+0000 7f04b27fc700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f04a80a11d0 0x7f04a80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:54.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.927+0000 7f04b27fc700 1 -- 172.21.15.93:0/2423280097 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f04ac014070 con 0x7f04bc1014c0 2023-11-16T16:54:54.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.928+0000 7f04c139b700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f04a80a11d0 0x7f04a80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:54.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.928+0000 7f04c139b700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f04a80a11d0 0x7f04a80a3680 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f04b8005950 tx=0x7f04b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:54.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:54.931+0000 7f04b27fc700 1 -- 172.21.15.93:0/2423280097 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f04ac051730 con 0x7f04bc1014c0 2023-11-16T16:54:54.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:54 smithi159 ceph-mon[131528]: pgmap v535: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.9 KiB/s wr, 14 op/s 2023-11-16T16:54:55.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:54 smithi125 ceph-mon[131076]: pgmap v535: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.9 KiB/s wr, 14 op/s 2023-11-16T16:54:55.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.129+0000 7f04c35ff700 1 -- 172.21.15.93:0/2423280097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f04bc1a0d10 con 0x7f04bc1014c0 2023-11-16T16:54:55.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.130+0000 7f04b27fc700 1 -- 172.21.15.93:0/2423280097 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f04ac096120 con 0x7f04bc1014c0 2023-11-16T16:54:55.131 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-11-16T16:54:55.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.132+0000 7f04c35ff700 1 -- 172.21.15.93:0/2423280097 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f04a80a11d0 msgr2=0x7f04a80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:55.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.132+0000 7f04c35ff700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f04a80a11d0 0x7f04a80a3680 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f04b8005950 tx=0x7f04b8009260 comp rx=0 tx=0).stop 2023-11-16T16:54:55.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.132+0000 7f04c35ff700 1 -- 172.21.15.93:0/2423280097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04bc1014c0 msgr2=0x7f04bc1988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:55.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.132+0000 7f04c35ff700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04bc1014c0 0x7f04bc1988a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00e9f0 tx=0x7f04ac00ed00 comp rx=0 tx=0).stop 2023-11-16T16:54:55.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.132+0000 7f04c35ff700 1 -- 172.21.15.93:0/2423280097 shutdown_connections 2023-11-16T16:54:55.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.132+0000 7f04c35ff700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f04a80a11d0 0x7f04a80a3680 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.132+0000 7f04c35ff700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04bc101e50 0x7f04bc19f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.132+0000 7f04c35ff700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04bc1002d0 0x7f04bc198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.132+0000 7f04c35ff700 1 --2- 172.21.15.93:0/2423280097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04bc1014c0 0x7f04bc1988a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.132+0000 7f04c35ff700 1 -- 172.21.15.93:0/2423280097 >> 172.21.15.93:0/2423280097 conn(0x7f04bc0fb8b0 msgr2=0x7f04bc0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:55.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.132+0000 7f04c35ff700 1 -- 172.21.15.93:0/2423280097 shutdown_connections 2023-11-16T16:54:55.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.132+0000 7f04c35ff700 1 -- 172.21.15.93:0/2423280097 wait complete. 2023-11-16T16:54:55.152 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:54:55.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.351+0000 7f8013bff700 1 -- 172.21.15.93:0/79959392 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f800c1014c0 msgr2=0x7f800c101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:55.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.351+0000 7f8013bff700 1 --2- 172.21.15.93:0/79959392 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f800c1014c0 0x7f800c101910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009990 tx=0x7f7ffc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.352+0000 7f8013bff700 1 -- 172.21.15.93:0/79959392 shutdown_connections 2023-11-16T16:54:55.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.352+0000 7f8013bff700 1 --2- 172.21.15.93:0/79959392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c101e50 0x7f800c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.352+0000 7f8013bff700 1 --2- 172.21.15.93:0/79959392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f800c1002d0 0x7f800c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.352+0000 7f8013bff700 1 --2- 172.21.15.93:0/79959392 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f800c1014c0 0x7f800c101910 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.352+0000 7f8013bff700 1 -- 172.21.15.93:0/79959392 >> 172.21.15.93:0/79959392 conn(0x7f800c0fb8b0 msgr2=0x7f800c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:55.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.352+0000 7f8013bff700 1 -- 172.21.15.93:0/79959392 shutdown_connections 2023-11-16T16:54:55.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.352+0000 7f8013bff700 1 -- 172.21.15.93:0/79959392 wait complete. 2023-11-16T16:54:55.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.353+0000 7f8013bff700 1 Processor -- start 2023-11-16T16:54:55.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.353+0000 7f8013bff700 1 -- start start 2023-11-16T16:54:55.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.354+0000 7f8013bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f800c1002d0 0x7f800c196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:55.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.354+0000 7f8013bff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1014c0 0x7f800c1965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:55.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.354+0000 7f8013bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f800c101e50 0x7f800c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:55.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.354+0000 7f8013bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c19dc00 con 0x7f800c1002d0 2023-11-16T16:54:55.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.354+0000 7f8013bff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c19dd40 con 0x7f800c1014c0 2023-11-16T16:54:55.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.354+0000 7f8013bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c19ded0 con 0x7f800c101e50 2023-11-16T16:54:55.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.354+0000 7f801199b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f800c1002d0 0x7f800c196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:55.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.354+0000 7f801199b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f800c1002d0 0x7f800c196080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57150/0 (socket says 172.21.15.93:57150) 2023-11-16T16:54:55.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.354+0000 7f801199b700 1 -- 172.21.15.93:0/1333688350 learned_addr learned my addr 172.21.15.93:0/1333688350 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:55.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.354+0000 7f801119a700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1014c0 0x7f800c1965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:55.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.354+0000 7f801219c700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f800c101e50 0x7f800c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:55.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.355+0000 7f801119a700 1 -- 172.21.15.93:0/1333688350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f800c101e50 msgr2=0x7f800c19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:55.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.355+0000 7f801119a700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f800c101e50 0x7f800c19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.355+0000 7f801119a700 1 -- 172.21.15.93:0/1333688350 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f800c1002d0 msgr2=0x7f800c196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:55.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.355+0000 7f801119a700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f800c1002d0 0x7f800c196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.355+0000 7f801119a700 1 -- 172.21.15.93:0/1333688350 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc0095f0 con 0x7f800c1014c0 2023-11-16T16:54:55.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.355+0000 7f801199b700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f800c1002d0 0x7f800c196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:54:55.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.355+0000 7f801119a700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1014c0 0x7f800c1965c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0098e0 tx=0x7f7ffc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:55.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.355+0000 7f8002ffd700 1 -- 172.21.15.93:0/1333688350 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc01d070 con 0x7f800c1014c0 2023-11-16T16:54:55.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.355+0000 7f8013bff700 1 -- 172.21.15.93:0/1333688350 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c10bc80 con 0x7f800c1014c0 2023-11-16T16:54:55.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.355+0000 7f8002ffd700 1 -- 172.21.15.93:0/1333688350 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ffc022470 con 0x7f800c1014c0 2023-11-16T16:54:55.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.355+0000 7f8002ffd700 1 -- 172.21.15.93:0/1333688350 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc017650 con 0x7f800c1014c0 2023-11-16T16:54:55.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.355+0000 7f8013bff700 1 -- 172.21.15.93:0/1333688350 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c10baf0 con 0x7f800c1014c0 2023-11-16T16:54:55.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.356+0000 7f8013bff700 1 -- 172.21.15.93:0/1333688350 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c107870 con 0x7f800c1014c0 2023-11-16T16:54:55.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.358+0000 7f8002ffd700 1 -- 172.21.15.93:0/1333688350 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7ffc022850 con 0x7f800c1014c0 2023-11-16T16:54:55.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.359+0000 7f8002ffd700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7ff80a1150 0x7f7ff80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:55.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.360+0000 7f8002ffd700 1 -- 172.21.15.93:0/1333688350 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f7ffc0ce890 con 0x7f800c1014c0 2023-11-16T16:54:55.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.360+0000 7f801199b700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7ff80a1150 0x7f7ff80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:55.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.360+0000 7f801199b700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7ff80a1150 0x7f7ff80a3600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8008005950 tx=0x7f8008009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:55.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.362+0000 7f8002ffd700 1 -- 172.21.15.93:0/1333688350 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7ffc097c40 con 0x7f800c1014c0 2023-11-16T16:54:55.564 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.563+0000 7f8013bff700 1 -- 172.21.15.93:0/1333688350 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f800c067b00 con 0x7f800c1014c0 2023-11-16T16:54:55.564 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.563+0000 7f8002ffd700 1 -- 172.21.15.93:0/1333688350 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7ffc09b260 con 0x7f800c1014c0 2023-11-16T16:54:55.565 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:54:55.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.565+0000 7f8013bff700 1 -- 172.21.15.93:0/1333688350 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7ff80a1150 msgr2=0x7f7ff80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:55.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.565+0000 7f8013bff700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7ff80a1150 0x7f7ff80a3600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8008005950 tx=0x7f8008009260 comp rx=0 tx=0).stop 2023-11-16T16:54:55.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.565+0000 7f8013bff700 1 -- 172.21.15.93:0/1333688350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1014c0 msgr2=0x7f800c1965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:55.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.565+0000 7f8013bff700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1014c0 0x7f800c1965c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0098e0 tx=0x7f7ffc005740 comp rx=0 tx=0).stop 2023-11-16T16:54:55.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.566+0000 7f8013bff700 1 -- 172.21.15.93:0/1333688350 shutdown_connections 2023-11-16T16:54:55.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.566+0000 7f8013bff700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f7ff80a1150 0x7f7ff80a3600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.566+0000 7f8013bff700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f800c1002d0 0x7f800c196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.566+0000 7f8013bff700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f800c101e50 0x7f800c19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.566+0000 7f8013bff700 1 --2- 172.21.15.93:0/1333688350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1014c0 0x7f800c1965c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.566+0000 7f8013bff700 1 -- 172.21.15.93:0/1333688350 >> 172.21.15.93:0/1333688350 conn(0x7f800c0fb8b0 msgr2=0x7f800c1049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:55.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.566+0000 7f8013bff700 1 -- 172.21.15.93:0/1333688350 shutdown_connections 2023-11-16T16:54:55.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.566+0000 7f8013bff700 1 -- 172.21.15.93:0/1333688350 wait complete. 2023-11-16T16:54:55.587 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:54:55.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.785+0000 7f3ff41ff700 1 -- 172.21.15.93:0/1079477153 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fec073550 msgr2=0x7f3fec0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:55.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.785+0000 7f3ff41ff700 1 --2- 172.21.15.93:0/1079477153 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fec073550 0x7f3fec0739c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009990 tx=0x7f3fdc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.785+0000 7f3ff41ff700 1 -- 172.21.15.93:0/1079477153 shutdown_connections 2023-11-16T16:54:55.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.785+0000 7f3ff41ff700 1 --2- 172.21.15.93:0/1079477153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fec108180 0x7f3fec10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.785+0000 7f3ff41ff700 1 --2- 172.21.15.93:0/1079477153 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fec074b30 0x7f3fec072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.785+0000 7f3ff41ff700 1 --2- 172.21.15.93:0/1079477153 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fec073550 0x7f3fec0739c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.785+0000 7f3ff41ff700 1 -- 172.21.15.93:0/1079477153 >> 172.21.15.93:0/1079477153 conn(0x7f3fec0fb8a0 msgr2=0x7f3fec0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:55.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.785+0000 7f3ff41ff700 1 -- 172.21.15.93:0/1079477153 shutdown_connections 2023-11-16T16:54:55.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.785+0000 7f3ff41ff700 1 -- 172.21.15.93:0/1079477153 wait complete. 2023-11-16T16:54:55.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.786+0000 7f3ff41ff700 1 Processor -- start 2023-11-16T16:54:55.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.786+0000 7f3ff41ff700 1 -- start start 2023-11-16T16:54:55.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.787+0000 7f3ff41ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fec073550 0x7f3fec1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:55.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.787+0000 7f3ff41ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fec074b30 0x7f3fec1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:55.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.787+0000 7f3ff41ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fec108180 0x7f3fec19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:55.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.787+0000 7f3ff41ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec19fe30 con 0x7f3fec074b30 2023-11-16T16:54:55.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.787+0000 7f3ff41ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec19ff70 con 0x7f3fec073550 2023-11-16T16:54:55.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.787+0000 7f3ff41ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec1a0100 con 0x7f3fec108180 2023-11-16T16:54:55.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.787+0000 7f3ff1f9b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fec073550 0x7f3fec1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:55.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.787+0000 7f3ff179a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fec074b30 0x7f3fec1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:55.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.787+0000 7f3ff1f9b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fec073550 0x7f3fec1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:38166/0 (socket says 172.21.15.93:38166) 2023-11-16T16:54:55.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.787+0000 7f3ff1f9b700 1 -- 172.21.15.93:0/3566245238 learned_addr learned my addr 172.21.15.93:0/3566245238 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:55.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.787+0000 7f3ff279c700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fec108180 0x7f3fec19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:55.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.788+0000 7f3ff1f9b700 1 -- 172.21.15.93:0/3566245238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fec108180 msgr2=0x7f3fec19f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:55.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.788+0000 7f3ff1f9b700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fec108180 0x7f3fec19f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.788+0000 7f3ff1f9b700 1 -- 172.21.15.93:0/3566245238 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fec074b30 msgr2=0x7f3fec1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:55.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.788+0000 7f3ff1f9b700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fec074b30 0x7f3fec1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.788+0000 7f3ff1f9b700 1 -- 172.21.15.93:0/3566245238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc0095f0 con 0x7f3fec073550 2023-11-16T16:54:55.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.788+0000 7f3ff179a700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fec074b30 0x7f3fec1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:54:55.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.788+0000 7f3ff1f9b700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fec073550 0x7f3fec1982b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800e970 tx=0x7f3fe800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:55.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.788+0000 7f3fe2ffd700 1 -- 172.21.15.93:0/3566245238 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8009790 con 0x7f3fec073550 2023-11-16T16:54:55.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.788+0000 7f3fe2ffd700 1 -- 172.21.15.93:0/3566245238 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fe800eec0 con 0x7f3fec073550 2023-11-16T16:54:55.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.788+0000 7f3ff41ff700 1 -- 172.21.15.93:0/3566245238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fec10dea0 con 0x7f3fec073550 2023-11-16T16:54:55.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.788+0000 7f3fe2ffd700 1 -- 172.21.15.93:0/3566245238 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8005590 con 0x7f3fec073550 2023-11-16T16:54:55.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.788+0000 7f3ff41ff700 1 -- 172.21.15.93:0/3566245238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec10dd10 con 0x7f3fec073550 2023-11-16T16:54:55.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.789+0000 7f3ff41ff700 1 -- 172.21.15.93:0/3566245238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fec106830 con 0x7f3fec073550 2023-11-16T16:54:55.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.791+0000 7f3fe2ffd700 1 -- 172.21.15.93:0/3566245238 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3fe800cbd0 con 0x7f3fec073550 2023-11-16T16:54:55.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.793+0000 7f3fe2ffd700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3fd80a1150 0x7f3fd80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:55.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.793+0000 7f3fe2ffd700 1 -- 172.21.15.93:0/3566245238 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f3fe8014070 con 0x7f3fec073550 2023-11-16T16:54:55.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.793+0000 7f3ff179a700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3fd80a1150 0x7f3fd80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:55.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.793+0000 7f3ff179a700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3fd80a1150 0x7f3fd80a3600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00b2e0 tx=0x7f3fdc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:55.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.795+0000 7f3fe2ffd700 1 -- 172.21.15.93:0/3566245238 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3fe8096930 con 0x7f3fec073550 2023-11-16T16:54:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:55 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1508861534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:54:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:55 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2423280097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:54:55.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:55 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1333688350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:54:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:55 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1508861534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:54:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:55 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2423280097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:54:55.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:55 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1333688350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:54:55.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.992+0000 7f3ff41ff700 1 -- 172.21.15.93:0/3566245238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3fec067b00 con 0x7f3fec073550 2023-11-16T16:54:55.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.993+0000 7f3fe2ffd700 1 -- 172.21.15.93:0/3566245238 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3fe8004ef0 con 0x7f3fec073550 2023-11-16T16:54:55.994 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-11-16T16:54:55.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.995+0000 7f3ff41ff700 1 -- 172.21.15.93:0/3566245238 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3fd80a1150 msgr2=0x7f3fd80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:55.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.995+0000 7f3ff41ff700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3fd80a1150 0x7f3fd80a3600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00b2e0 tx=0x7f3fdc005fb0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.995+0000 7f3ff41ff700 1 -- 172.21.15.93:0/3566245238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fec073550 msgr2=0x7f3fec1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:55.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.995+0000 7f3ff41ff700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fec073550 0x7f3fec1982b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800e970 tx=0x7f3fe800ec80 comp rx=0 tx=0).stop 2023-11-16T16:54:55.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.995+0000 7f3ff41ff700 1 -- 172.21.15.93:0/3566245238 shutdown_connections 2023-11-16T16:54:55.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.995+0000 7f3ff41ff700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3fd80a1150 0x7f3fd80a3600 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.995+0000 7f3ff41ff700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fec108180 0x7f3fec19f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.995+0000 7f3ff41ff700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fec073550 0x7f3fec1982b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.995+0000 7f3ff41ff700 1 --2- 172.21.15.93:0/3566245238 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fec074b30 0x7f3fec1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:55.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.995+0000 7f3ff41ff700 1 -- 172.21.15.93:0/3566245238 >> 172.21.15.93:0/3566245238 conn(0x7f3fec0fb8a0 msgr2=0x7f3fec0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:55.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.995+0000 7f3ff41ff700 1 -- 172.21.15.93:0/3566245238 shutdown_connections 2023-11-16T16:54:55.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:55.995+0000 7f3ff41ff700 1 -- 172.21.15.93:0/3566245238 wait complete. 2023-11-16T16:54:56.015 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-16T16:54:56.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:55 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1508861534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:54:56.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:55 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2423280097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:54:56.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:55 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1333688350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:54:56.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.215+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/749506160 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3b81014b0 msgr2=0x7ff3b8101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:56.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.215+0000 7ff3bf7ff700 1 --2- 172.21.15.93:0/749506160 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3b81014b0 0x7ff3b8101900 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8009990 tx=0x7ff3a8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.215+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/749506160 shutdown_connections 2023-11-16T16:54:56.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.215+0000 7ff3bf7ff700 1 --2- 172.21.15.93:0/749506160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3b8101e40 0x7ff3b8108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.215+0000 7ff3bf7ff700 1 --2- 172.21.15.93:0/749506160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3b81002c0 0x7ff3b81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.215+0000 7ff3bf7ff700 1 --2- 172.21.15.93:0/749506160 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3b81014b0 0x7ff3b8101900 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.215+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/749506160 >> 172.21.15.93:0/749506160 conn(0x7ff3b80fb8a0 msgr2=0x7ff3b80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:56.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.215+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/749506160 shutdown_connections 2023-11-16T16:54:56.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.215+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/749506160 wait complete. 2023-11-16T16:54:56.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.216+0000 7ff3bf7ff700 1 Processor -- start 2023-11-16T16:54:56.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.217+0000 7ff3bf7ff700 1 -- start start 2023-11-16T16:54:56.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.217+0000 7ff3bf7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3b81002c0 0x7ff3b81960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:56.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.217+0000 7ff3bf7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3b81014b0 0x7ff3b8196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:56.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.217+0000 7ff3bf7ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3b8101e40 0x7ff3b819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:56.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.217+0000 7ff3bf7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b819dc40 con 0x7ff3b81002c0 2023-11-16T16:54:56.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.217+0000 7ff3bf7ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b819dd80 con 0x7ff3b8101e40 2023-11-16T16:54:56.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.217+0000 7ff3bf7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b819df10 con 0x7ff3b81014b0 2023-11-16T16:54:56.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.217+0000 7ff3bd59b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3b81002c0 0x7ff3b81960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:56.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.217+0000 7ff3bd59b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3b81002c0 0x7ff3b81960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57174/0 (socket says 172.21.15.93:57174) 2023-11-16T16:54:56.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.217+0000 7ff3bd59b700 1 -- 172.21.15.93:0/2008944368 learned_addr learned my addr 172.21.15.93:0/2008944368 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:56.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.217+0000 7ff3bcd9a700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3b81014b0 0x7ff3b8196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:56.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.217+0000 7ff3bdd9c700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3b8101e40 0x7ff3b819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:56.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.218+0000 7ff3bd59b700 1 -- 172.21.15.93:0/2008944368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3b81014b0 msgr2=0x7ff3b8196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:56.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.218+0000 7ff3bd59b700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3b81014b0 0x7ff3b8196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.218+0000 7ff3bd59b700 1 -- 172.21.15.93:0/2008944368 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3b8101e40 msgr2=0x7ff3b819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:56.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.218+0000 7ff3bd59b700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3b8101e40 0x7ff3b819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.218+0000 7ff3bd59b700 1 -- 172.21.15.93:0/2008944368 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a80095f0 con 0x7ff3b81002c0 2023-11-16T16:54:56.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.218+0000 7ff3bd59b700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3b81002c0 0x7ff3b81960c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400e970 tx=0x7ff3b400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:56.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.218+0000 7ff3ae7fc700 1 -- 172.21.15.93:0/2008944368 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4009790 con 0x7ff3b81002c0 2023-11-16T16:54:56.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.218+0000 7ff3ae7fc700 1 -- 172.21.15.93:0/2008944368 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3b40044e0 con 0x7ff3b81002c0 2023-11-16T16:54:56.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.218+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/2008944368 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b810bc70 con 0x7ff3b81002c0 2023-11-16T16:54:56.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.218+0000 7ff3ae7fc700 1 -- 172.21.15.93:0/2008944368 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4003e30 con 0x7ff3b81002c0 2023-11-16T16:54:56.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.218+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/2008944368 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b810bae0 con 0x7ff3b81002c0 2023-11-16T16:54:56.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.219+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/2008944368 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b8107860 con 0x7ff3b81002c0 2023-11-16T16:54:56.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.221+0000 7ff3ae7fc700 1 -- 172.21.15.93:0/2008944368 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff3b4003f90 con 0x7ff3b81002c0 2023-11-16T16:54:56.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.222+0000 7ff3ae7fc700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff3a40a1150 0x7ff3a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:56.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.222+0000 7ff3ae7fc700 1 -- 172.21.15.93:0/2008944368 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7ff3b4014070 con 0x7ff3b81002c0 2023-11-16T16:54:56.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.223+0000 7ff3bcd9a700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff3a40a1150 0x7ff3a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:56.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.223+0000 7ff3bcd9a700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff3a40a1150 0x7ff3a40a3600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800b2e0 tx=0x7ff3a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:56.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.226+0000 7ff3ae7fc700 1 -- 172.21.15.93:0/2008944368 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff3b4096360 con 0x7ff3b81002c0 2023-11-16T16:54:56.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.425+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/2008944368 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff3b8067b00 con 0x7ff3b81002c0 2023-11-16T16:54:56.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.425+0000 7ff3ae7fc700 1 -- 172.21.15.93:0/2008944368 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff3b4099980 con 0x7ff3b81002c0 2023-11-16T16:54:56.427 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:54:56.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.428+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/2008944368 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff3a40a1150 msgr2=0x7ff3a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:56.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.428+0000 7ff3bf7ff700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff3a40a1150 0x7ff3a40a3600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800b2e0 tx=0x7ff3a8005fb0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.428+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/2008944368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3b81002c0 msgr2=0x7ff3b81960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:56.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.428+0000 7ff3bf7ff700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3b81002c0 0x7ff3b81960c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400e970 tx=0x7ff3b400ed30 comp rx=0 tx=0).stop 2023-11-16T16:54:56.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.428+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/2008944368 shutdown_connections 2023-11-16T16:54:56.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.428+0000 7ff3bf7ff700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff3a40a1150 0x7ff3a40a3600 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.428+0000 7ff3bf7ff700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3b81002c0 0x7ff3b81960c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.428+0000 7ff3bf7ff700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3b8101e40 0x7ff3b819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.428+0000 7ff3bf7ff700 1 --2- 172.21.15.93:0/2008944368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3b81014b0 0x7ff3b8196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.428+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/2008944368 >> 172.21.15.93:0/2008944368 conn(0x7ff3b80fb8a0 msgr2=0x7ff3b81049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:56.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.428+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/2008944368 shutdown_connections 2023-11-16T16:54:56.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.428+0000 7ff3bf7ff700 1 -- 172.21.15.93:0/2008944368 wait complete. 2023-11-16T16:54:56.449 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-11-16T16:54:56.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.647+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/4186248145 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e9410c860 msgr2=0x7f3e9410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:56.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.647+0000 7f3e9b9ff700 1 --2- 172.21.15.93:0/4186248145 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e9410c860 0x7f3e9410ec40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009f90 tx=0x7f3e8c00b330 comp rx=0 tx=0).stop 2023-11-16T16:54:56.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.647+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/4186248145 shutdown_connections 2023-11-16T16:54:56.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.647+0000 7f3e9b9ff700 1 --2- 172.21.15.93:0/4186248145 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e9410c860 0x7f3e9410ec40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.647+0000 7f3e9b9ff700 1 --2- 172.21.15.93:0/4186248145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e94074b30 0x7f3e94072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.647+0000 7f3e9b9ff700 1 --2- 172.21.15.93:0/4186248145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e94073550 0x7f3e940739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.647+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/4186248145 >> 172.21.15.93:0/4186248145 conn(0x7f3e940fb870 msgr2=0x7f3e940fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:54:56.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.647+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/4186248145 shutdown_connections 2023-11-16T16:54:56.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.647+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/4186248145 wait complete. 2023-11-16T16:54:56.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.648+0000 7f3e9b9ff700 1 Processor -- start 2023-11-16T16:54:56.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.648+0000 7f3e9b9ff700 1 -- start start 2023-11-16T16:54:56.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.648+0000 7f3e9b9ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e94073550 0x7f3e9419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:56.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.648+0000 7f3e9b9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e94074b30 0x7f3e9419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:56.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.648+0000 7f3e9b9ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e9410c860 0x7f3e941a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:56.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.648+0000 7f3e9b9ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e941a4230 con 0x7f3e94073550 2023-11-16T16:54:56.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.648+0000 7f3e9b9ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e941a4370 con 0x7f3e9410c860 2023-11-16T16:54:56.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.648+0000 7f3e9b9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e941a4500 con 0x7f3e94074b30 2023-11-16T16:54:56.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.649+0000 7f3e9979b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e94073550 0x7f3e9419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:56.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.649+0000 7f3e9979b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e94073550 0x7f3e9419c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57196/0 (socket says 172.21.15.93:57196) 2023-11-16T16:54:56.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.649+0000 7f3e9979b700 1 -- 172.21.15.93:0/743550330 learned_addr learned my addr 172.21.15.93:0/743550330 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:54:56.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.649+0000 7f3e98f9a700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e94074b30 0x7f3e9419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:56.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.649+0000 7f3e99f9c700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e9410c860 0x7f3e941a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:56.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.649+0000 7f3e9979b700 1 -- 172.21.15.93:0/743550330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e94074b30 msgr2=0x7f3e9419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:56.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.649+0000 7f3e9979b700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e94074b30 0x7f3e9419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.649+0000 7f3e9979b700 1 -- 172.21.15.93:0/743550330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e9410c860 msgr2=0x7f3e941a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:54:56.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.649+0000 7f3e9979b700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e9410c860 0x7f3e941a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:54:56.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.649+0000 7f3e9979b700 1 -- 172.21.15.93:0/743550330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c009c40 con 0x7f3e94073550 2023-11-16T16:54:56.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.649+0000 7f3e9979b700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e94073550 0x7f3e9419c6b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e970 tx=0x7f3e9000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:56.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.649+0000 7f3e8a7fc700 1 -- 172.21.15.93:0/743550330 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90009790 con 0x7f3e94073550 2023-11-16T16:54:56.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.650+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/743550330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e94112270 con 0x7f3e94073550 2023-11-16T16:54:56.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.650+0000 7f3e8a7fc700 1 -- 172.21.15.93:0/743550330 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e9000c7c0 con 0x7f3e94073550 2023-11-16T16:54:56.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.650+0000 7f3e8a7fc700 1 -- 172.21.15.93:0/743550330 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90005460 con 0x7f3e94073550 2023-11-16T16:54:56.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.650+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/743550330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e941120e0 con 0x7f3e94073550 2023-11-16T16:54:56.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.651+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/743550330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e941023b0 con 0x7f3e94073550 2023-11-16T16:54:56.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.653+0000 7f3e8a7fc700 1 -- 172.21.15.93:0/743550330 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3e90005600 con 0x7f3e94073550 2023-11-16T16:54:56.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.654+0000 7f3e8a7fc700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3e800a1150 0x7f3e800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:54:56.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.654+0000 7f3e8a7fc700 1 -- 172.21.15.93:0/743550330 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f3e90014070 con 0x7f3e94073550 2023-11-16T16:54:56.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.654+0000 7f3e98f9a700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3e800a1150 0x7f3e800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:54:56.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.655+0000 7f3e98f9a700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3e800a1150 0x7f3e800a3600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3e94074850 tx=0x7f3e84009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:54:56.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.658+0000 7f3e8a7fc700 1 -- 172.21.15.93:0/743550330 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3e90096200 con 0x7f3e94073550 2023-11-16T16:54:56.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:56 smithi093 ceph-mon[97863]: pgmap v536: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2023-11-16T16:54:56.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:56 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3566245238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:54:56.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:56 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2008944368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:54:56.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.913+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/743550330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f3e9402cfe0 con 0x7f3e94073550 2023-11-16T16:54:56.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.923+0000 7f3e8a7fc700 1 -- 172.21.15.93:0/743550330 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733244 (secure 0 0 0) 0x7f3e90099820 con 0x7f3e94073550 2023-11-16T16:54:56.935 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:54:56.935 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:54:56.936 INFO:teuthology.orchestra.run.smithi093.stdout: "epoch": 18, 2023-11-16T16:54:56.936 INFO:teuthology.orchestra.run.smithi093.stdout: "active_gid": 14152, 2023-11-16T16:54:56.936 INFO:teuthology.orchestra.run.smithi093.stdout: "active_name": "x", 2023-11-16T16:54:56.936 INFO:teuthology.orchestra.run.smithi093.stdout: "active_addrs": { 2023-11-16T16:54:56.936 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-11-16T16:54:56.936 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:56.937 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-11-16T16:54:56.937 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:6800", 2023-11-16T16:54:56.937 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 3283196047 2023-11-16T16:54:56.937 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.937 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:56.937 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v1", 2023-11-16T16:54:56.937 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:6801", 2023-11-16T16:54:56.938 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 3283196047 2023-11-16T16:54:56.938 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:56.938 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-11-16T16:54:56.938 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.938 INFO:teuthology.orchestra.run.smithi093.stdout: "active_addr": "172.21.15.93:6801/3283196047", 2023-11-16T16:54:56.938 INFO:teuthology.orchestra.run.smithi093.stdout: "active_change": "2023-11-16T16:39:50.432504+0000", 2023-11-16T16:54:56.938 INFO:teuthology.orchestra.run.smithi093.stdout: "active_mgr_features": 4540138322906710015, 2023-11-16T16:54:56.939 INFO:teuthology.orchestra.run.smithi093.stdout: "available": true, 2023-11-16T16:54:56.939 INFO:teuthology.orchestra.run.smithi093.stdout: "standbys": [ 2023-11-16T16:54:56.939 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:56.939 INFO:teuthology.orchestra.run.smithi093.stdout: "gid": 24109, 2023-11-16T16:54:56.939 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "z", 2023-11-16T16:54:56.939 INFO:teuthology.orchestra.run.smithi093.stdout: "mgr_features": 4540138322906710015, 2023-11-16T16:54:56.939 INFO:teuthology.orchestra.run.smithi093.stdout: "available_modules": [ 2023-11-16T16:54:56.940 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:56.940 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alerts", 2023-11-16T16:54:56.940 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:56.940 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:56.940 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:56.940 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:54:56.941 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:54:56.941 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:56.941 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.941 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.941 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:54:56.941 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.941 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.941 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.942 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How frequently to reexamine health status", 2023-11-16T16:54:56.942 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.942 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.942 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.942 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.942 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:56.942 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:56.943 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.943 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.943 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.943 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:56.943 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.943 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.943 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:56.944 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:56.944 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:56.944 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:56.944 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:56.944 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:56.944 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:56.944 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:56.944 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:56.945 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.945 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.945 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.945 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.945 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:56.945 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:56.945 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:56.946 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.946 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.946 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:56.946 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.946 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.946 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.946 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:56.947 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.947 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.947 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.947 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.947 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:56.947 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:56.947 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.948 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.948 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.948 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:56.948 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.948 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.948 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:56.948 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:56.948 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:56.949 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:56.949 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:56.949 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:56.949 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:56.949 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:56.949 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:56.949 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.950 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.950 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.950 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.950 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:56.950 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:56.950 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:56.951 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.951 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.951 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:56.951 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.951 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.951 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.951 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:56.951 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.952 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.952 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.952 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.952 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_destination": { 2023-11-16T16:54:56.952 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_destination", 2023-11-16T16:54:56.952 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.952 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.953 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.953 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:56.953 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.953 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.953 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.953 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email address to send alerts to", 2023-11-16T16:54:56.953 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.953 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.954 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.954 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.954 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_from_name": { 2023-11-16T16:54:56.954 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_from_name", 2023-11-16T16:54:56.954 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.954 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.954 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.955 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "Ceph", 2023-11-16T16:54:56.955 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.955 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.955 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.955 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email From: name", 2023-11-16T16:54:56.955 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.955 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.956 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.956 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.956 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_host": { 2023-11-16T16:54:56.956 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_host", 2023-11-16T16:54:56.956 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.956 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.956 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.956 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:56.957 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.957 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.957 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.957 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP server", 2023-11-16T16:54:56.957 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.957 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.957 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.957 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.958 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_password": { 2023-11-16T16:54:56.958 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_password", 2023-11-16T16:54:56.958 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.958 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.958 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.958 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:56.958 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.959 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.959 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.959 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Password to authenticate with", 2023-11-16T16:54:56.959 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.959 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.959 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.959 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.960 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_port": { 2023-11-16T16:54:56.960 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_port", 2023-11-16T16:54:56.960 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:56.960 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.960 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.960 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "465", 2023-11-16T16:54:56.960 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.961 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.961 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.961 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP port", 2023-11-16T16:54:56.961 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.961 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.961 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.961 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.961 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_sender": { 2023-11-16T16:54:56.962 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_sender", 2023-11-16T16:54:56.962 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.962 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.962 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.962 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:56.962 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.962 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.963 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.963 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP envelope sender", 2023-11-16T16:54:56.963 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.963 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.963 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.963 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.963 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_ssl": { 2023-11-16T16:54:56.963 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_ssl", 2023-11-16T16:54:56.964 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:56.964 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.964 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.964 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:56.964 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.964 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.964 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.964 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-16T16:54:56.965 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.965 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.965 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.965 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.965 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_user": { 2023-11-16T16:54:56.965 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_user", 2023-11-16T16:54:56.965 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.966 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.966 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.966 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:56.966 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.966 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.966 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.966 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "User to authenticate as", 2023-11-16T16:54:56.966 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.967 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.967 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.967 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:56.967 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:56.967 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.967 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:56.967 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "balancer", 2023-11-16T16:54:56.968 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:56.968 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:56.968 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:56.968 INFO:teuthology.orchestra.run.smithi093.stdout: "active": { 2023-11-16T16:54:56.968 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "active", 2023-11-16T16:54:56.968 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:56.968 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.968 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.969 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:56.969 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.969 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.969 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.969 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically balance PGs across cluster", 2023-11-16T16:54:56.969 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.969 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.969 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.970 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.970 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_time": { 2023-11-16T16:54:56.970 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_time", 2023-11-16T16:54:56.970 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.970 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.970 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.970 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0000", 2023-11-16T16:54:56.971 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.971 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.971 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.971 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "beginning time of day to automatically balance", 2023-11-16T16:54:56.971 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-16T16:54:56.971 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.971 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.972 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.972 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_weekday": { 2023-11-16T16:54:56.972 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_weekday", 2023-11-16T16:54:56.972 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:54:56.972 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.972 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.972 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:54:56.972 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-11-16T16:54:56.973 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-11-16T16:54:56.973 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.973 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-16T16:54:56.973 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-16T16:54:56.973 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.973 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.973 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.973 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_max_iterations": { 2023-11-16T16:54:56.973 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_max_iterations", 2023-11-16T16:54:56.974 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:54:56.974 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.974 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.974 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "25", 2023-11-16T16:54:56.974 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-11-16T16:54:56.974 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "250", 2023-11-16T16:54:56.974 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.975 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-16T16:54:56.975 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.975 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.975 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.975 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.975 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_metrics": { 2023-11-16T16:54:56.975 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_metrics", 2023-11-16T16:54:56.975 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.976 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.976 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.976 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "pgs,objects,bytes", 2023-11-16T16:54:56.976 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.976 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.976 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.976 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-16T16:54:56.976 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-16T16:54:56.977 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.977 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.977 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.977 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_step": { 2023-11-16T16:54:56.977 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_step", 2023-11-16T16:54:56.977 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:54:56.977 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.977 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.978 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.5", 2023-11-16T16:54:56.978 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0.001", 2023-11-16T16:54:56.978 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "0.999", 2023-11-16T16:54:56.978 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.978 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "aggressiveness of optimization", 2023-11-16T16:54:56.978 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-16T16:54:56.978 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.979 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.979 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.979 INFO:teuthology.orchestra.run.smithi093.stdout: "end_time": { 2023-11-16T16:54:56.979 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_time", 2023-11-16T16:54:56.979 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.979 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.979 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.979 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2359", 2023-11-16T16:54:56.980 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.980 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.980 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.980 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ending time of day to automatically balance", 2023-11-16T16:54:56.980 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-16T16:54:56.980 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.980 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.980 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.981 INFO:teuthology.orchestra.run.smithi093.stdout: "end_weekday": { 2023-11-16T16:54:56.981 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_weekday", 2023-11-16T16:54:56.981 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:54:56.981 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.981 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.981 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:54:56.981 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-11-16T16:54:56.981 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-11-16T16:54:56.982 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.982 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-16T16:54:56.982 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-16T16:54:56.982 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.982 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.982 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.982 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:56.983 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:56.983 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.983 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.983 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.983 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:56.983 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.983 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.983 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:56.984 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:56.984 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:56.984 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:56.984 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:56.984 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:56.984 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:56.984 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:56.984 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:56.985 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.985 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.985 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.985 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.985 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:56.985 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:56.985 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:56.985 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.986 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.986 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:56.986 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.986 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.986 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.986 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:56.986 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.987 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.987 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.987 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.987 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:56.987 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:56.987 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.987 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.987 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.988 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:56.988 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.988 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.988 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:56.988 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:56.988 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:56.988 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:56.988 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:56.989 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:56.989 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:56.989 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:56.989 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:56.989 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.989 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.989 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.989 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.990 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:56.990 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:56.990 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:56.990 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.990 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.990 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:56.990 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.991 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.991 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.991 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:56.991 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.991 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.991 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.991 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.991 INFO:teuthology.orchestra.run.smithi093.stdout: "min_score": { 2023-11-16T16:54:56.992 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_score", 2023-11-16T16:54:56.992 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:54:56.992 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.992 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.992 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:54:56.992 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.992 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.992 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.992 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-16T16:54:56.993 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.993 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.993 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.993 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.993 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-11-16T16:54:56.993 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-11-16T16:54:56.993 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.993 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.994 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.994 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "upmap", 2023-11-16T16:54:56.994 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.994 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.994 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:56.994 INFO:teuthology.orchestra.run.smithi093.stdout: "crush-compat", 2023-11-16T16:54:56.994 INFO:teuthology.orchestra.run.smithi093.stdout: "none", 2023-11-16T16:54:56.995 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap" 2023-11-16T16:54:56.995 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:56.995 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Balancer mode", 2023-11-16T16:54:56.995 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.995 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.995 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.995 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.995 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_ids": { 2023-11-16T16:54:56.996 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_ids", 2023-11-16T16:54:56.996 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:56.996 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.996 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.996 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:56.996 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.996 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.996 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.996 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-16T16:54:56.997 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.997 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.997 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.997 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.997 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:54:56.997 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:54:56.997 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:56.998 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:56.998 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:56.998 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:54:56.998 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:56.998 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:56.998 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:56.998 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-16T16:54:56.999 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:56.999 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:56.999 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:56.999 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:56.999 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_deviation": { 2023-11-16T16:54:56.999 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_deviation", 2023-11-16T16:54:56.999 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.000 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.000 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.000 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-11-16T16:54:57.000 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-11-16T16:54:57.000 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.000 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.000 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-16T16:54:57.000 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-16T16:54:57.001 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.001 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.001 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.001 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_optimizations": { 2023-11-16T16:54:57.001 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_optimizations", 2023-11-16T16:54:57.001 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:54:57.001 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.001 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.002 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.002 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.002 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.002 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.002 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-16T16:54:57.002 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.002 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.003 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.003 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.003 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.003 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.003 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.003 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cephadm", 2023-11-16T16:54:57.003 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.003 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.004 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.004 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_down_multiplier": { 2023-11-16T16:54:57.004 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_down_multiplier", 2023-11-16T16:54:57.004 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:54:57.004 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.004 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.004 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-11-16T16:54:57.004 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.005 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.005 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.005 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-16T16:54:57.005 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.005 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.005 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.005 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.005 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_refresh_rate": { 2023-11-16T16:54:57.006 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_refresh_rate", 2023-11-16T16:54:57.006 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.006 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.006 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.006 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "20", 2023-11-16T16:54:57.006 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.006 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.006 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.007 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-16T16:54:57.007 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.007 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.007 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.007 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.007 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_starting_port": { 2023-11-16T16:54:57.007 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_starting_port", 2023-11-16T16:54:57.007 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.008 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.008 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.008 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4721", 2023-11-16T16:54:57.008 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.008 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.008 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.008 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-16T16:54:57.008 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.009 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.009 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.009 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.009 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_password": { 2023-11-16T16:54:57.009 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_password", 2023-11-16T16:54:57.009 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.009 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.010 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.010 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.010 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.010 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.010 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.010 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web password", 2023-11-16T16:54:57.010 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.011 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.011 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.011 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.011 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_user": { 2023-11-16T16:54:57.011 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_user", 2023-11-16T16:54:57.011 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.011 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.011 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.012 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.012 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.012 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.012 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.012 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web user", 2023-11-16T16:54:57.012 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.012 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.012 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.013 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.013 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_ptrace": { 2023-11-16T16:54:57.013 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_ptrace", 2023-11-16T16:54:57.013 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.013 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.013 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.013 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.013 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.014 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.014 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.014 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-16T16:54:57.014 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-16T16:54:57.014 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.014 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.014 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.015 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_interval": { 2023-11-16T16:54:57.015 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_interval", 2023-11-16T16:54:57.015 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.015 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.015 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.015 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:54:57.015 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.015 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.015 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.016 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-16T16:54:57.016 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.016 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.016 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.016 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.016 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_memory_target_ratio": { 2023-11-16T16:54:57.016 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_memory_target_ratio", 2023-11-16T16:54:57.016 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:54:57.017 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.017 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.017 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.7", 2023-11-16T16:54:57.017 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.017 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.017 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.017 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-16T16:54:57.017 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.018 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.018 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.018 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.018 INFO:teuthology.orchestra.run.smithi093.stdout: "cgroups_split": { 2023-11-16T16:54:57.018 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cgroups_split", 2023-11-16T16:54:57.018 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.018 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.018 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.019 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.019 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.019 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.019 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.019 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-16T16:54:57.019 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.019 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.020 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.020 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.020 INFO:teuthology.orchestra.run.smithi093.stdout: "config_checks_enabled": { 2023-11-16T16:54:57.020 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_checks_enabled", 2023-11-16T16:54:57.020 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.020 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.020 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.020 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.021 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.021 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.021 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.021 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-16T16:54:57.021 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.021 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.021 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.021 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.022 INFO:teuthology.orchestra.run.smithi093.stdout: "config_dashboard": { 2023-11-16T16:54:57.022 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_dashboard", 2023-11-16T16:54:57.022 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.022 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.022 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.022 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.022 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.023 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.023 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.023 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-16T16:54:57.023 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.023 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.023 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.023 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.023 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_alertmanager": { 2023-11-16T16:54:57.024 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_alertmanager", 2023-11-16T16:54:57.024 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.024 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.024 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.024 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-16T16:54:57.024 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.024 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.024 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.025 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-11-16T16:54:57.025 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.025 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.025 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.025 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.025 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_base": { 2023-11-16T16:54:57.025 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_base", 2023-11-16T16:54:57.026 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.026 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.026 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.026 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-16T16:54:57.026 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.026 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.026 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.026 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Container image name, without the tag", 2023-11-16T16:54:57.027 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.027 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.027 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.027 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.027 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_elasticsearch": { 2023-11-16T16:54:57.027 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_elasticsearch", 2023-11-16T16:54:57.027 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.027 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.028 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.028 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-16T16:54:57.028 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.028 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.028 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.028 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "elasticsearch container image", 2023-11-16T16:54:57.029 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.029 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.029 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.029 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.029 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_grafana": { 2023-11-16T16:54:57.030 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_grafana", 2023-11-16T16:54:57.030 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.030 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.030 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.030 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-16T16:54:57.031 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.031 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.031 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.031 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-11-16T16:54:57.031 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.031 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.031 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.032 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.032 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_haproxy": { 2023-11-16T16:54:57.032 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_haproxy", 2023-11-16T16:54:57.032 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.032 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.032 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.032 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-16T16:54:57.032 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.033 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.033 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.033 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "HAproxy container image", 2023-11-16T16:54:57.033 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.033 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.033 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.033 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.033 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_agent": { 2023-11-16T16:54:57.034 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_agent", 2023-11-16T16:54:57.034 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.034 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.034 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.034 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-16T16:54:57.034 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.034 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.034 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.035 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger agent container image", 2023-11-16T16:54:57.035 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.035 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.035 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.035 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.035 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_collector": { 2023-11-16T16:54:57.035 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_collector", 2023-11-16T16:54:57.035 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.036 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.036 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.036 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-16T16:54:57.036 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.036 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.036 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.036 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger collector container image", 2023-11-16T16:54:57.036 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.037 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.037 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.037 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.037 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_query": { 2023-11-16T16:54:57.037 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_query", 2023-11-16T16:54:57.037 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.037 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.037 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.038 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-16T16:54:57.038 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.038 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.038 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.038 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger query container image", 2023-11-16T16:54:57.038 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.038 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.038 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.039 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.039 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_keepalived": { 2023-11-16T16:54:57.039 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_keepalived", 2023-11-16T16:54:57.039 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.039 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.039 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.039 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-16T16:54:57.039 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.040 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.040 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.040 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Keepalived container image", 2023-11-16T16:54:57.040 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.040 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.040 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.040 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.041 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_loki": { 2023-11-16T16:54:57.041 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_loki", 2023-11-16T16:54:57.041 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.041 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.041 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.041 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-16T16:54:57.041 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.041 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.042 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.042 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Loki container image", 2023-11-16T16:54:57.042 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.042 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.042 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.042 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.042 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_node_exporter": { 2023-11-16T16:54:57.042 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_node_exporter", 2023-11-16T16:54:57.043 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.043 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.043 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.043 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-16T16:54:57.043 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.043 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.043 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.043 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-11-16T16:54:57.044 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.044 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.044 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.044 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.044 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_prometheus": { 2023-11-16T16:54:57.044 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_prometheus", 2023-11-16T16:54:57.044 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.044 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.045 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.045 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-16T16:54:57.045 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.045 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.045 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.045 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-11-16T16:54:57.045 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.045 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.046 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.046 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.046 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_promtail": { 2023-11-16T16:54:57.046 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_promtail", 2023-11-16T16:54:57.046 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.046 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.046 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.046 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-16T16:54:57.047 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.047 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.047 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.047 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Promtail container image", 2023-11-16T16:54:57.047 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.047 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.047 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.047 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.048 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_snmp_gateway": { 2023-11-16T16:54:57.048 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_snmp_gateway", 2023-11-16T16:54:57.048 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.048 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.048 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.048 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-16T16:54:57.048 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.048 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.049 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.049 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SNMP Gateway container image", 2023-11-16T16:54:57.049 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.049 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.049 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.049 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.049 INFO:teuthology.orchestra.run.smithi093.stdout: "container_init": { 2023-11-16T16:54:57.050 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_init", 2023-11-16T16:54:57.050 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.050 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.050 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.050 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.050 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.050 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.051 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.051 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Run podman/docker with `--init`", 2023-11-16T16:54:57.051 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.051 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.051 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.051 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.051 INFO:teuthology.orchestra.run.smithi093.stdout: "daemon_cache_timeout": { 2023-11-16T16:54:57.051 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "daemon_cache_timeout", 2023-11-16T16:54:57.052 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.052 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.052 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.052 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:54:57.052 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.052 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.052 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.052 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-16T16:54:57.053 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.053 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.053 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.053 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.053 INFO:teuthology.orchestra.run.smithi093.stdout: "default_cephadm_command_timeout": { 2023-11-16T16:54:57.053 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_cephadm_command_timeout", 2023-11-16T16:54:57.053 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.054 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.054 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.054 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "900", 2023-11-16T16:54:57.054 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.054 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.054 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.054 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-16T16:54:57.054 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.055 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.055 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.055 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.055 INFO:teuthology.orchestra.run.smithi093.stdout: "default_registry": { 2023-11-16T16:54:57.055 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_registry", 2023-11-16T16:54:57.055 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.055 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.055 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.056 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io", 2023-11-16T16:54:57.056 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.056 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.056 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.056 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-16T16:54:57.056 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.056 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.056 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.057 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.057 INFO:teuthology.orchestra.run.smithi093.stdout: "device_cache_timeout": { 2023-11-16T16:54:57.057 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_cache_timeout", 2023-11-16T16:54:57.057 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.057 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.057 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.057 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1800", 2023-11-16T16:54:57.057 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.058 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.058 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.058 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache device inventory", 2023-11-16T16:54:57.058 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.058 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.058 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.058 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.059 INFO:teuthology.orchestra.run.smithi093.stdout: "device_enhanced_scan": { 2023-11-16T16:54:57.059 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_enhanced_scan", 2023-11-16T16:54:57.059 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.059 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.059 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.059 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.059 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.059 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.060 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.060 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-16T16:54:57.060 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.060 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.060 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.060 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.060 INFO:teuthology.orchestra.run.smithi093.stdout: "facts_cache_timeout": { 2023-11-16T16:54:57.060 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "facts_cache_timeout", 2023-11-16T16:54:57.061 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.061 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.061 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.061 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:54:57.061 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.061 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.061 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.061 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache host facts data", 2023-11-16T16:54:57.062 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.062 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.062 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.062 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.062 INFO:teuthology.orchestra.run.smithi093.stdout: "host_check_interval": { 2023-11-16T16:54:57.062 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "host_check_interval", 2023-11-16T16:54:57.062 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.063 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.063 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.063 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:54:57.063 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.063 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.063 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.063 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to perform a host check", 2023-11-16T16:54:57.063 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.064 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.064 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.064 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.064 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.064 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.064 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.064 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.064 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.065 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.065 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.065 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.065 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.065 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.065 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.065 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.065 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.066 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.066 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.066 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.066 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.066 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.066 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.066 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.066 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.067 INFO:teuthology.orchestra.run.smithi093.stdout: "log_refresh_metadata": { 2023-11-16T16:54:57.067 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_refresh_metadata", 2023-11-16T16:54:57.067 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.067 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.067 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.067 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.067 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.067 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.068 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.068 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-16T16:54:57.068 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.068 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.068 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.068 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.068 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.068 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.069 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.069 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.069 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.069 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.069 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.069 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.069 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.069 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-16T16:54:57.070 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.070 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.070 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.070 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.070 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.070 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.070 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.070 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.071 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.071 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.071 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.071 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.071 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.071 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.071 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.072 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.072 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.072 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.072 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.072 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.072 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.072 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.072 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.073 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.073 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.073 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.073 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.073 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.073 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.073 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.073 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.073 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.074 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.074 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.074 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.074 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.074 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.074 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.074 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.074 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-16T16:54:57.075 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-16T16:54:57.075 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.075 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.075 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.075 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.075 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.075 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.076 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.076 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-16T16:54:57.076 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.076 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.076 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.076 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.076 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-16T16:54:57.076 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-16T16:54:57.077 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.077 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.077 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.077 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "*", 2023-11-16T16:54:57.077 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.077 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.077 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.077 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-16T16:54:57.078 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.078 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.078 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.078 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.078 INFO:teuthology.orchestra.run.smithi093.stdout: "max_count_per_host": { 2023-11-16T16:54:57.078 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_count_per_host", 2023-11-16T16:54:57.078 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.079 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.079 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.079 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.079 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.079 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.079 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.079 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of daemons per service per host", 2023-11-16T16:54:57.079 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.080 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.080 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.080 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.080 INFO:teuthology.orchestra.run.smithi093.stdout: "max_osd_draining_count": { 2023-11-16T16:54:57.080 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_osd_draining_count", 2023-11-16T16:54:57.080 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.080 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.080 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.081 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.081 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.081 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.081 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.081 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-16T16:54:57.081 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.081 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.081 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.082 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.082 INFO:teuthology.orchestra.run.smithi093.stdout: "migration_current": { 2023-11-16T16:54:57.082 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "migration_current", 2023-11-16T16:54:57.082 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.082 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.082 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.082 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.082 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.083 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.083 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.083 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "internal - do not modify", 2023-11-16T16:54:57.083 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.083 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.083 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.083 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.083 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-11-16T16:54:57.084 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-11-16T16:54:57.084 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.084 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.084 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.084 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "root", 2023-11-16T16:54:57.084 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.084 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.084 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.085 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm-package", 2023-11-16T16:54:57.085 INFO:teuthology.orchestra.run.smithi093.stdout: "root" 2023-11-16T16:54:57.085 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.085 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "mode for remote execution of cephadm", 2023-11-16T16:54:57.085 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.085 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.085 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.085 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.086 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_alerts_path": { 2023-11-16T16:54:57.086 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_alerts_path", 2023-11-16T16:54:57.086 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.086 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.086 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.086 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-16T16:54:57.086 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.086 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.087 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.087 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-16T16:54:57.087 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.087 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.087 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.087 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.087 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_password": { 2023-11-16T16:54:57.087 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_password", 2023-11-16T16:54:57.088 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.088 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.088 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.088 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.088 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.088 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.088 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.088 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web password", 2023-11-16T16:54:57.088 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.089 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.089 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.089 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.089 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_user": { 2023-11-16T16:54:57.089 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_user", 2023-11-16T16:54:57.089 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.089 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.090 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.090 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.090 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.090 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.090 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.090 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web user", 2023-11-16T16:54:57.090 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.090 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_insecure": { 2023-11-16T16:54:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_insecure", 2023-11-16T16:54:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.092 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.092 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.092 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.092 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-16T16:54:57.092 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.092 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.092 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.092 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.093 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_password": { 2023-11-16T16:54:57.093 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_password", 2023-11-16T16:54:57.093 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.093 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.093 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.093 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.093 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.093 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.094 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.094 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-16T16:54:57.094 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.094 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.094 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.094 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.094 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_url": { 2023-11-16T16:54:57.094 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_url", 2023-11-16T16:54:57.095 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.095 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.095 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.095 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.095 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.095 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.095 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.095 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-16T16:54:57.096 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.096 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.096 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.096 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.096 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_username": { 2023-11-16T16:54:57.096 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_username", 2023-11-16T16:54:57.096 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.096 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.097 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.097 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.097 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.097 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.097 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.097 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-16T16:54:57.097 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.097 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.098 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.098 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.098 INFO:teuthology.orchestra.run.smithi093.stdout: "secure_monitoring_stack": { 2023-11-16T16:54:57.098 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "secure_monitoring_stack", 2023-11-16T16:54:57.098 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.098 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.098 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.099 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.099 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.099 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.099 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.099 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-16T16:54:57.099 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.099 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.100 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.100 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.100 INFO:teuthology.orchestra.run.smithi093.stdout: "service_discovery_port": { 2023-11-16T16:54:57.100 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "service_discovery_port", 2023-11-16T16:54:57.100 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.100 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.100 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.100 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8765", 2023-11-16T16:54:57.101 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.101 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.101 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.101 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "cephadm service discovery port", 2023-11-16T16:54:57.101 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.101 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.101 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.101 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.101 INFO:teuthology.orchestra.run.smithi093.stdout: "ssh_config_file": { 2023-11-16T16:54:57.102 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssh_config_file", 2023-11-16T16:54:57.102 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.102 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.102 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.102 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.102 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.102 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.103 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.103 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-16T16:54:57.103 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.103 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.103 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.103 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.103 INFO:teuthology.orchestra.run.smithi093.stdout: "use_agent": { 2023-11-16T16:54:57.103 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_agent", 2023-11-16T16:54:57.104 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.104 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.104 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.104 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.104 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.104 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.104 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.104 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-16T16:54:57.105 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.105 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.105 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.105 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.105 INFO:teuthology.orchestra.run.smithi093.stdout: "use_repo_digest": { 2023-11-16T16:54:57.105 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_repo_digest", 2023-11-16T16:54:57.105 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.105 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.106 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.106 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.106 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.106 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.106 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.106 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-16T16:54:57.106 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.106 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.107 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.107 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.107 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_failed_host_check": { 2023-11-16T16:54:57.107 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_failed_host_check", 2023-11-16T16:54:57.107 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.107 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.107 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.107 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.108 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.108 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.108 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.108 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if the host check fails", 2023-11-16T16:54:57.108 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.108 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.108 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.108 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.109 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_daemons": { 2023-11-16T16:54:57.109 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_daemons", 2023-11-16T16:54:57.109 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.109 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.109 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.109 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.109 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.110 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.110 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.110 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-16T16:54:57.110 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.110 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.110 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.110 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.110 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_hosts": { 2023-11-16T16:54:57.111 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_hosts", 2023-11-16T16:54:57.111 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.111 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.111 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.111 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.111 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.111 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.111 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.113 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-16T16:54:57.113 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.113 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.113 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.113 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.113 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.113 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.114 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.114 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crash", 2023-11-16T16:54:57.114 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.114 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.114 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.114 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.114 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.115 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.115 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.115 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.115 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.115 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.115 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.115 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.115 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.116 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.116 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.116 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.116 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.116 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.116 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.116 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.116 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.116 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.117 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.117 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.117 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.117 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.117 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.117 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.117 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.117 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.118 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.118 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.118 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.118 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.118 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.118 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.118 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.118 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.119 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.119 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.119 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.119 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.119 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.119 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.119 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.120 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.120 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.120 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.120 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.120 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.120 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.120 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.120 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.121 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.121 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.121 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.121 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.121 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.121 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.121 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.121 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.122 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.122 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.122 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.122 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.122 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.122 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.122 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.123 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.123 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.123 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.123 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.123 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.123 INFO:teuthology.orchestra.run.smithi093.stdout: "retain_interval": { 2023-11-16T16:54:57.123 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retain_interval", 2023-11-16T16:54:57.123 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.124 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.124 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.124 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "31536000", 2023-11-16T16:54:57.124 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.124 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.124 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.124 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-16T16:54:57.124 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.125 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.125 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.125 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.125 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_recent_interval": { 2023-11-16T16:54:57.125 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_recent_interval", 2023-11-16T16:54:57.125 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.125 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.125 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.126 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1209600", 2023-11-16T16:54:57.126 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.126 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.126 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.126 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-16T16:54:57.126 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.126 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.126 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.127 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.127 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.127 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.127 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.127 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dashboard", 2023-11-16T16:54:57.127 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.127 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.127 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.127 INFO:teuthology.orchestra.run.smithi093.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-16T16:54:57.128 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-16T16:54:57.128 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.128 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.128 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.128 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.128 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.128 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.129 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.129 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.129 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.129 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.129 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.129 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.129 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_HOST": { 2023-11-16T16:54:57.129 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-16T16:54:57.130 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.130 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.130 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.130 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.130 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.130 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.130 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.130 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.131 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.131 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.131 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.131 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.131 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-16T16:54:57.131 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-16T16:54:57.131 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.131 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.132 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.132 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.132 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.132 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.132 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.132 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.132 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.132 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.133 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.133 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.133 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_ENABLED": { 2023-11-16T16:54:57.133 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_ENABLED", 2023-11-16T16:54:57.133 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.133 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.133 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.133 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.134 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.134 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.134 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.134 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.134 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.134 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.134 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.134 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.135 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-16T16:54:57.135 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-16T16:54:57.135 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.135 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.135 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.135 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.135 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.135 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.136 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.136 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.136 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.136 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.136 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.136 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.136 INFO:teuthology.orchestra.run.smithi093.stdout: "ENABLE_BROWSABLE_API": { 2023-11-16T16:54:57.136 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-16T16:54:57.137 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.137 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.137 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.137 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.137 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.137 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.137 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.137 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.138 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.138 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.138 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.138 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.138 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-16T16:54:57.138 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-16T16:54:57.138 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.139 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.139 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.139 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.139 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.139 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.139 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.139 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.139 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.140 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.140 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.140 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.140 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-16T16:54:57.140 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-16T16:54:57.140 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.140 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.140 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.141 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.141 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.141 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.141 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.141 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.141 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.141 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.141 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.142 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.142 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-16T16:54:57.142 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-16T16:54:57.142 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.142 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.142 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.142 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.142 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.143 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.143 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.143 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.143 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.143 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.143 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.143 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.143 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-16T16:54:57.143 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-16T16:54:57.144 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.144 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.144 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.144 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.144 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.144 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.144 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.144 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.145 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.145 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.145 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.145 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.145 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-16T16:54:57.145 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-16T16:54:57.145 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.146 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.146 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.146 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.146 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.146 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.146 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.146 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.146 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.147 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.147 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.147 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.147 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-16T16:54:57.147 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-16T16:54:57.147 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.147 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.147 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.147 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.148 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.148 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.148 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.148 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.148 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.148 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.148 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.149 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.149 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-16T16:54:57.149 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-16T16:54:57.149 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.149 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.149 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.149 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.150 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.150 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.150 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.150 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.150 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.150 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.150 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.150 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.151 INFO:teuthology.orchestra.run.smithi093.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-16T16:54:57.151 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-16T16:54:57.151 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.151 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.151 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.151 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.151 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.151 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.152 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.152 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.152 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.152 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.152 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.152 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.152 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_PASSWORD": { 2023-11-16T16:54:57.152 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-16T16:54:57.153 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.153 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.153 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.153 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.153 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.153 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.153 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.153 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.154 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.154 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.154 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.154 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.154 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-16T16:54:57.154 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-16T16:54:57.154 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.154 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.155 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.155 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.155 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.155 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.155 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.155 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.155 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.155 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.156 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.156 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.156 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_URL": { 2023-11-16T16:54:57.156 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_URL", 2023-11-16T16:54:57.156 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.156 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.156 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.156 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.157 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.157 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.157 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.157 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.157 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.157 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.157 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.157 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.158 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_USERNAME": { 2023-11-16T16:54:57.158 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-16T16:54:57.158 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.158 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.158 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.158 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.158 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.159 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.159 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.159 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.159 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.159 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.159 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.159 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.160 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-16T16:54:57.160 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-16T16:54:57.160 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.160 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.160 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.160 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.160 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.160 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.161 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.161 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.161 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.161 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.161 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.161 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.161 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-16T16:54:57.162 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-16T16:54:57.162 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.162 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.162 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.162 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.162 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.162 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.162 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.163 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.163 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.163 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.163 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.163 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.163 INFO:teuthology.orchestra.run.smithi093.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-16T16:54:57.163 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-16T16:54:57.163 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.164 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.164 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.164 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.164 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.164 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.164 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.164 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.164 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.165 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.165 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.165 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.165 INFO:teuthology.orchestra.run.smithi093.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-16T16:54:57.165 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-16T16:54:57.165 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.165 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.165 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.166 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.166 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.166 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.166 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.166 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.166 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.166 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.166 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.167 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.167 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_HOST": { 2023-11-16T16:54:57.167 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-16T16:54:57.167 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.167 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.167 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.167 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.167 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.168 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.168 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.168 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.168 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.168 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.168 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.168 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.168 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-16T16:54:57.169 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-16T16:54:57.169 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.169 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.169 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.169 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.169 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.169 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.170 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.170 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.170 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.170 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.170 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.170 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.170 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-16T16:54:57.171 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-16T16:54:57.171 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.171 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.171 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.171 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.171 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.171 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.171 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.172 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.172 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.172 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.172 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.172 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.172 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-16T16:54:57.172 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-16T16:54:57.172 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.173 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.173 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.173 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.173 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.173 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.173 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.173 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.173 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.174 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.174 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.174 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.174 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-16T16:54:57.174 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-16T16:54:57.174 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.174 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.175 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.175 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.175 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.175 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.175 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.175 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.175 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.175 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.176 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.176 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.176 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-16T16:54:57.176 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-16T16:54:57.176 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.176 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.176 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.176 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.177 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.177 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.177 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.177 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.177 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.177 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.177 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.177 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.178 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-16T16:54:57.178 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-16T16:54:57.178 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.178 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.178 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.178 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.178 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.179 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.179 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.179 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.179 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.179 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.179 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.179 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.180 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-16T16:54:57.180 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-16T16:54:57.180 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.180 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.180 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.180 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.180 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.180 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.181 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.181 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.181 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.181 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.181 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.181 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.181 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-16T16:54:57.181 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-16T16:54:57.182 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.182 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.182 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.182 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.182 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.182 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.182 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.183 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.183 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.183 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.183 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.183 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.183 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_ENABLED": { 2023-11-16T16:54:57.183 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-16T16:54:57.183 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.184 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.184 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.184 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.184 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.184 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.184 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.184 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.184 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-16T16:54:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-16T16:54:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-16T16:54:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-16T16:54:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-16T16:54:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-16T16:54:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-16T16:54:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8", 2023-11-16T16:54:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-16T16:54:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-16T16:54:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "45", 2023-11-16T16:54:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ACCESS_KEY": { 2023-11-16T16:54:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-16T16:54:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-16T16:54:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-16T16:54:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.195 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.195 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.195 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.195 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.195 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.195 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.195 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.195 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.196 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.196 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SECRET_KEY": { 2023-11-16T16:54:57.196 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-16T16:54:57.196 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.196 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.196 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.196 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.196 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.197 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.197 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.197 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.197 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.197 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.197 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.197 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.197 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SSL_VERIFY": { 2023-11-16T16:54:57.198 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-16T16:54:57.198 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.198 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.198 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.198 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.198 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.198 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.199 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.199 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.199 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.199 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.199 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.199 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.199 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-16T16:54:57.200 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-16T16:54:57.200 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.200 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.200 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.200 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:54:57.200 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.200 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.200 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.201 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.201 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.201 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.201 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.201 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.201 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-16T16:54:57.201 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-16T16:54:57.201 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.202 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.202 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.202 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.202 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.202 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.202 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.202 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.202 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.203 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.203 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.203 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.203 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-16T16:54:57.203 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-16T16:54:57.203 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.203 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.203 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.204 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-11-16T16:54:57.204 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.204 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.204 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.204 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.204 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.204 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.204 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.205 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.205 INFO:teuthology.orchestra.run.smithi093.stdout: "cross_origin_url": { 2023-11-16T16:54:57.205 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cross_origin_url", 2023-11-16T16:54:57.205 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.205 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.205 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.205 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.205 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.206 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.206 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.206 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.206 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.206 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.206 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.206 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.206 INFO:teuthology.orchestra.run.smithi093.stdout: "crt_file": { 2023-11-16T16:54:57.207 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crt_file", 2023-11-16T16:54:57.207 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.207 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.207 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.207 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.207 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.207 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.207 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.208 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.208 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.208 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.208 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.208 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.208 INFO:teuthology.orchestra.run.smithi093.stdout: "debug": { 2023-11-16T16:54:57.208 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "debug", 2023-11-16T16:54:57.208 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.209 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.209 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.209 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.209 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.209 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.209 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.209 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable/disable debug options", 2023-11-16T16:54:57.210 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.210 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.210 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.210 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.210 INFO:teuthology.orchestra.run.smithi093.stdout: "jwt_token_ttl": { 2023-11-16T16:54:57.210 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "jwt_token_ttl", 2023-11-16T16:54:57.210 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.210 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.211 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.211 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "28800", 2023-11-16T16:54:57.211 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.211 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.211 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.211 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.211 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.211 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.212 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.212 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.212 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-11-16T16:54:57.212 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-11-16T16:54:57.212 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.212 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.212 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.212 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.213 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.213 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.213 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.213 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.213 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.213 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.213 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.213 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.214 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.214 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.214 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.214 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.214 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.214 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.214 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.215 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.215 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.215 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.215 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.215 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.215 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.215 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.215 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.216 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.216 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.216 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.216 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.216 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.216 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.216 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.217 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.217 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.217 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.217 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.217 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.217 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.217 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.217 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.217 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.218 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.218 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.218 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.218 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.218 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.218 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.218 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.218 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.219 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.219 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.219 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.219 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.219 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.219 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.219 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.219 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.220 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.220 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.220 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.220 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.220 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.220 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.220 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.220 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.220 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.221 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.221 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.221 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.221 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.221 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.221 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.221 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.221 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.222 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.222 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.222 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.222 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.222 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.222 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.222 INFO:teuthology.orchestra.run.smithi093.stdout: "motd": { 2023-11-16T16:54:57.222 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "motd", 2023-11-16T16:54:57.223 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.223 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.223 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.223 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.223 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.223 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.223 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.223 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "The message of the day", 2023-11-16T16:54:57.224 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.224 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.224 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.224 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.224 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect_resolve_ip_addr": { 2023-11-16T16:54:57.224 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "redirect_resolve_ip_addr", 2023-11-16T16:54:57.224 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.224 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.224 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.225 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.225 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.225 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.225 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.225 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.225 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.225 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.225 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.225 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.226 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-11-16T16:54:57.226 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-11-16T16:54:57.226 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.226 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.226 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.226 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-11-16T16:54:57.226 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.226 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.227 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.227 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.227 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.227 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.227 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.227 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.227 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-11-16T16:54:57.227 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-11-16T16:54:57.228 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.228 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.228 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.228 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8080", 2023-11-16T16:54:57.228 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.228 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.228 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.228 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.229 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.229 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.229 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.229 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.229 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-11-16T16:54:57.229 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-11-16T16:54:57.229 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.229 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.230 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.230 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.230 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.230 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.230 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.230 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.230 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.231 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.231 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.231 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.231 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl_server_port": { 2023-11-16T16:54:57.231 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl_server_port", 2023-11-16T16:54:57.231 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.231 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.231 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.232 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8443", 2023-11-16T16:54:57.232 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.232 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.232 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.232 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.232 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.232 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.232 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.233 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.233 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-11-16T16:54:57.233 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-11-16T16:54:57.233 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.233 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.233 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.233 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "redirect", 2023-11-16T16:54:57.233 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.234 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.234 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.234 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.234 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect" 2023-11-16T16:54:57.234 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.234 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.234 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.234 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.235 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.235 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.235 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-11-16T16:54:57.235 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-11-16T16:54:57.235 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.235 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.235 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.235 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-11-16T16:54:57.236 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-11-16T16:54:57.236 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-11-16T16:54:57.236 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.236 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.236 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.236 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.236 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.236 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.237 INFO:teuthology.orchestra.run.smithi093.stdout: "url_prefix": { 2023-11-16T16:54:57.237 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url_prefix", 2023-11-16T16:54:57.237 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.237 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.237 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.237 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.237 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.237 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.238 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.238 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.238 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.238 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.238 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.238 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.238 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.238 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.239 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.239 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "devicehealth", 2023-11-16T16:54:57.239 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.239 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.239 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.239 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_monitoring": { 2023-11-16T16:54:57.239 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_monitoring", 2023-11-16T16:54:57.240 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.240 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.240 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.240 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.240 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.240 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.240 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.240 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "monitor device health metrics", 2023-11-16T16:54:57.241 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.241 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.241 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.241 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.241 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.241 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.241 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.241 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.242 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.242 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.242 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.242 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.242 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.242 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.242 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.242 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.243 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.243 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.243 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.243 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.243 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.243 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.243 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.243 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.244 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.244 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.244 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.244 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.244 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.244 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.244 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.245 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.245 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.245 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.245 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.245 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.245 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.245 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.245 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.246 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.246 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.246 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.246 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.246 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.246 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.246 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.246 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.247 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.247 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.247 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.247 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.247 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.247 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.247 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.247 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.247 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.248 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.248 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.248 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.248 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.248 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.248 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.248 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.248 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.249 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.249 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.249 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.249 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.249 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.249 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.249 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.250 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.250 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.250 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.250 INFO:teuthology.orchestra.run.smithi093.stdout: "mark_out_threshold": { 2023-11-16T16:54:57.250 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mark_out_threshold", 2023-11-16T16:54:57.250 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.250 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.250 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.251 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2419200", 2023-11-16T16:54:57.251 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.251 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.251 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.251 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-16T16:54:57.251 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.251 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.252 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.252 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.252 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_name": { 2023-11-16T16:54:57.252 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_name", 2023-11-16T16:54:57.252 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.252 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.252 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.252 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "device_health_metrics", 2023-11-16T16:54:57.253 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.253 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.253 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.253 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-16T16:54:57.253 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.253 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.253 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.253 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.254 INFO:teuthology.orchestra.run.smithi093.stdout: "retention_period": { 2023-11-16T16:54:57.254 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retention_period", 2023-11-16T16:54:57.254 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.254 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.254 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.254 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15552000", 2023-11-16T16:54:57.254 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.255 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.255 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.255 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain device health metrics", 2023-11-16T16:54:57.255 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.255 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.255 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.255 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.255 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_frequency": { 2023-11-16T16:54:57.256 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_frequency", 2023-11-16T16:54:57.256 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.256 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.256 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.256 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-11-16T16:54:57.256 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.256 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.256 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.257 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-16T16:54:57.257 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.257 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.257 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.257 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.257 INFO:teuthology.orchestra.run.smithi093.stdout: "self_heal": { 2023-11-16T16:54:57.257 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "self_heal", 2023-11-16T16:54:57.257 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-16T16:54:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.259 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.259 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.259 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.259 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:54:57.259 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:54:57.259 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.259 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.259 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.260 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:54:57.260 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.260 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.260 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.260 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and check device health", 2023-11-16T16:54:57.260 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.260 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.260 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.261 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.261 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_threshold": { 2023-11-16T16:54:57.261 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_threshold", 2023-11-16T16:54:57.261 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.261 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.261 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.261 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7257600", 2023-11-16T16:54:57.261 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.262 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.262 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.262 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-16T16:54:57.262 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.262 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.262 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.262 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.263 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.263 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.263 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.263 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "diskprediction_local", 2023-11-16T16:54:57.263 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.263 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.263 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.263 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.264 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.264 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.264 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.264 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.264 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.264 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.264 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.264 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.265 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.265 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.265 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.265 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.265 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.265 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.265 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.265 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.266 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.266 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.266 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.266 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.266 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.266 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.266 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.266 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.267 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.267 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.267 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.267 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.267 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.267 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.267 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.267 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.268 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.268 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.268 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.268 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.268 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.268 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.268 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.268 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.268 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.269 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.269 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.269 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.269 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.269 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.269 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.269 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.269 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.270 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.270 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.270 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.270 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.270 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.270 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.270 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.270 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.270 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.271 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.271 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.271 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.271 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.271 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.271 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.271 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.271 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.272 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.272 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.272 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.272 INFO:teuthology.orchestra.run.smithi093.stdout: "predict_interval": { 2023-11-16T16:54:57.272 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predict_interval", 2023-11-16T16:54:57.272 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.272 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.272 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.272 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-11-16T16:54:57.273 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.273 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.273 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.273 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.273 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.273 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.273 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.273 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.273 INFO:teuthology.orchestra.run.smithi093.stdout: "predictor_model": { 2023-11-16T16:54:57.274 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predictor_model", 2023-11-16T16:54:57.274 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.274 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.274 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.274 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "prophetstor", 2023-11-16T16:54:57.274 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.274 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.274 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.274 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.275 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.275 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.275 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.275 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.275 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:54:57.275 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:54:57.275 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.275 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.275 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.276 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:54:57.276 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.276 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.276 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.276 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.276 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.276 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.276 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.276 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.277 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.277 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.277 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.277 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "influx", 2023-11-16T16:54:57.277 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": false, 2023-11-16T16:54:57.277 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "influxdb python module not found", 2023-11-16T16:54:57.277 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.277 INFO:teuthology.orchestra.run.smithi093.stdout: "batch_size": { 2023-11-16T16:54:57.278 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "batch_size", 2023-11-16T16:54:57.278 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.278 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.278 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.278 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5000", 2023-11-16T16:54:57.278 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.278 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.278 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.279 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-16T16:54:57.279 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.279 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.279 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.279 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.279 INFO:teuthology.orchestra.run.smithi093.stdout: "database": { 2023-11-16T16:54:57.279 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "database", 2023-11-16T16:54:57.280 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.280 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.280 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.280 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "ceph", 2023-11-16T16:54:57.280 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.280 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.280 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.280 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-16T16:54:57.281 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.281 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.281 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.281 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.281 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": { 2023-11-16T16:54:57.281 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "hostname", 2023-11-16T16:54:57.281 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.282 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.282 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.282 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.282 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.282 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.282 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.282 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server hostname", 2023-11-16T16:54:57.283 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.283 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.283 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.283 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.283 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:54:57.283 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:54:57.283 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.283 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.283 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.284 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "30", 2023-11-16T16:54:57.284 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "5", 2023-11-16T16:54:57.284 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.284 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.284 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-16T16:54:57.284 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.284 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.284 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.284 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.285 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.285 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.285 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.285 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.285 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.285 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.285 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.285 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.285 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.286 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.286 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.286 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.286 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.286 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.286 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.286 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.286 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.287 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.287 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.287 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.287 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.287 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.287 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.287 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.287 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.287 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.288 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.288 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.288 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.288 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.288 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.288 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.288 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.288 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.289 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.289 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.289 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.289 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.289 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.289 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.289 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.289 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.290 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.290 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.290 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.290 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.290 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.290 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.290 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.290 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.291 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.291 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.291 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.291 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.291 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.291 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.291 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.291 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.292 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.292 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.292 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.292 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.292 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.292 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.292 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.292 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.292 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.293 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.293 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.293 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.293 INFO:teuthology.orchestra.run.smithi093.stdout: "password": { 2023-11-16T16:54:57.293 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "password", 2023-11-16T16:54:57.293 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.293 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.293 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.293 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.294 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.294 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.294 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.294 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "password of InfluxDB server user", 2023-11-16T16:54:57.294 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.294 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.294 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.294 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.295 INFO:teuthology.orchestra.run.smithi093.stdout: "port": { 2023-11-16T16:54:57.295 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "port", 2023-11-16T16:54:57.295 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.295 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.295 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.295 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8086", 2023-11-16T16:54:57.295 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.295 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.295 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.296 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server port", 2023-11-16T16:54:57.296 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.296 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.296 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.296 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.296 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-11-16T16:54:57.296 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-11-16T16:54:57.296 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.297 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.297 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.297 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "false", 2023-11-16T16:54:57.297 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.297 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.297 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.297 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-16T16:54:57.297 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.297 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.298 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.298 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.298 INFO:teuthology.orchestra.run.smithi093.stdout: "threads": { 2023-11-16T16:54:57.298 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threads", 2023-11-16T16:54:57.298 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.298 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.298 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.298 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-11-16T16:54:57.299 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-11-16T16:54:57.299 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "32", 2023-11-16T16:54:57.299 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.299 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-16T16:54:57.299 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.299 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.299 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.299 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.300 INFO:teuthology.orchestra.run.smithi093.stdout: "username": { 2023-11-16T16:54:57.300 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "username", 2023-11-16T16:54:57.300 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.300 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.300 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.300 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.300 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.300 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.300 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.301 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "username of InfluxDB server user", 2023-11-16T16:54:57.301 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.301 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.301 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.301 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.301 INFO:teuthology.orchestra.run.smithi093.stdout: "verify_ssl": { 2023-11-16T16:54:57.301 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "verify_ssl", 2023-11-16T16:54:57.301 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.301 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.302 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.302 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "true", 2023-11-16T16:54:57.302 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.302 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.302 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.302 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-16T16:54:57.302 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.302 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.303 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.303 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.303 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.303 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.303 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.303 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "insights", 2023-11-16T16:54:57.303 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.303 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.304 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.304 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.304 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.304 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.304 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.304 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.304 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.304 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.304 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.305 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.305 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.305 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.305 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.305 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.305 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.305 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.305 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.305 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.306 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.306 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.306 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.306 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.306 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.306 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.306 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.306 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.307 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.307 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.307 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.307 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.307 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.307 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.307 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.307 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.308 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.308 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.308 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.308 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.308 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.308 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.308 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.308 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.309 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.309 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.309 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.309 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.309 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.309 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.309 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.309 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.310 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.310 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.310 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.310 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.310 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.310 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.310 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.310 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.310 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.311 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.311 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.311 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.311 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.311 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.311 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.311 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.311 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.312 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.312 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.312 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.312 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.312 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.312 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.312 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.312 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "iostat", 2023-11-16T16:54:57.312 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.313 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.313 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.313 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.313 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.313 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.313 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.313 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.313 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.313 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.314 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.314 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.314 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.314 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.314 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.314 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.314 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.314 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.315 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.315 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.315 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.315 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.315 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.315 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.315 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.315 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.316 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.316 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.316 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.316 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.316 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.316 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.316 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.316 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.316 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.317 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.317 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.317 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.317 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.317 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.317 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.317 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.317 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.318 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.318 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.318 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.318 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.318 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.318 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.318 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.318 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.319 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.319 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.319 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.319 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.319 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.319 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.319 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.319 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.319 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.320 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.320 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.320 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.320 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.320 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.320 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.320 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.321 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.321 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.321 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.321 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.321 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.321 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.321 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.321 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.321 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.322 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "k8sevents", 2023-11-16T16:54:57.322 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.322 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.322 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.322 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_event_retention_days": { 2023-11-16T16:54:57.322 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ceph_event_retention_days", 2023-11-16T16:54:57.322 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.323 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.323 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.323 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7", 2023-11-16T16:54:57.323 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.323 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.323 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.323 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-16T16:54:57.323 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.324 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.324 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.324 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.324 INFO:teuthology.orchestra.run.smithi093.stdout: "config_check_secs": { 2023-11-16T16:54:57.324 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_check_secs", 2023-11-16T16:54:57.324 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.324 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.324 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "10", 2023-11-16T16:54:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-16T16:54:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.326 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.326 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.326 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.326 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.326 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.326 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.326 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.326 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.327 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.327 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.327 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.327 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.327 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.327 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.327 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.327 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.328 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.328 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.328 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.328 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.328 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.328 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.328 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.328 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.329 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.329 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.329 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.329 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.329 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.329 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.329 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.330 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.330 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.330 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.330 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.330 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.330 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.330 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.330 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.331 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.331 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.331 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.331 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.331 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.331 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.331 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.331 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.332 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.332 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.332 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.332 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.332 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.332 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.332 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.332 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.333 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.333 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.333 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.333 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.333 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.333 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.333 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.333 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.334 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.334 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.334 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.334 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.334 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.334 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.334 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.334 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.335 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.335 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.335 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.335 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "localpool", 2023-11-16T16:54:57.335 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.335 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.335 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.335 INFO:teuthology.orchestra.run.smithi093.stdout: "failure_domain": { 2023-11-16T16:54:57.336 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "failure_domain", 2023-11-16T16:54:57.336 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.336 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.336 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.336 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "host", 2023-11-16T16:54:57.336 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.336 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.336 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.337 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "failure domain for any created local pool", 2023-11-16T16:54:57.337 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-16T16:54:57.337 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.337 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.337 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.337 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.337 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.337 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.338 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.338 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.338 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.338 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.338 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.338 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.338 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.338 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.339 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.339 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.339 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.339 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.339 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.339 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.339 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.339 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.340 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.340 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.340 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.340 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.340 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.340 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.340 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.340 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.341 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.341 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.341 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.341 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.341 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.341 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.341 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.341 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.342 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.342 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.342 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.342 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.342 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.342 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.342 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.342 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.343 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.343 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.343 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.343 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.343 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.343 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.343 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.343 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.344 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.344 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.344 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.344 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.344 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.344 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.344 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.344 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.345 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.345 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.345 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.345 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.345 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.345 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.345 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.345 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.346 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.346 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.346 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.346 INFO:teuthology.orchestra.run.smithi093.stdout: "min_size": { 2023-11-16T16:54:57.346 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_size", 2023-11-16T16:54:57.346 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.346 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.346 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.347 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.347 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.347 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.347 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.347 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default min_size for any created local pool", 2023-11-16T16:54:57.347 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-16T16:54:57.347 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.347 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.348 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.348 INFO:teuthology.orchestra.run.smithi093.stdout: "num_rep": { 2023-11-16T16:54:57.348 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "num_rep", 2023-11-16T16:54:57.348 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.348 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.348 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.348 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3", 2023-11-16T16:54:57.348 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.349 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.349 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.349 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default replica count for any created local pool", 2023-11-16T16:54:57.349 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.349 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.349 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.349 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.349 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_num": { 2023-11-16T16:54:57.350 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_num", 2023-11-16T16:54:57.350 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.350 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.350 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.350 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "128", 2023-11-16T16:54:57.350 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.350 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.350 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.351 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default pg_num for any created local pool", 2023-11-16T16:54:57.351 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.351 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.351 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.351 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.351 INFO:teuthology.orchestra.run.smithi093.stdout: "prefix": { 2023-11-16T16:54:57.351 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prefix", 2023-11-16T16:54:57.351 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.352 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.352 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.352 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.352 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.352 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.352 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.352 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name prefix for any created local pool", 2023-11-16T16:54:57.352 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.353 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.353 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.353 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.353 INFO:teuthology.orchestra.run.smithi093.stdout: "subtree": { 2023-11-16T16:54:57.353 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "subtree", 2023-11-16T16:54:57.353 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.353 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.353 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.354 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "rack", 2023-11-16T16:54:57.354 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.354 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.354 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.354 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-16T16:54:57.354 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-16T16:54:57.354 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.354 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.355 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.355 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.355 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.355 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.355 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mds_autoscaler", 2023-11-16T16:54:57.355 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.355 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.355 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.356 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.356 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.356 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.356 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.356 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.356 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.356 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.357 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.357 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.357 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.357 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.357 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.357 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.357 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.357 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.358 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.358 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.358 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.358 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.358 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.358 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.358 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.358 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.359 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.359 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.359 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.359 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.359 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.359 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.359 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.359 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.360 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.360 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.360 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.360 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.360 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.360 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.360 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.361 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.361 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.361 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.361 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.361 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.361 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.361 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.361 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.362 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.362 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.362 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.362 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.362 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.362 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.362 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.362 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.363 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.363 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.363 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.363 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.363 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.363 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.363 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.363 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.364 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.364 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.364 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.364 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.364 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.364 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.364 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.364 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.365 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.365 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.365 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.365 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirroring", 2023-11-16T16:54:57.365 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.365 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.365 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.365 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.366 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.366 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.366 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.366 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.366 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.366 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.366 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.366 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.366 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.367 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.367 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.367 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.367 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.367 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.367 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.367 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.367 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.367 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.368 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.368 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.368 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.368 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.368 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.368 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.368 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.369 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.369 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.369 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.369 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.369 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.369 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.369 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.369 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.370 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.370 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.370 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.370 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.370 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.370 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.370 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.371 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.371 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.371 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.371 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.371 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.371 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.371 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.371 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.372 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.372 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.372 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.372 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.372 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.372 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.372 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.372 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.373 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.373 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.373 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.373 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.373 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.373 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.373 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.373 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.374 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.374 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.374 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.374 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.374 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.374 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.374 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.374 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.375 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "nfs", 2023-11-16T16:54:57.375 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.375 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.375 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.375 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.375 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.375 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.375 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.376 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.376 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.376 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.376 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.376 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.376 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.376 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.376 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.377 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.377 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.377 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.377 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.377 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.377 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.377 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.377 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.378 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.378 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.378 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.378 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.378 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.378 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.378 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.378 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.379 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.379 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.379 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.379 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.379 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.379 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.379 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.379 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.380 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.380 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.380 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.380 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.380 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.380 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.381 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.381 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.381 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.381 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.381 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.381 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.381 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.381 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.382 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.382 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.382 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.382 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.382 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.382 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.382 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.382 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.383 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.383 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.383 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.383 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.383 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.383 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.383 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.384 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.384 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.384 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.384 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.384 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.384 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.384 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.384 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.385 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-11-16T16:54:57.385 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.385 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.385 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.385 INFO:teuthology.orchestra.run.smithi093.stdout: "fail_fs": { 2023-11-16T16:54:57.385 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "fail_fs", 2023-11-16T16:54:57.385 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.385 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.386 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.386 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.386 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.386 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.386 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.386 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-16T16:54:57.386 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.386 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.387 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.387 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.387 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.387 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.387 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.387 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.387 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.387 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.388 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.388 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.388 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.388 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.388 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.388 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.388 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.388 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.388 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.389 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.389 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.389 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.389 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.389 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.389 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.389 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.390 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.390 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.390 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.390 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.390 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.390 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.390 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.390 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.391 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.391 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.391 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.391 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.391 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.391 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.391 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.391 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.392 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.392 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.392 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.392 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.392 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.392 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.392 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.392 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.393 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.393 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.393 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.393 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.393 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.393 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.393 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.393 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.394 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.394 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.394 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.394 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.394 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.394 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.394 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.394 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.395 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.395 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.395 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.395 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.395 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.395 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.395 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.395 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.396 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator": { 2023-11-16T16:54:57.396 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-11-16T16:54:57.396 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.396 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.396 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.396 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.396 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.396 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.397 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.397 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm", 2023-11-16T16:54:57.397 INFO:teuthology.orchestra.run.smithi093.stdout: "rook", 2023-11-16T16:54:57.397 INFO:teuthology.orchestra.run.smithi093.stdout: "test_orchestrator" 2023-11-16T16:54:57.397 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.397 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Orchestrator backend", 2023-11-16T16:54:57.397 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.397 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.398 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.398 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.398 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.398 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.398 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.398 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_perf_query", 2023-11-16T16:54:57.398 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.399 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.399 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.399 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.399 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.399 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.399 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.399 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.399 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.400 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.400 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.400 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.400 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.400 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.400 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.400 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.400 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.401 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.401 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.401 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.401 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.401 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.401 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.401 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.401 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.402 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.402 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.402 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.402 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.402 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.402 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.402 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.403 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.403 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.403 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.403 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.403 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.403 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.403 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.403 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.403 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.404 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.404 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.404 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.404 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.404 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.404 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.404 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.404 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.405 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.405 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.405 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.405 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.405 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.405 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.405 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.405 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.406 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.406 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.406 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.406 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.406 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.406 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.406 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.406 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.407 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.407 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.407 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.407 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.407 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.407 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.407 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.407 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.408 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.408 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.408 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.408 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_support", 2023-11-16T16:54:57.408 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.408 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.408 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.408 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.409 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.409 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.409 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.409 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.409 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.409 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.409 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.410 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.410 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.410 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.410 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.410 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.410 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.410 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.410 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.411 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.411 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.411 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.411 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.411 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.411 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.411 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.411 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.412 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.412 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.412 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.412 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.412 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.412 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.412 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.412 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.413 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.413 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.413 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.413 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.413 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.413 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.413 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.413 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.414 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.414 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.414 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.414 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.414 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.414 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.414 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.415 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.415 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.415 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.415 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.415 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.415 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.415 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.415 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.416 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.416 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.416 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.416 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.416 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.416 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.416 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.416 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.417 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.417 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.417 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.417 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.417 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.417 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.417 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.417 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.418 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.418 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.418 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_autoscaler", 2023-11-16T16:54:57.418 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.418 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.418 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.418 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.418 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.419 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.419 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.419 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.419 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.419 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.419 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.419 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.419 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.420 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.420 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.420 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.420 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.420 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.420 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.420 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.420 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.421 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.421 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.421 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.421 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.421 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.421 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.421 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.421 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.422 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.422 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.422 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.422 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.422 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.422 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.422 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.422 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.423 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.423 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.423 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.423 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.423 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.423 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.423 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.423 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.423 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.425 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.425 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.425 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.425 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.425 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.425 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.425 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.425 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.425 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.426 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.426 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.426 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.426 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.426 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.426 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.426 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.426 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.426 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.427 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.427 INFO:teuthology.orchestra.run.smithi093.stdout: "noautoscale": { 2023-11-16T16:54:57.427 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "noautoscale", 2023-11-16T16:54:57.427 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.427 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.427 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.427 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.427 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.427 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.428 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.428 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "global autoscale flag", 2023-11-16T16:54:57.428 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-16T16:54:57.428 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.428 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.428 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.428 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:54:57.428 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:54:57.429 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.429 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.429 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.429 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:54:57.429 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.429 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.429 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.429 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.430 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.430 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.430 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.430 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.430 INFO:teuthology.orchestra.run.smithi093.stdout: "threshold": { 2023-11-16T16:54:57.430 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threshold", 2023-11-16T16:54:57.430 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:54:57.430 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.431 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.431 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-11-16T16:54:57.431 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1.0", 2023-11-16T16:54:57.431 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.431 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.431 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "scaling threshold", 2023-11-16T16:54:57.431 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-16T16:54:57.431 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.431 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.432 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.432 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.432 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.432 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.432 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "progress", 2023-11-16T16:54:57.432 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.432 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.432 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.432 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_pg_recovery_event": { 2023-11-16T16:54:57.433 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_pg_recovery_event", 2023-11-16T16:54:57.433 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.433 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.433 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.433 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.433 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.433 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.433 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.433 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-16T16:54:57.434 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.434 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.434 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.434 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.434 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-11-16T16:54:57.434 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-11-16T16:54:57.434 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.434 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.434 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.435 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.435 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.435 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.435 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.435 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.435 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.435 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.435 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.435 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.436 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.436 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.436 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.436 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.436 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.436 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.436 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.436 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.437 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.437 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.437 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.437 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.437 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.437 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.437 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.437 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.437 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.438 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.438 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.438 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.438 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.438 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.438 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.438 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.438 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.439 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.439 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.439 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.439 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.439 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.439 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.439 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.439 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.440 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.440 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.440 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.440 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.440 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.440 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.440 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.440 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.440 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.441 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.441 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.441 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.441 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.441 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.441 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.441 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.441 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.442 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.442 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.442 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.442 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.442 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.442 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.442 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.442 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.443 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.443 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.443 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.443 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.443 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.443 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.443 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.443 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.444 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.444 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.444 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.444 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.444 INFO:teuthology.orchestra.run.smithi093.stdout: "max_completed_events": { 2023-11-16T16:54:57.444 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_completed_events", 2023-11-16T16:54:57.444 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.444 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.445 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.445 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "50", 2023-11-16T16:54:57.445 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.445 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.445 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.445 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "number of past completed events to remember", 2023-11-16T16:54:57.445 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.445 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.445 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.446 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.446 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:54:57.446 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:54:57.446 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.446 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.446 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.446 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-11-16T16:54:57.446 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.447 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.447 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.447 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long the module is going to sleep", 2023-11-16T16:54:57.447 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.447 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.447 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.447 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.447 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.448 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.448 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.448 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus", 2023-11-16T16:54:57.448 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.448 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.448 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.448 INFO:teuthology.orchestra.run.smithi093.stdout: "cache": { 2023-11-16T16:54:57.448 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cache", 2023-11-16T16:54:57.449 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.449 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.449 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.449 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.449 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.449 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.449 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.450 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.450 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.450 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.450 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.450 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.450 INFO:teuthology.orchestra.run.smithi093.stdout: "exclude_perf_counters": { 2023-11-16T16:54:57.450 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "exclude_perf_counters", 2023-11-16T16:54:57.450 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.451 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.451 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.451 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.451 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.451 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.451 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.451 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-16T16:54:57.452 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-16T16:54:57.452 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.452 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.452 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.452 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.452 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.452 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.452 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.453 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.453 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.453 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.453 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.453 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.453 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.453 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.453 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.454 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.454 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.454 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.454 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.454 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.454 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.454 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.454 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.455 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.455 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.455 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.455 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.455 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.455 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.455 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.455 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.456 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.456 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.456 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.456 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.456 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.456 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.456 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.456 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.457 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.457 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.457 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.457 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.457 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.457 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.457 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.457 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.458 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.458 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.458 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.458 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.458 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.458 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.458 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.458 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.459 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.459 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.459 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.459 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.459 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.459 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.459 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.459 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.460 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.460 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.460 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.460 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.460 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.460 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.460 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.461 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.461 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.461 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.461 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools": { 2023-11-16T16:54:57.461 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools", 2023-11-16T16:54:57.461 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.461 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.461 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.462 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.462 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.462 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.462 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.462 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.462 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.462 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.462 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.463 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.463 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-16T16:54:57.463 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-16T16:54:57.463 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.463 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.463 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.463 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300", 2023-11-16T16:54:57.463 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.464 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.464 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.464 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.464 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.464 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.464 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.464 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.464 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_interval": { 2023-11-16T16:54:57.465 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_interval", 2023-11-16T16:54:57.465 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:54:57.465 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.465 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.465 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15.0", 2023-11-16T16:54:57.465 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.465 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.465 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.466 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.466 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.466 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.466 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.466 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.466 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-11-16T16:54:57.466 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-11-16T16:54:57.466 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.466 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.467 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.467 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-11-16T16:54:57.467 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.467 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.467 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.467 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-16T16:54:57.467 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.467 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.468 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.468 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.468 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-11-16T16:54:57.468 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-11-16T16:54:57.468 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.468 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.468 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.468 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "9283", 2023-11-16T16:54:57.469 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.469 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.469 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.469 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-16T16:54:57.469 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.469 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.469 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.470 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.470 INFO:teuthology.orchestra.run.smithi093.stdout: "stale_cache_strategy": { 2023-11-16T16:54:57.470 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stale_cache_strategy", 2023-11-16T16:54:57.470 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.470 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.470 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.470 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "log", 2023-11-16T16:54:57.470 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.471 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.471 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.471 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.471 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.471 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.471 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.471 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.471 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-11-16T16:54:57.471 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-11-16T16:54:57.472 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.472 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.472 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.472 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "default", 2023-11-16T16:54:57.472 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.472 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.472 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.473 INFO:teuthology.orchestra.run.smithi093.stdout: "default", 2023-11-16T16:54:57.473 INFO:teuthology.orchestra.run.smithi093.stdout: "error" 2023-11-16T16:54:57.473 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.473 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.473 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.473 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.473 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.473 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.474 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-11-16T16:54:57.474 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-11-16T16:54:57.474 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.474 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.474 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.474 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-11-16T16:54:57.474 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-11-16T16:54:57.474 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-11-16T16:54:57.475 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.475 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.475 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.475 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.475 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.475 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.475 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.475 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.476 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.476 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_support", 2023-11-16T16:54:57.476 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.476 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.476 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.476 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.476 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.476 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.477 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.477 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.477 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.477 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.477 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.477 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.477 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.477 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.478 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.478 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.478 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.478 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.478 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.478 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.478 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.479 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.479 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.479 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.479 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.479 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.479 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.479 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.479 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.480 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.480 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.480 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.480 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.480 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.480 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.480 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.481 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.481 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.481 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.481 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.481 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.481 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.481 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.481 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.481 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.482 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.482 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.482 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.482 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.482 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.482 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.482 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.483 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.483 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.483 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.483 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.483 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.483 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.483 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.483 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.484 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.484 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.484 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.484 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.484 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.484 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.484 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.484 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.485 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.485 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.485 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.485 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.485 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.485 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_snap_create": { 2023-11-16T16:54:57.485 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_snap_create", 2023-11-16T16:54:57.485 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.486 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.486 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.486 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.486 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.486 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.486 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.486 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.487 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.487 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.487 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.487 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.487 INFO:teuthology.orchestra.run.smithi093.stdout: "mirror_snapshot_schedule": { 2023-11-16T16:54:57.487 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirror_snapshot_schedule", 2023-11-16T16:54:57.487 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.487 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.488 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.488 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.488 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.488 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.488 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.488 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.488 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.488 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.489 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.489 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.489 INFO:teuthology.orchestra.run.smithi093.stdout: "trash_purge_schedule": { 2023-11-16T16:54:57.489 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "trash_purge_schedule", 2023-11-16T16:54:57.489 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.489 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.489 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.489 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.490 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.490 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.490 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.490 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.490 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.490 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.490 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.490 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.491 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.491 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.491 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.491 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "restful", 2023-11-16T16:54:57.491 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.491 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.491 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.491 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_auth": { 2023-11-16T16:54:57.492 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_auth", 2023-11-16T16:54:57.492 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.492 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.492 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.492 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.492 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.492 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.493 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.493 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.493 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.493 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.493 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.493 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.493 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-11-16T16:54:57.493 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-11-16T16:54:57.493 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.494 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.494 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.494 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.494 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.494 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.494 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.494 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.495 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.495 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.495 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.495 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.495 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.495 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.495 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.495 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.496 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.496 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.496 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.496 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.496 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.496 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.496 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.496 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.497 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.497 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.497 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.497 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.497 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.497 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.497 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.497 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.498 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.498 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.498 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.498 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.498 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.498 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.498 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.499 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.499 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.499 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.499 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.499 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.499 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.499 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.499 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.500 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.500 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.500 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.500 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.500 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.501 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.501 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.501 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.501 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.501 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.501 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.501 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.501 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.502 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.502 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.502 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.502 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.502 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.502 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.502 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.503 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.503 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.503 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.503 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.503 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.503 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.503 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.503 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.504 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.504 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.504 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.504 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.504 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.504 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.504 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.504 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-11-16T16:54:57.505 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-11-16T16:54:57.505 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.505 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.505 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.505 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.505 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.505 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.505 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.505 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.506 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.506 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.506 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.506 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.506 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-11-16T16:54:57.506 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-11-16T16:54:57.506 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.507 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.507 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.507 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.507 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.507 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.507 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.507 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.507 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.508 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.508 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.508 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.508 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.508 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.508 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.508 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rgw", 2023-11-16T16:54:57.508 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.509 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.509 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.509 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.509 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.509 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.509 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.509 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.510 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.510 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.510 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.510 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.510 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.510 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.510 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.511 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.511 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.511 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.511 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.511 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.511 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.511 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.511 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.511 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.512 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.512 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.512 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.512 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.512 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.512 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.512 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.512 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.513 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.513 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.513 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.513 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.513 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.513 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.513 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.513 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.514 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.514 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.514 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.514 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.514 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.514 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.514 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.515 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.515 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.515 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.515 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.515 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.515 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.515 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.515 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.516 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.516 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.516 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.516 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.516 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.516 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.516 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.516 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.517 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.517 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.517 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.517 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.517 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.517 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.517 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.517 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.518 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.518 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.518 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.518 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.518 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.518 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rook", 2023-11-16T16:54:57.518 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.518 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.519 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.519 INFO:teuthology.orchestra.run.smithi093.stdout: "drive_group_interval": { 2023-11-16T16:54:57.519 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "drive_group_interval", 2023-11-16T16:54:57.519 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:54:57.519 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.519 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.519 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300.0", 2023-11-16T16:54:57.521 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.521 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.521 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.521 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-16T16:54:57.521 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.521 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.522 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.522 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.522 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.522 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.522 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.522 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.522 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.523 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.523 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.523 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.523 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.523 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.523 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.523 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.523 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.523 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.524 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.524 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.524 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.524 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.524 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.524 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.524 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.524 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.525 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.525 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.525 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.525 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.525 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.525 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.525 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.526 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.526 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.526 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.526 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.526 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.526 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.526 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.526 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.527 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.527 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.527 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.527 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.527 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.527 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.527 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.527 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.528 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.528 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.528 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.528 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.528 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.528 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.528 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.528 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.529 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.529 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.529 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.529 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.529 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.529 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.529 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.530 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.530 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.530 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.530 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.530 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.530 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.530 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.530 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.531 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.531 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.531 INFO:teuthology.orchestra.run.smithi093.stdout: "storage_class": { 2023-11-16T16:54:57.531 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "storage_class", 2023-11-16T16:54:57.531 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.531 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.531 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.532 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "local", 2023-11-16T16:54:57.532 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.532 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.532 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.532 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-16T16:54:57.532 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.532 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.532 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.532 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.533 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.533 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.533 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.533 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "selftest", 2023-11-16T16:54:57.533 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.533 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.533 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.533 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.534 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.534 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.534 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.534 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.534 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.534 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.534 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.534 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.535 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.535 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.535 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.535 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.535 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.535 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.535 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.535 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.536 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.536 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.536 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.536 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.536 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.536 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.536 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.536 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.536 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.537 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.537 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.537 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.537 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.537 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.537 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.537 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.537 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.538 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.538 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.538 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.538 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.538 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.538 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.538 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.538 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.539 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.539 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.539 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.539 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.539 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.539 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.539 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.539 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.540 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.540 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.540 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.540 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.540 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.540 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.540 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.540 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.541 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.541 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.541 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.541 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.541 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.541 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.541 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.541 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.541 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.542 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.542 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.542 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.542 INFO:teuthology.orchestra.run.smithi093.stdout: "roption1": { 2023-11-16T16:54:57.542 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption1", 2023-11-16T16:54:57.542 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.542 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.543 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.543 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.543 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.543 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.543 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.543 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.543 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.543 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.543 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.544 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.544 INFO:teuthology.orchestra.run.smithi093.stdout: "roption2": { 2023-11-16T16:54:57.544 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption2", 2023-11-16T16:54:57.544 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.544 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.544 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.544 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "xyz", 2023-11-16T16:54:57.544 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.545 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.545 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.545 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.545 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.545 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.545 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.545 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.545 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption1": { 2023-11-16T16:54:57.546 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption1", 2023-11-16T16:54:57.546 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.546 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.546 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.546 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.546 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.546 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.546 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.546 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.547 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.547 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.547 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.547 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.547 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption2": { 2023-11-16T16:54:57.547 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption2", 2023-11-16T16:54:57.547 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.547 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.548 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.548 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.548 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.548 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.548 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.548 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.548 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.548 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.549 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.549 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.549 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption3": { 2023-11-16T16:54:57.549 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption3", 2023-11-16T16:54:57.549 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:54:57.549 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.549 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.550 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.550 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.550 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.550 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.550 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.550 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.550 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.550 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.551 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.551 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption4": { 2023-11-16T16:54:57.551 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption4", 2023-11-16T16:54:57.551 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.551 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.551 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.551 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.552 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.552 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.552 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.552 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.552 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.552 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.552 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.552 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.553 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption5": { 2023-11-16T16:54:57.553 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption5", 2023-11-16T16:54:57.553 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.553 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.553 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.553 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.553 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.553 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.554 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.554 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.554 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.554 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.554 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.554 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.554 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption6": { 2023-11-16T16:54:57.554 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption6", 2023-11-16T16:54:57.555 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.555 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.555 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.555 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.555 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.555 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.555 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.555 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.555 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.556 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.556 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.556 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.556 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption7": { 2023-11-16T16:54:57.556 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption7", 2023-11-16T16:54:57.556 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.556 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.556 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.556 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.557 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-11-16T16:54:57.557 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "42", 2023-11-16T16:54:57.557 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.557 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.557 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.557 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.557 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.557 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.558 INFO:teuthology.orchestra.run.smithi093.stdout: "testkey": { 2023-11-16T16:54:57.558 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testkey", 2023-11-16T16:54:57.558 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.558 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.558 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.558 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.558 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.558 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.559 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.559 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.559 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.559 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.559 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.559 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.559 INFO:teuthology.orchestra.run.smithi093.stdout: "testlkey": { 2023-11-16T16:54:57.559 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testlkey", 2023-11-16T16:54:57.560 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.560 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.560 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.560 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.560 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.560 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.560 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.560 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.560 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.561 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.561 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.561 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.561 INFO:teuthology.orchestra.run.smithi093.stdout: "testnewline": { 2023-11-16T16:54:57.561 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testnewline", 2023-11-16T16:54:57.561 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.561 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.561 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.561 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.562 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.562 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.562 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.562 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.562 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.562 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.562 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.562 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.563 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.563 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.563 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.563 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-11-16T16:54:57.563 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.563 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.563 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.563 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_m_granularity": { 2023-11-16T16:54:57.563 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_m_granularity", 2023-11-16T16:54:57.564 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.564 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.564 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.564 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.564 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.564 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.564 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.564 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow minute scheduled snapshots", 2023-11-16T16:54:57.564 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.565 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.565 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.565 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.565 INFO:teuthology.orchestra.run.smithi093.stdout: "dump_on_update": { 2023-11-16T16:54:57.565 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dump_on_update", 2023-11-16T16:54:57.565 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.565 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.565 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.565 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.566 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.566 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.566 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.566 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "dump database to debug log on update", 2023-11-16T16:54:57.566 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.566 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.566 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.566 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.567 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.567 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.567 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.567 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.567 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.567 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.567 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.567 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.567 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.568 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.568 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.568 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.568 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.569 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.569 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.569 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.569 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.569 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.569 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.569 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.569 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.570 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.570 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.570 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.570 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.570 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.570 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.570 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.570 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.571 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.571 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.571 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.571 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.571 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.572 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.572 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.572 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.572 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.572 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.572 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.572 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.572 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.573 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.573 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.573 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.573 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.573 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.573 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.573 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.573 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.573 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.574 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.574 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.574 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.574 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.574 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.574 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.574 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.574 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.574 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.575 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.575 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.575 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.575 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.575 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.575 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.575 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.575 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.576 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.576 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.576 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.576 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.576 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.576 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stats", 2023-11-16T16:54:57.576 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.576 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.576 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.576 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.577 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.577 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.577 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.577 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.577 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.577 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.577 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.577 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.577 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.578 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.578 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.578 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.578 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.578 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.578 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.578 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.578 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.579 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.579 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.579 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.579 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.579 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.579 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.579 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.579 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.580 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.580 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.580 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.580 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.580 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.580 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.580 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.580 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.580 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.581 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.581 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.581 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.581 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.581 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.581 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.581 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.581 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.581 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.582 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.582 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.582 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.582 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.582 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.582 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.582 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.582 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.583 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.583 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.583 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.583 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.583 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.583 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.583 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.583 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.583 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.584 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.584 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.584 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.584 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.584 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.584 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.584 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.584 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.584 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.585 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.585 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.585 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.585 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "status", 2023-11-16T16:54:57.585 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.586 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.586 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.586 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.586 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.586 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.586 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.586 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.586 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.586 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.587 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.587 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.587 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.587 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.587 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.587 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.587 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.587 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.587 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.588 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.588 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.588 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.588 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.588 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.588 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.588 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.588 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.589 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.589 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.589 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.589 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.589 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.589 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.589 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.589 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.590 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.590 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.590 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.590 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.590 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.590 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.590 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.590 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.591 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.591 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.591 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.591 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.591 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.591 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.591 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.591 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.591 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.592 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.592 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.592 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.592 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.592 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.592 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.592 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.592 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.592 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.593 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.593 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.593 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.593 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.593 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.593 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.593 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.593 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.593 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.594 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.594 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.594 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.594 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.594 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.594 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.594 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telegraf", 2023-11-16T16:54:57.594 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.595 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.595 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.595 INFO:teuthology.orchestra.run.smithi093.stdout: "address": { 2023-11-16T16:54:57.595 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "address", 2023-11-16T16:54:57.595 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.595 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.595 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.595 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-16T16:54:57.596 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.596 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.596 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.596 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.596 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.596 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.596 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.596 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.596 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:54:57.597 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:54:57.597 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.597 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.597 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.597 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15", 2023-11-16T16:54:57.597 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.597 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.597 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.597 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.598 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.598 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.598 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.598 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.598 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.598 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.598 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.598 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.599 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.599 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.599 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.599 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.599 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.599 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.599 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.599 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.600 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.600 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.600 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.600 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.600 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.600 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.600 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.600 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.601 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.601 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.601 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.601 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.601 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.601 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.601 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.601 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.601 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.602 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.602 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.602 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.602 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.602 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.602 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.602 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.602 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.603 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.603 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.603 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.603 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.603 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.603 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.603 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.603 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.603 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.604 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.604 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.604 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.604 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.604 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.604 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.604 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.604 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.605 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.605 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.605 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.605 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.605 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.605 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.605 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.605 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.605 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.606 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.606 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.606 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.606 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.606 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.606 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.606 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.606 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.607 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.607 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.607 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telemetry", 2023-11-16T16:54:57.607 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.607 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.607 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.607 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_basic": { 2023-11-16T16:54:57.607 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_basic", 2023-11-16T16:54:57.607 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.608 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.608 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.608 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.608 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.608 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.608 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.608 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-16T16:54:57.608 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.608 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.609 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.609 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.609 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_crash": { 2023-11-16T16:54:57.609 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_crash", 2023-11-16T16:54:57.609 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.609 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.609 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.609 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.610 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.610 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.610 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.610 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-16T16:54:57.610 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.610 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.610 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.610 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.611 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_device": { 2023-11-16T16:54:57.611 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_device", 2023-11-16T16:54:57.611 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.611 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.611 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.611 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.611 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.611 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.611 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.612 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-16T16:54:57.612 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.612 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.612 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.612 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.612 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_ident": { 2023-11-16T16:54:57.612 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_ident", 2023-11-16T16:54:57.612 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.613 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.613 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.613 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.613 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.613 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.613 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.613 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-16T16:54:57.613 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.613 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.614 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.614 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.614 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_perf": { 2023-11-16T16:54:57.614 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_perf", 2023-11-16T16:54:57.614 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.614 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.614 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.614 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.614 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.615 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.615 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.615 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-16T16:54:57.615 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.615 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.615 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.615 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.615 INFO:teuthology.orchestra.run.smithi093.stdout: "contact": { 2023-11-16T16:54:57.615 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "contact", 2023-11-16T16:54:57.616 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.616 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.616 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.616 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.616 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.616 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.616 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.616 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.617 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.617 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.617 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.617 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.617 INFO:teuthology.orchestra.run.smithi093.stdout: "description": { 2023-11-16T16:54:57.617 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "description", 2023-11-16T16:54:57.617 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.617 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.617 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.618 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.618 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.618 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.618 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.618 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.618 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.618 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.618 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.619 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.619 INFO:teuthology.orchestra.run.smithi093.stdout: "device_url": { 2023-11-16T16:54:57.619 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_url", 2023-11-16T16:54:57.619 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.619 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.619 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.619 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-16T16:54:57.619 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.620 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.620 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.620 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.620 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.620 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.620 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.620 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.620 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-11-16T16:54:57.620 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-11-16T16:54:57.621 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.621 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.621 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.621 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.621 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.621 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.621 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.621 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.621 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.622 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.622 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.622 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.622 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:54:57.622 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:54:57.622 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.622 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.622 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.623 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "24", 2023-11-16T16:54:57.623 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "8", 2023-11-16T16:54:57.623 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.623 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.623 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.623 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.623 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.623 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.623 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.624 INFO:teuthology.orchestra.run.smithi093.stdout: "last_opt_revision": { 2023-11-16T16:54:57.624 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "last_opt_revision", 2023-11-16T16:54:57.624 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.624 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.624 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.624 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1", 2023-11-16T16:54:57.624 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.624 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.624 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.625 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.625 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.625 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.625 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.625 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.625 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard": { 2023-11-16T16:54:57.625 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard", 2023-11-16T16:54:57.625 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.625 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.626 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.626 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.626 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.626 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.626 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.626 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.626 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.626 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.627 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.627 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.627 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard_description": { 2023-11-16T16:54:57.627 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard_description", 2023-11-16T16:54:57.627 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.627 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.627 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.627 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.627 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.628 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.628 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.628 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.628 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.628 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.628 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.628 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.628 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.629 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.629 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.629 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.629 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.629 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.629 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.629 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.630 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.630 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.630 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.630 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.630 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.630 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.630 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.630 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.631 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.631 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.631 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.631 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.631 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.631 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.631 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.631 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.632 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.632 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.632 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.632 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.632 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.632 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.632 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.632 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.633 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.633 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.633 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.633 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.633 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.633 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.633 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.633 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.633 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.634 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.634 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.634 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.634 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.634 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.634 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.634 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.634 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.635 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.635 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.635 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.635 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.635 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.635 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.635 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.635 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.636 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.636 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.636 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.636 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.636 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.636 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.636 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.636 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.637 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.637 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.637 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.637 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.637 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.637 INFO:teuthology.orchestra.run.smithi093.stdout: "organization": { 2023-11-16T16:54:57.637 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "organization", 2023-11-16T16:54:57.637 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.638 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.638 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.638 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.638 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.638 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.638 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.638 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.638 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.639 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.639 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.639 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.639 INFO:teuthology.orchestra.run.smithi093.stdout: "proxy": { 2023-11-16T16:54:57.639 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "proxy", 2023-11-16T16:54:57.639 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.639 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.639 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.640 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.640 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.640 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.640 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.640 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.640 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.640 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.641 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.641 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.641 INFO:teuthology.orchestra.run.smithi093.stdout: "url": { 2023-11-16T16:54:57.641 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url", 2023-11-16T16:54:57.641 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.641 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.641 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.641 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-16T16:54:57.641 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.642 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.642 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.642 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.642 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.642 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.642 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.642 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.643 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.643 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.643 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.643 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "test_orchestrator", 2023-11-16T16:54:57.643 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.643 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.643 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.643 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.643 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.644 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.644 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.644 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.644 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.644 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.644 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.644 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.644 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.645 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.645 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.645 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.645 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.645 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.645 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.645 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.646 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.646 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.646 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.646 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.646 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.646 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.646 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.646 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.647 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.647 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.647 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.647 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.647 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.647 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.647 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.647 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.648 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.648 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.648 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.648 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.648 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.648 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.648 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.648 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.649 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.649 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.649 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.649 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.649 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.649 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.649 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.649 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.650 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.650 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.650 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.650 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.650 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.650 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.650 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.650 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.651 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.651 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.651 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.651 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.651 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.651 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.651 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.651 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.652 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.652 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.652 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.652 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.652 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.652 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.652 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.652 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.653 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "volumes", 2023-11-16T16:54:57.653 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.653 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.653 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.653 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.653 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.653 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.653 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.653 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.654 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.654 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.654 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.654 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.654 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.654 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.654 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.654 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.655 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.655 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.655 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.655 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.655 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.655 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.655 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.655 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.656 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.656 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.656 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.656 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.656 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.656 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.656 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.656 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.657 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.657 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.657 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.657 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.657 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.657 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.657 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.657 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.658 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.658 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.658 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.658 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.658 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.658 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.658 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.658 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.659 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.659 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.659 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.659 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.659 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.659 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.659 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.660 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.660 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.660 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.660 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.660 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.660 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.660 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.660 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.661 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.661 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.661 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.661 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.661 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.661 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.661 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.661 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.661 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.662 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.662 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_clones": { 2023-11-16T16:54:57.662 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_clones", 2023-11-16T16:54:57.662 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.662 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.662 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.662 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4", 2023-11-16T16:54:57.663 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.663 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.663 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.663 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-16T16:54:57.663 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.663 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.663 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.663 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.664 INFO:teuthology.orchestra.run.smithi093.stdout: "snapshot_clone_delay": { 2023-11-16T16:54:57.664 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snapshot_clone_delay", 2023-11-16T16:54:57.664 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.664 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.664 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.664 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:54:57.664 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.664 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.664 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.665 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-16T16:54:57.665 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.665 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.665 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.665 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.665 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.665 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.665 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.665 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix", 2023-11-16T16:54:57.666 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.666 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.666 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.666 INFO:teuthology.orchestra.run.smithi093.stdout: "discovery_interval": { 2023-11-16T16:54:57.666 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "discovery_interval", 2023-11-16T16:54:57.666 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:54:57.666 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.666 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.666 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "100", 2023-11-16T16:54:57.667 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.667 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.667 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.667 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.667 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.667 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.667 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.667 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.667 INFO:teuthology.orchestra.run.smithi093.stdout: "identifier": { 2023-11-16T16:54:57.668 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "identifier", 2023-11-16T16:54:57.668 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.668 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.668 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.668 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.668 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.668 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.668 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.669 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.669 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.669 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.669 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.669 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.669 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:54:57.669 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:54:57.669 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.670 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.670 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.670 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:54:57.670 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.670 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.670 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.670 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.670 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.671 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.671 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.671 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.671 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.671 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.671 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.671 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.671 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.671 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.672 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.672 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.672 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.672 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.672 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.672 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.672 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.672 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.673 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.673 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.673 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.673 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.673 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.673 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.673 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.673 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.673 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.674 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.674 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.674 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.674 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.674 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.674 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.674 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.674 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.675 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.675 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.675 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.675 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.675 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.675 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.675 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.675 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.675 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.675 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.676 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.676 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.676 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.676 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.676 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.676 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.676 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.676 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.677 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.677 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.677 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.677 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.677 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.677 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.677 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.677 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.677 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.678 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.678 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.678 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.678 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.678 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.678 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.678 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.678 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.679 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.679 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.679 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.679 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.679 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_host": { 2023-11-16T16:54:57.679 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_host", 2023-11-16T16:54:57.679 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.679 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.679 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.680 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.680 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.680 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.680 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.680 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.680 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.680 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.680 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.680 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.681 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_port": { 2023-11-16T16:54:57.681 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_port", 2023-11-16T16:54:57.681 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.681 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.681 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.681 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10051", 2023-11-16T16:54:57.681 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.681 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.681 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.682 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.682 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.682 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.682 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.682 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.682 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_sender": { 2023-11-16T16:54:57.682 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_sender", 2023-11-16T16:54:57.682 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.682 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.683 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.683 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-16T16:54:57.683 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.683 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.683 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.683 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.683 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.683 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.683 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.684 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.684 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.684 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.684 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-11-16T16:54:57.684 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.684 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.684 INFO:teuthology.orchestra.run.smithi093.stdout: "gid": 24115, 2023-11-16T16:54:57.684 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "y", 2023-11-16T16:54:57.685 INFO:teuthology.orchestra.run.smithi093.stdout: "mgr_features": 4540138322906710015, 2023-11-16T16:54:57.685 INFO:teuthology.orchestra.run.smithi093.stdout: "available_modules": [ 2023-11-16T16:54:57.685 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.685 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alerts", 2023-11-16T16:54:57.685 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.685 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.685 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.685 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:54:57.686 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:54:57.686 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.686 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.686 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.686 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:54:57.686 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.686 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.687 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.687 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How frequently to reexamine health status", 2023-11-16T16:54:57.687 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.687 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.687 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.687 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.687 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.687 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.688 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.688 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.688 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.688 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.688 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.688 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.688 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.689 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.689 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.689 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.689 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.689 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.689 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.689 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.689 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.690 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.690 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.690 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.690 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.690 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.690 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.690 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.690 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.691 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.691 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.691 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.691 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.691 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.691 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.691 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.691 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.692 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.692 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.692 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.692 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.692 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.692 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.692 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.692 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.693 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.693 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.693 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.693 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.693 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.693 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.693 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.693 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.694 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.694 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.694 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.694 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.694 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.694 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.694 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.694 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.695 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.695 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.695 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.695 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.695 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.695 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.695 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.695 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.696 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.696 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.696 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.696 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.696 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.696 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_destination": { 2023-11-16T16:54:57.696 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_destination", 2023-11-16T16:54:57.696 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.697 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.697 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.697 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.697 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.697 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.697 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.697 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email address to send alerts to", 2023-11-16T16:54:57.697 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.698 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.698 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.698 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.698 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_from_name": { 2023-11-16T16:54:57.698 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_from_name", 2023-11-16T16:54:57.698 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.698 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.698 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.699 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "Ceph", 2023-11-16T16:54:57.699 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.699 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.699 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.699 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email From: name", 2023-11-16T16:54:57.699 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.699 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.700 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.700 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.700 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_host": { 2023-11-16T16:54:57.700 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_host", 2023-11-16T16:54:57.700 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.700 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.700 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.700 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.701 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.701 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.701 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.701 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP server", 2023-11-16T16:54:57.701 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.701 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.701 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.701 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.702 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_password": { 2023-11-16T16:54:57.702 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_password", 2023-11-16T16:54:57.702 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.702 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.702 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.702 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.702 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.702 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.703 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.703 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Password to authenticate with", 2023-11-16T16:54:57.703 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.703 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.703 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.703 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.703 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_port": { 2023-11-16T16:54:57.703 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_port", 2023-11-16T16:54:57.704 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.704 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.704 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.704 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "465", 2023-11-16T16:54:57.704 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.704 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.704 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.704 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP port", 2023-11-16T16:54:57.705 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.705 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.705 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.705 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.705 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_sender": { 2023-11-16T16:54:57.705 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_sender", 2023-11-16T16:54:57.705 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.705 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.706 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.706 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.706 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.706 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.706 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.706 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP envelope sender", 2023-11-16T16:54:57.706 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.706 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.707 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.707 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.707 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_ssl": { 2023-11-16T16:54:57.707 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_ssl", 2023-11-16T16:54:57.707 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.707 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.707 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.707 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.708 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.708 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.708 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.708 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-16T16:54:57.708 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.708 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.708 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.708 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.709 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_user": { 2023-11-16T16:54:57.709 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_user", 2023-11-16T16:54:57.709 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.709 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.709 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.709 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.709 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.709 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.710 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.710 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "User to authenticate as", 2023-11-16T16:54:57.710 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.710 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.710 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.710 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.710 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.711 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.711 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.711 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "balancer", 2023-11-16T16:54:57.711 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.711 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.711 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.711 INFO:teuthology.orchestra.run.smithi093.stdout: "active": { 2023-11-16T16:54:57.711 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "active", 2023-11-16T16:54:57.712 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.712 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.712 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.712 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.712 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.712 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.712 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.712 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically balance PGs across cluster", 2023-11-16T16:54:57.713 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.713 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.713 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.713 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.713 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_time": { 2023-11-16T16:54:57.713 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_time", 2023-11-16T16:54:57.713 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.713 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.714 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.714 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0000", 2023-11-16T16:54:57.714 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.714 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.714 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.714 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "beginning time of day to automatically balance", 2023-11-16T16:54:57.714 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-16T16:54:57.714 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.715 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.715 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.715 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_weekday": { 2023-11-16T16:54:57.715 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_weekday", 2023-11-16T16:54:57.715 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:54:57.715 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.715 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.715 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:54:57.716 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-11-16T16:54:57.716 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-11-16T16:54:57.716 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.716 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-16T16:54:57.716 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-16T16:54:57.716 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.716 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.717 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.717 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_max_iterations": { 2023-11-16T16:54:57.717 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_max_iterations", 2023-11-16T16:54:57.717 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:54:57.717 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.717 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.717 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "25", 2023-11-16T16:54:57.717 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-11-16T16:54:57.718 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "250", 2023-11-16T16:54:57.718 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.718 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-16T16:54:57.718 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.718 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.718 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.718 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.718 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_metrics": { 2023-11-16T16:54:57.719 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_metrics", 2023-11-16T16:54:57.719 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.719 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.719 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.719 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "pgs,objects,bytes", 2023-11-16T16:54:57.719 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.719 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.720 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.720 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-16T16:54:57.720 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-16T16:54:57.720 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.720 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.720 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.720 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_step": { 2023-11-16T16:54:57.720 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_step", 2023-11-16T16:54:57.721 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:54:57.721 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.721 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.721 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.5", 2023-11-16T16:54:57.721 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0.001", 2023-11-16T16:54:57.721 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "0.999", 2023-11-16T16:54:57.721 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.721 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "aggressiveness of optimization", 2023-11-16T16:54:57.722 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-16T16:54:57.722 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.722 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.722 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.722 INFO:teuthology.orchestra.run.smithi093.stdout: "end_time": { 2023-11-16T16:54:57.722 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_time", 2023-11-16T16:54:57.722 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.722 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.723 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.723 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2359", 2023-11-16T16:54:57.723 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.723 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.723 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.723 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ending time of day to automatically balance", 2023-11-16T16:54:57.723 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-16T16:54:57.725 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.725 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.725 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.725 INFO:teuthology.orchestra.run.smithi093.stdout: "end_weekday": { 2023-11-16T16:54:57.725 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_weekday", 2023-11-16T16:54:57.725 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:54:57.725 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.726 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.726 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:54:57.726 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-11-16T16:54:57.726 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-11-16T16:54:57.726 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.726 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-16T16:54:57.726 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-16T16:54:57.726 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.727 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.727 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.727 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.727 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.727 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.727 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.727 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.727 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.728 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.728 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.728 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.728 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.728 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.728 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.728 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.728 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.729 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.729 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.729 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.729 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.729 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.729 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.730 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.730 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.730 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.730 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.730 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.730 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.730 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.730 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.731 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.731 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.731 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.731 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.731 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.731 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.731 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.731 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.731 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.732 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.732 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.732 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.732 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.732 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.732 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.732 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.732 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.733 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.733 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.733 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.733 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.733 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.733 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.733 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.733 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.733 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.734 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.734 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.734 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.734 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.734 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.734 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.734 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.734 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.735 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.735 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.735 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.735 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.735 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.735 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.735 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.735 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.736 INFO:teuthology.orchestra.run.smithi093.stdout: "min_score": { 2023-11-16T16:54:57.736 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_score", 2023-11-16T16:54:57.736 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:54:57.736 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.736 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.736 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:54:57.736 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.736 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.737 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.737 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-16T16:54:57.737 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.737 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.737 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.737 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.737 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-11-16T16:54:57.737 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-11-16T16:54:57.738 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.738 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.738 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.738 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "upmap", 2023-11-16T16:54:57.738 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.738 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.738 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.738 INFO:teuthology.orchestra.run.smithi093.stdout: "crush-compat", 2023-11-16T16:54:57.739 INFO:teuthology.orchestra.run.smithi093.stdout: "none", 2023-11-16T16:54:57.739 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap" 2023-11-16T16:54:57.739 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.739 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Balancer mode", 2023-11-16T16:54:57.739 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.739 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.739 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.740 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.740 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_ids": { 2023-11-16T16:54:57.740 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_ids", 2023-11-16T16:54:57.740 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.740 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.740 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.740 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.740 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.741 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.741 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.741 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-16T16:54:57.741 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.741 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.741 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.741 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.741 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:54:57.741 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:54:57.742 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.742 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.742 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.742 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:54:57.742 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.742 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.742 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.743 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-16T16:54:57.743 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.743 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.743 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.743 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.743 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_deviation": { 2023-11-16T16:54:57.743 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_deviation", 2023-11-16T16:54:57.743 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.744 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.744 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.744 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-11-16T16:54:57.744 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-11-16T16:54:57.744 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.744 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.744 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-16T16:54:57.744 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-16T16:54:57.745 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.745 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.745 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.745 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_optimizations": { 2023-11-16T16:54:57.745 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_optimizations", 2023-11-16T16:54:57.745 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:54:57.745 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.745 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.746 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.746 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.746 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.746 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.746 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-16T16:54:57.746 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.746 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.746 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.747 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.747 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.747 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.747 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.747 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cephadm", 2023-11-16T16:54:57.747 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.747 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.747 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.747 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_down_multiplier": { 2023-11-16T16:54:57.748 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_down_multiplier", 2023-11-16T16:54:57.748 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:54:57.748 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.748 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.748 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-11-16T16:54:57.748 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.748 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.748 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.749 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-16T16:54:57.749 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.749 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.749 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.749 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.749 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_refresh_rate": { 2023-11-16T16:54:57.749 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_refresh_rate", 2023-11-16T16:54:57.749 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.749 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.750 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.750 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "20", 2023-11-16T16:54:57.750 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.750 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.750 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.750 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-16T16:54:57.750 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.750 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.751 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.751 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.751 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_starting_port": { 2023-11-16T16:54:57.751 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_starting_port", 2023-11-16T16:54:57.751 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.751 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.751 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.751 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4721", 2023-11-16T16:54:57.752 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.752 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.752 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.752 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-16T16:54:57.752 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.752 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.752 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.753 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.753 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_password": { 2023-11-16T16:54:57.753 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_password", 2023-11-16T16:54:57.753 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.753 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.753 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.753 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.753 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.754 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.754 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.754 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web password", 2023-11-16T16:54:57.754 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.754 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.754 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.754 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.754 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_user": { 2023-11-16T16:54:57.755 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_user", 2023-11-16T16:54:57.755 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.755 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.755 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.755 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.755 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.755 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.755 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.756 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web user", 2023-11-16T16:54:57.756 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.756 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.756 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.756 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.756 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_ptrace": { 2023-11-16T16:54:57.756 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_ptrace", 2023-11-16T16:54:57.756 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.757 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.757 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.757 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.757 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.757 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.757 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.757 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-16T16:54:57.757 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-16T16:54:57.757 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.758 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.758 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.758 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_interval": { 2023-11-16T16:54:57.758 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_interval", 2023-11-16T16:54:57.758 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.758 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.758 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.759 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:54:57.759 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.759 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.759 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.759 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-16T16:54:57.759 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.759 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.759 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.760 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.760 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_memory_target_ratio": { 2023-11-16T16:54:57.760 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_memory_target_ratio", 2023-11-16T16:54:57.760 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:54:57.760 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.760 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.760 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.7", 2023-11-16T16:54:57.761 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.761 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.761 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.761 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-16T16:54:57.761 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.761 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.761 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.761 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.762 INFO:teuthology.orchestra.run.smithi093.stdout: "cgroups_split": { 2023-11-16T16:54:57.762 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cgroups_split", 2023-11-16T16:54:57.762 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.762 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.762 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.762 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.762 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.762 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.763 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.763 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-16T16:54:57.763 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.763 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.763 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.763 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.763 INFO:teuthology.orchestra.run.smithi093.stdout: "config_checks_enabled": { 2023-11-16T16:54:57.763 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_checks_enabled", 2023-11-16T16:54:57.764 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.764 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.764 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.764 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.764 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.764 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.764 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.764 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-16T16:54:57.765 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.765 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.765 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.765 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.765 INFO:teuthology.orchestra.run.smithi093.stdout: "config_dashboard": { 2023-11-16T16:54:57.765 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_dashboard", 2023-11-16T16:54:57.765 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.765 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.766 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.766 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.766 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.766 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.766 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.766 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-16T16:54:57.766 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.766 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.767 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.767 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.767 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_alertmanager": { 2023-11-16T16:54:57.767 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_alertmanager", 2023-11-16T16:54:57.767 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.767 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.767 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.767 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-16T16:54:57.768 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.768 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.768 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.768 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-11-16T16:54:57.768 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.768 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.768 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.768 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.769 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_base": { 2023-11-16T16:54:57.769 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_base", 2023-11-16T16:54:57.769 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.769 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.769 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.769 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-16T16:54:57.770 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.770 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.770 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.770 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Container image name, without the tag", 2023-11-16T16:54:57.770 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.770 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.770 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.770 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.771 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_elasticsearch": { 2023-11-16T16:54:57.771 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_elasticsearch", 2023-11-16T16:54:57.771 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.771 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.771 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.771 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-16T16:54:57.771 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.771 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.772 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.772 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "elasticsearch container image", 2023-11-16T16:54:57.772 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.772 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.772 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.772 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.772 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_grafana": { 2023-11-16T16:54:57.773 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_grafana", 2023-11-16T16:54:57.773 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.773 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.773 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.773 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-16T16:54:57.773 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.773 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.773 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.774 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-11-16T16:54:57.774 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.774 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.774 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.774 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.774 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_haproxy": { 2023-11-16T16:54:57.774 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_haproxy", 2023-11-16T16:54:57.774 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.775 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.775 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.775 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-16T16:54:57.775 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.775 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.775 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.775 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "HAproxy container image", 2023-11-16T16:54:57.775 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.776 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.776 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.776 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.776 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_agent": { 2023-11-16T16:54:57.776 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_agent", 2023-11-16T16:54:57.776 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.776 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.777 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.777 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-16T16:54:57.777 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.777 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.777 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.777 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger agent container image", 2023-11-16T16:54:57.777 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.777 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.778 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.778 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.778 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_collector": { 2023-11-16T16:54:57.778 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_collector", 2023-11-16T16:54:57.778 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.778 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.778 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.778 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-16T16:54:57.779 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.779 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.779 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.779 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger collector container image", 2023-11-16T16:54:57.779 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.779 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.779 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.780 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.780 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_query": { 2023-11-16T16:54:57.780 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_query", 2023-11-16T16:54:57.780 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.780 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.780 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.780 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-16T16:54:57.780 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.781 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.781 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.781 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger query container image", 2023-11-16T16:54:57.781 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.781 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.781 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.781 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.781 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_keepalived": { 2023-11-16T16:54:57.782 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_keepalived", 2023-11-16T16:54:57.782 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.782 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.782 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.782 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-16T16:54:57.782 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.782 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.783 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.783 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Keepalived container image", 2023-11-16T16:54:57.783 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.783 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.783 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.783 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.783 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_loki": { 2023-11-16T16:54:57.783 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_loki", 2023-11-16T16:54:57.783 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.784 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.784 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.784 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-16T16:54:57.784 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.784 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.784 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.784 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Loki container image", 2023-11-16T16:54:57.784 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.785 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.785 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.785 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.785 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_node_exporter": { 2023-11-16T16:54:57.785 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_node_exporter", 2023-11-16T16:54:57.785 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.785 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.785 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.786 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-16T16:54:57.786 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.786 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.786 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.786 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-11-16T16:54:57.786 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.786 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.786 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.787 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.787 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_prometheus": { 2023-11-16T16:54:57.787 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_prometheus", 2023-11-16T16:54:57.787 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.787 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.787 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.787 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-16T16:54:57.787 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.788 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.788 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.788 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-11-16T16:54:57.788 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.788 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.788 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.788 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.788 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_promtail": { 2023-11-16T16:54:57.789 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_promtail", 2023-11-16T16:54:57.789 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.789 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.789 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.789 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-16T16:54:57.789 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.789 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.790 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.790 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Promtail container image", 2023-11-16T16:54:57.790 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.790 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.790 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.790 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.790 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_snmp_gateway": { 2023-11-16T16:54:57.791 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_snmp_gateway", 2023-11-16T16:54:57.791 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.791 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.791 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.791 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-16T16:54:57.791 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.791 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.791 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.791 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SNMP Gateway container image", 2023-11-16T16:54:57.792 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.792 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.792 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.792 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.792 INFO:teuthology.orchestra.run.smithi093.stdout: "container_init": { 2023-11-16T16:54:57.792 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_init", 2023-11-16T16:54:57.792 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.792 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.793 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.793 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.793 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.793 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.793 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.793 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Run podman/docker with `--init`", 2023-11-16T16:54:57.793 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.793 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.794 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.794 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.794 INFO:teuthology.orchestra.run.smithi093.stdout: "daemon_cache_timeout": { 2023-11-16T16:54:57.794 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "daemon_cache_timeout", 2023-11-16T16:54:57.794 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.794 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.794 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.794 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:54:57.795 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.795 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.795 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.795 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-16T16:54:57.795 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.795 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.795 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.795 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.796 INFO:teuthology.orchestra.run.smithi093.stdout: "default_cephadm_command_timeout": { 2023-11-16T16:54:57.796 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_cephadm_command_timeout", 2023-11-16T16:54:57.796 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.796 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.796 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.796 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "900", 2023-11-16T16:54:57.796 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.796 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.797 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.797 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-16T16:54:57.797 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.797 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.797 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.797 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.797 INFO:teuthology.orchestra.run.smithi093.stdout: "default_registry": { 2023-11-16T16:54:57.797 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_registry", 2023-11-16T16:54:57.798 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.798 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.798 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.798 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io", 2023-11-16T16:54:57.798 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.798 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.798 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.798 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-16T16:54:57.799 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.799 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.799 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.799 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.799 INFO:teuthology.orchestra.run.smithi093.stdout: "device_cache_timeout": { 2023-11-16T16:54:57.799 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_cache_timeout", 2023-11-16T16:54:57.799 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.800 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.800 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.800 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1800", 2023-11-16T16:54:57.800 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.800 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.800 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.800 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache device inventory", 2023-11-16T16:54:57.800 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.801 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.801 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.801 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.801 INFO:teuthology.orchestra.run.smithi093.stdout: "device_enhanced_scan": { 2023-11-16T16:54:57.801 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_enhanced_scan", 2023-11-16T16:54:57.801 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.801 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.801 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.802 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.802 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.802 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.802 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.802 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-16T16:54:57.802 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.802 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.802 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.803 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.803 INFO:teuthology.orchestra.run.smithi093.stdout: "facts_cache_timeout": { 2023-11-16T16:54:57.803 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "facts_cache_timeout", 2023-11-16T16:54:57.803 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.803 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.803 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.803 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:54:57.804 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.804 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.804 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.804 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache host facts data", 2023-11-16T16:54:57.804 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.804 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.804 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.804 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.805 INFO:teuthology.orchestra.run.smithi093.stdout: "host_check_interval": { 2023-11-16T16:54:57.805 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "host_check_interval", 2023-11-16T16:54:57.805 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.805 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.805 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.805 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:54:57.805 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.805 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.806 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.806 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to perform a host check", 2023-11-16T16:54:57.806 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.806 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.806 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.806 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.806 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.807 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.807 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.807 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.807 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.807 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.807 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.807 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.807 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.808 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.808 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.808 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.808 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.808 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.808 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.808 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.808 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.809 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.809 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.809 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.809 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.809 INFO:teuthology.orchestra.run.smithi093.stdout: "log_refresh_metadata": { 2023-11-16T16:54:57.809 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_refresh_metadata", 2023-11-16T16:54:57.809 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.810 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.810 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.810 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.810 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.810 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.810 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.810 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-16T16:54:57.810 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.811 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.811 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.811 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.811 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.811 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.811 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.811 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.811 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.812 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.812 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.812 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.812 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.812 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-16T16:54:57.812 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.812 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.812 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.813 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.813 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.813 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.813 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.813 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.813 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.813 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.813 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.813 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.814 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.814 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.814 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.814 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.814 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.814 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.814 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.815 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.815 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.815 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.815 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.815 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.815 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.815 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.815 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.816 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.816 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.816 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.816 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.816 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.816 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.816 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.816 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.817 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.817 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.817 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.817 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.817 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-16T16:54:57.817 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-16T16:54:57.817 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.817 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.818 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.818 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.818 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.818 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.818 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.818 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-16T16:54:57.818 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.818 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.819 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.819 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.819 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-16T16:54:57.819 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-16T16:54:57.819 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.819 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.819 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.820 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "*", 2023-11-16T16:54:57.820 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.820 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.820 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.820 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-16T16:54:57.820 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.820 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.820 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.821 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.821 INFO:teuthology.orchestra.run.smithi093.stdout: "max_count_per_host": { 2023-11-16T16:54:57.821 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_count_per_host", 2023-11-16T16:54:57.821 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.821 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.821 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.821 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.821 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.822 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.822 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.822 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of daemons per service per host", 2023-11-16T16:54:57.822 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.822 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.822 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.822 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.822 INFO:teuthology.orchestra.run.smithi093.stdout: "max_osd_draining_count": { 2023-11-16T16:54:57.823 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_osd_draining_count", 2023-11-16T16:54:57.823 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.823 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.823 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.823 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.823 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.823 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.823 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.823 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-16T16:54:57.824 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.824 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.824 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.824 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.824 INFO:teuthology.orchestra.run.smithi093.stdout: "migration_current": { 2023-11-16T16:54:57.824 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "migration_current", 2023-11-16T16:54:57.824 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.824 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.825 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.825 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.825 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.825 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.825 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.825 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "internal - do not modify", 2023-11-16T16:54:57.825 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.825 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.826 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.826 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.826 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-11-16T16:54:57.826 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-11-16T16:54:57.826 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.826 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.826 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.827 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "root", 2023-11-16T16:54:57.827 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.827 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.827 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.827 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm-package", 2023-11-16T16:54:57.827 INFO:teuthology.orchestra.run.smithi093.stdout: "root" 2023-11-16T16:54:57.827 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.827 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "mode for remote execution of cephadm", 2023-11-16T16:54:57.828 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.828 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.828 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.828 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.828 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_alerts_path": { 2023-11-16T16:54:57.828 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_alerts_path", 2023-11-16T16:54:57.828 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.828 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.829 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.829 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-16T16:54:57.829 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.829 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.829 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.829 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-16T16:54:57.829 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.829 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.830 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.830 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.830 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_password": { 2023-11-16T16:54:57.830 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_password", 2023-11-16T16:54:57.830 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.830 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.830 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.830 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.831 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.831 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.831 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.831 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web password", 2023-11-16T16:54:57.831 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.831 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.831 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.831 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.832 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_user": { 2023-11-16T16:54:57.832 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_user", 2023-11-16T16:54:57.832 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.832 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.832 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.832 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.832 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.833 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.833 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.833 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web user", 2023-11-16T16:54:57.833 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.833 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.833 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.833 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.833 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_insecure": { 2023-11-16T16:54:57.834 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_insecure", 2023-11-16T16:54:57.834 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.834 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.834 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.834 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.834 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.834 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.834 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.835 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-16T16:54:57.835 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.835 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.835 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.835 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.835 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_password": { 2023-11-16T16:54:57.835 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_password", 2023-11-16T16:54:57.835 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.836 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.836 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.836 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.836 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.836 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.836 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.836 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-16T16:54:57.836 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.837 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.837 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.837 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.837 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_url": { 2023-11-16T16:54:57.837 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_url", 2023-11-16T16:54:57.837 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.837 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.837 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.838 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.838 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.838 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.838 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.838 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-16T16:54:57.838 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.838 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.838 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.838 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.839 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_username": { 2023-11-16T16:54:57.839 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_username", 2023-11-16T16:54:57.839 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.839 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.839 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.839 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.839 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.840 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.840 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.840 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-16T16:54:57.840 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.840 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.840 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.840 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.840 INFO:teuthology.orchestra.run.smithi093.stdout: "secure_monitoring_stack": { 2023-11-16T16:54:57.841 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "secure_monitoring_stack", 2023-11-16T16:54:57.841 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.841 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.841 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.841 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.841 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.841 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.841 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.842 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-16T16:54:57.842 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.842 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.842 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.842 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.842 INFO:teuthology.orchestra.run.smithi093.stdout: "service_discovery_port": { 2023-11-16T16:54:57.842 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "service_discovery_port", 2023-11-16T16:54:57.843 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.843 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.843 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.843 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8765", 2023-11-16T16:54:57.843 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.843 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.843 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.843 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "cephadm service discovery port", 2023-11-16T16:54:57.844 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.844 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.844 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.844 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.844 INFO:teuthology.orchestra.run.smithi093.stdout: "ssh_config_file": { 2023-11-16T16:54:57.844 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssh_config_file", 2023-11-16T16:54:57.844 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.844 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.845 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.845 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.845 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.845 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.845 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.845 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-16T16:54:57.845 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.845 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.846 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.846 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.846 INFO:teuthology.orchestra.run.smithi093.stdout: "use_agent": { 2023-11-16T16:54:57.846 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_agent", 2023-11-16T16:54:57.846 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.846 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.846 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.846 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.847 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.847 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.847 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.847 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-16T16:54:57.847 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.847 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.847 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.847 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.848 INFO:teuthology.orchestra.run.smithi093.stdout: "use_repo_digest": { 2023-11-16T16:54:57.848 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_repo_digest", 2023-11-16T16:54:57.848 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.848 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.848 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.848 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.848 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.849 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.849 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.849 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-16T16:54:57.849 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.849 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.849 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.849 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.849 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_failed_host_check": { 2023-11-16T16:54:57.850 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_failed_host_check", 2023-11-16T16:54:57.850 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.850 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.850 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.850 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.850 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.850 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.851 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.851 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if the host check fails", 2023-11-16T16:54:57.851 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.851 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.851 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.851 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.851 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_daemons": { 2023-11-16T16:54:57.851 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_daemons", 2023-11-16T16:54:57.852 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.852 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.852 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.852 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.852 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.852 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.852 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.853 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-16T16:54:57.853 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.853 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.853 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.853 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.853 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_hosts": { 2023-11-16T16:54:57.853 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_hosts", 2023-11-16T16:54:57.853 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.854 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.854 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.854 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.854 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.854 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.854 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.854 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-16T16:54:57.854 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.855 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.855 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.855 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.855 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.855 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.855 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.855 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crash", 2023-11-16T16:54:57.856 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.856 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.856 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.856 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.856 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.856 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.856 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.856 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.857 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.857 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.857 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.857 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.857 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.857 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.857 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.857 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.858 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.858 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.858 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.858 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.858 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.858 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.858 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.858 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.859 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.859 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.859 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.859 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.859 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.859 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.859 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.860 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.860 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.860 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.860 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.860 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.860 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.860 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.860 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.861 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.861 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.861 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.861 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.861 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.861 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.861 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.861 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.861 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.862 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.862 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.862 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.862 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.862 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.862 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.862 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.862 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.863 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.863 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.863 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.863 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.863 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.863 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.863 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.863 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.864 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.864 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.864 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.864 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.864 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.864 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.864 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.864 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.865 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.865 INFO:teuthology.orchestra.run.smithi093.stdout: "retain_interval": { 2023-11-16T16:54:57.865 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retain_interval", 2023-11-16T16:54:57.865 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.865 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.865 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.865 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "31536000", 2023-11-16T16:54:57.865 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.866 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.866 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.866 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-16T16:54:57.866 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.866 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.866 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.866 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.867 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_recent_interval": { 2023-11-16T16:54:57.867 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_recent_interval", 2023-11-16T16:54:57.867 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.867 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.867 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.867 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1209600", 2023-11-16T16:54:57.867 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.867 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.868 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.868 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-16T16:54:57.868 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.868 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.868 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.868 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.868 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.868 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.869 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.869 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dashboard", 2023-11-16T16:54:57.869 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.869 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.869 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.869 INFO:teuthology.orchestra.run.smithi093.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-16T16:54:57.869 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-16T16:54:57.869 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.870 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.870 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.870 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.870 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.870 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.870 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.870 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.870 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.871 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.871 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.871 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.871 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_HOST": { 2023-11-16T16:54:57.871 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-16T16:54:57.871 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.871 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.872 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.872 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.872 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.872 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.872 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.872 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.873 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.873 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.873 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.873 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.873 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-16T16:54:57.873 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-16T16:54:57.873 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.874 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.874 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.874 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.874 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.874 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.874 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.874 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.874 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.875 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.875 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.875 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.875 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_ENABLED": { 2023-11-16T16:54:57.875 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_ENABLED", 2023-11-16T16:54:57.875 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.876 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.876 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.876 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.876 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.876 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.876 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.876 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.877 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.877 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.877 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.877 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.877 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-16T16:54:57.877 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-16T16:54:57.877 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.877 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.878 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.878 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.878 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.878 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.878 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.878 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.879 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.879 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.879 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.879 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.879 INFO:teuthology.orchestra.run.smithi093.stdout: "ENABLE_BROWSABLE_API": { 2023-11-16T16:54:57.879 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-16T16:54:57.879 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.880 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.880 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.880 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.880 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.880 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.880 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.880 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.880 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.881 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.881 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.881 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.881 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-16T16:54:57.881 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-16T16:54:57.881 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.881 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.881 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.882 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.882 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.882 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.882 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.882 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.882 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.882 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.883 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.883 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.883 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-16T16:54:57.883 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-16T16:54:57.883 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.883 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.883 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.883 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.884 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.884 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.884 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.884 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.884 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.884 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.884 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.884 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.885 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-16T16:54:57.885 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-16T16:54:57.885 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.885 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.885 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.885 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.885 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.885 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.885 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.886 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.886 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.886 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.886 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.886 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.886 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-16T16:54:57.886 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-16T16:54:57.887 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.887 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.887 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.887 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.887 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.887 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.887 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.887 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.888 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.888 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.888 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.888 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.888 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-16T16:54:57.888 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-16T16:54:57.888 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.888 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.889 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.889 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.889 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.889 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.889 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.889 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.889 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.889 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.890 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.890 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.890 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-16T16:54:57.890 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-16T16:54:57.890 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.890 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.890 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.891 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.891 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.891 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.891 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.891 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.891 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.891 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.891 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.892 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.892 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-16T16:54:57.892 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-16T16:54:57.892 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.892 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.892 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.892 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.892 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.892 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.893 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.893 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.893 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.893 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.893 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.893 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.893 INFO:teuthology.orchestra.run.smithi093.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-16T16:54:57.893 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-16T16:54:57.894 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.894 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.894 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.894 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.894 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.894 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.894 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.894 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.895 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.895 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.895 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.895 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.895 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_PASSWORD": { 2023-11-16T16:54:57.895 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-16T16:54:57.895 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.895 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.896 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.896 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.896 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.896 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.896 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.896 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.896 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.897 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.897 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.897 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.897 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-16T16:54:57.897 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-16T16:54:57.897 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.897 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.897 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.897 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.898 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.898 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.898 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.898 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.898 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.898 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.898 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.899 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.899 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_URL": { 2023-11-16T16:54:57.899 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_URL", 2023-11-16T16:54:57.899 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.899 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.899 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.899 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.900 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.900 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.900 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.900 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.900 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.900 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.900 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.900 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.901 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_USERNAME": { 2023-11-16T16:54:57.901 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-16T16:54:57.901 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.901 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.901 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.901 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.901 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.901 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.902 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.902 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.902 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.902 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.902 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.902 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.902 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-16T16:54:57.903 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-16T16:54:57.903 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.903 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.903 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.903 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.903 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.903 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.903 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.904 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.904 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.904 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.904 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.904 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.904 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-16T16:54:57.904 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-16T16:54:57.904 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.905 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.905 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.905 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.905 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.905 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.905 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.905 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.905 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.906 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.906 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.906 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.906 INFO:teuthology.orchestra.run.smithi093.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-16T16:54:57.906 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-16T16:54:57.906 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.906 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.906 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.907 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.907 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.907 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.907 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.907 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.907 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.907 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.907 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.908 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.908 INFO:teuthology.orchestra.run.smithi093.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-16T16:54:57.908 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-16T16:54:57.908 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.908 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.908 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.908 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.908 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.909 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.909 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.909 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.909 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.909 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.909 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.909 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.910 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_HOST": { 2023-11-16T16:54:57.910 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-16T16:54:57.910 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.910 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.910 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.910 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.910 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.910 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.911 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.911 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.911 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.911 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.911 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.911 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.911 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-16T16:54:57.911 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-16T16:54:57.912 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.912 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.912 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.912 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.912 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.912 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.912 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.913 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.913 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.913 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.913 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.913 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.913 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-16T16:54:57.913 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-16T16:54:57.913 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.914 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.914 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.914 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.914 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.914 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.914 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.914 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.914 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.915 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.915 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.915 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.915 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-16T16:54:57.915 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-16T16:54:57.915 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.915 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.915 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.916 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.916 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.916 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.916 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.916 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.916 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.916 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.916 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.917 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.917 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-16T16:54:57.917 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-16T16:54:57.917 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.917 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.917 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.917 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.917 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.918 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.918 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.918 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.918 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.918 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.918 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.918 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.918 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-16T16:54:57.918 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-16T16:54:57.919 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.919 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.919 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.919 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.919 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.919 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.919 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.920 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.920 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.920 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.920 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.920 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.920 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-16T16:54:57.920 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-16T16:54:57.920 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.920 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.921 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.921 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.921 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.921 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.921 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.922 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.922 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.922 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.922 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.923 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.923 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-16T16:54:57.923 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-16T16:54:57.923 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.923 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.924 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.924 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.924 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.924 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.924 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.924 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.924 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.924 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.925 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.925 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.925 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-16T16:54:57.925 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-16T16:54:57.925 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.925 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.926 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.926 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.926 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.926 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.926 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.927 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.927 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.927 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.927 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.927 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.927 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_ENABLED": { 2023-11-16T16:54:57.927 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-16T16:54:57.927 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.929 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.929 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.929 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.930 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.930 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.930 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.930 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.930 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.930 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.930 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.931 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.931 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-16T16:54:57.931 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-16T16:54:57.931 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.931 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.931 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.931 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-16T16:54:57.931 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.932 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.932 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.932 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.932 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.932 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.932 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.932 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.932 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-16T16:54:57.933 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-16T16:54:57.933 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.933 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.933 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.933 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.933 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.933 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.933 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.934 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.934 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.934 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.934 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.934 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.934 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-16T16:54:57.934 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-16T16:54:57.934 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.935 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.935 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.935 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8", 2023-11-16T16:54:57.935 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.935 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.935 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.935 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.936 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.936 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.936 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.936 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.936 INFO:teuthology.orchestra.run.smithi093.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-16T16:54:57.936 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-16T16:54:57.936 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.936 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.937 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.937 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "45", 2023-11-16T16:54:57.937 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.937 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.938 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.938 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.938 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.938 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.939 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.939 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.939 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ACCESS_KEY": { 2023-11-16T16:54:57.939 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-16T16:54:57.939 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.939 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.940 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.940 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.940 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.940 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.940 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.940 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.940 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.941 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.941 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.941 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.941 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-16T16:54:57.941 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-16T16:54:57.941 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.941 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.942 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.942 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:54:57.942 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.942 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.942 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.942 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.942 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.942 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.943 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.943 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.943 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SECRET_KEY": { 2023-11-16T16:54:57.943 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-16T16:54:57.943 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.943 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.943 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.943 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.944 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.944 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.944 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.944 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.944 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.944 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.944 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.944 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.945 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SSL_VERIFY": { 2023-11-16T16:54:57.945 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-16T16:54:57.945 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.945 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.945 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.945 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.945 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.945 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.946 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.946 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.946 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.946 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.946 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.946 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.946 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-16T16:54:57.946 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-16T16:54:57.947 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.947 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.947 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.947 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:54:57.947 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.947 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.947 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.947 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.948 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.948 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.948 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.948 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.948 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-16T16:54:57.948 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-16T16:54:57.948 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.948 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.948 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.949 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:57.949 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.949 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.949 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.949 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.949 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.949 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.949 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.950 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.950 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-16T16:54:57.950 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-16T16:54:57.950 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.950 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.950 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.950 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-11-16T16:54:57.951 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.951 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.951 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.951 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.951 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.951 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.951 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.951 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.952 INFO:teuthology.orchestra.run.smithi093.stdout: "cross_origin_url": { 2023-11-16T16:54:57.952 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cross_origin_url", 2023-11-16T16:54:57.952 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.952 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.952 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.952 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.952 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.952 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.952 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.953 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.953 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.953 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.953 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.953 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.953 INFO:teuthology.orchestra.run.smithi093.stdout: "crt_file": { 2023-11-16T16:54:57.953 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crt_file", 2023-11-16T16:54:57.953 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.954 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.954 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.954 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.954 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.954 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.954 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.954 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.954 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.955 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.955 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.955 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.955 INFO:teuthology.orchestra.run.smithi093.stdout: "debug": { 2023-11-16T16:54:57.955 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "debug", 2023-11-16T16:54:57.955 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.955 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.955 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.955 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.956 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.956 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.956 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.956 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable/disable debug options", 2023-11-16T16:54:57.956 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.956 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.956 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.956 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.957 INFO:teuthology.orchestra.run.smithi093.stdout: "jwt_token_ttl": { 2023-11-16T16:54:57.957 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "jwt_token_ttl", 2023-11-16T16:54:57.957 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.957 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.957 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.957 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "28800", 2023-11-16T16:54:57.957 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.957 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.957 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.958 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.958 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.958 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.958 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.958 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.958 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-11-16T16:54:57.958 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-11-16T16:54:57.958 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.959 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.959 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.959 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.959 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.959 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.959 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.959 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.960 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.960 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.960 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.960 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.960 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.960 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.960 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.960 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.961 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.961 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.961 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.961 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.961 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.961 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.961 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.961 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.961 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.962 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.962 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.962 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.962 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.962 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.962 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.962 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.962 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.963 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.963 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.963 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.963 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.963 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.963 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.963 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.963 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.964 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.964 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.964 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.964 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.964 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.964 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.964 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.964 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.964 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.965 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.965 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.965 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.965 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.965 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.965 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.965 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.965 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.966 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.966 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.966 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.966 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.966 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.966 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.966 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.966 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.967 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.967 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.967 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.967 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.967 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.967 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.967 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.967 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.967 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.968 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.968 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.968 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.968 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.968 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.968 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.968 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.968 INFO:teuthology.orchestra.run.smithi093.stdout: "motd": { 2023-11-16T16:54:57.968 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "motd", 2023-11-16T16:54:57.969 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.969 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.969 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.969 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.969 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.969 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.969 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.969 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "The message of the day", 2023-11-16T16:54:57.970 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.970 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.970 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.970 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.970 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect_resolve_ip_addr": { 2023-11-16T16:54:57.970 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "redirect_resolve_ip_addr", 2023-11-16T16:54:57.970 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.971 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.971 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.971 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.971 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.971 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.971 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.971 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.971 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.971 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.972 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.972 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.972 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-11-16T16:54:57.972 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-11-16T16:54:57.972 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.972 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.972 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.972 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-11-16T16:54:57.972 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.973 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.973 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.973 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.973 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.973 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.973 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.973 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.973 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-11-16T16:54:57.974 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-11-16T16:54:57.974 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.974 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.974 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.974 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8080", 2023-11-16T16:54:57.974 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.974 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.974 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.975 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.975 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.975 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.975 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.975 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.975 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-11-16T16:54:57.975 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-11-16T16:54:57.975 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.976 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.976 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.976 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.976 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.976 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.976 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.976 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.976 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.976 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.977 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.977 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.977 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl_server_port": { 2023-11-16T16:54:57.977 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl_server_port", 2023-11-16T16:54:57.977 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.977 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.977 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.977 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8443", 2023-11-16T16:54:57.977 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.977 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.978 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.978 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.978 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.978 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.978 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.978 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.978 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-11-16T16:54:57.978 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-11-16T16:54:57.979 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.979 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.979 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.979 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "redirect", 2023-11-16T16:54:57.979 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.979 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.979 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.980 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.980 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect" 2023-11-16T16:54:57.980 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.980 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.980 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.980 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.980 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.980 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.980 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-11-16T16:54:57.981 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-11-16T16:54:57.981 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:57.981 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.981 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.981 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-11-16T16:54:57.981 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-11-16T16:54:57.981 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-11-16T16:54:57.981 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.981 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.982 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.982 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.982 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.982 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.982 INFO:teuthology.orchestra.run.smithi093.stdout: "url_prefix": { 2023-11-16T16:54:57.982 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url_prefix", 2023-11-16T16:54:57.982 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.982 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.983 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:57.983 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.983 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.983 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.983 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.983 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.983 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.983 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.983 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.984 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.984 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:57.984 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.984 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:57.984 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "devicehealth", 2023-11-16T16:54:57.984 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:57.984 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:57.984 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:57.984 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_monitoring": { 2023-11-16T16:54:57.985 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_monitoring", 2023-11-16T16:54:57.985 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.985 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.985 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.985 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:57.985 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.985 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.985 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.985 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "monitor device health metrics", 2023-11-16T16:54:57.986 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.986 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.986 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.986 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.986 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:57.986 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:57.986 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.986 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.987 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.987 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:57.987 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.987 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.987 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.987 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.987 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.987 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.987 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.988 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.988 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.988 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.988 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.988 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.988 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.988 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.988 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.988 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:57.989 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:57.989 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.989 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.989 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.989 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.989 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.989 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.989 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.990 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.990 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.990 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.990 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.990 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.990 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:57.990 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:57.990 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.991 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.991 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.991 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:57.991 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.991 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.991 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:57.991 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:57.991 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:57.991 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:57.992 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:57.992 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:57.992 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:57.992 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:57.992 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.992 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.992 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.992 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.992 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.993 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:57.993 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:57.993 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:57.993 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.993 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.993 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:57.993 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.993 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.993 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.994 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:57.994 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.994 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.994 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.994 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.994 INFO:teuthology.orchestra.run.smithi093.stdout: "mark_out_threshold": { 2023-11-16T16:54:57.994 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mark_out_threshold", 2023-11-16T16:54:57.994 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.995 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.995 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.995 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2419200", 2023-11-16T16:54:57.995 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.995 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.995 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.995 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-16T16:54:57.995 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.995 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.996 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.996 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.996 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_name": { 2023-11-16T16:54:57.996 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_name", 2023-11-16T16:54:57.996 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:57.996 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.996 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.996 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "device_health_metrics", 2023-11-16T16:54:57.997 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.997 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.997 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.997 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-16T16:54:57.997 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.997 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.997 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.997 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.997 INFO:teuthology.orchestra.run.smithi093.stdout: "retention_period": { 2023-11-16T16:54:57.998 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retention_period", 2023-11-16T16:54:57.998 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:57.998 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:57.998 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:57.998 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15552000", 2023-11-16T16:54:57.998 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:57.998 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:57.998 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:57.999 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain device health metrics", 2023-11-16T16:54:57.999 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:57.999 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:57.999 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:57.999 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:57.999 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_frequency": { 2023-11-16T16:54:57.999 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_frequency", 2023-11-16T16:54:58.000 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:58.000 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.000 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.000 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-11-16T16:54:58.000 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.000 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.000 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.000 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-16T16:54:58.001 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.001 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.001 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.001 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.001 INFO:teuthology.orchestra.run.smithi093.stdout: "self_heal": { 2023-11-16T16:54:58.001 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "self_heal", 2023-11-16T16:54:58.001 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.001 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.001 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.001 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:54:58.002 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.002 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.002 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.002 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-16T16:54:58.002 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.002 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.002 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.002 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.003 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:54:58.003 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:54:58.003 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:58.003 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.003 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.003 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:54:58.003 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.003 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.003 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.004 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and check device health", 2023-11-16T16:54:58.004 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.004 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.004 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.004 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.004 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_threshold": { 2023-11-16T16:54:58.004 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_threshold", 2023-11-16T16:54:58.004 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:58.005 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.005 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.005 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7257600", 2023-11-16T16:54:58.005 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.005 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.005 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.005 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-16T16:54:58.005 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.005 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.006 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.006 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.006 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.006 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.006 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.006 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "diskprediction_local", 2023-11-16T16:54:58.006 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:58.006 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:58.007 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.007 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.007 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.007 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.007 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.007 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.007 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.007 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.007 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.008 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.008 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.008 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.008 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.008 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.008 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.008 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.008 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.008 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.009 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.009 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.009 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.009 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.009 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.009 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.009 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.009 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.010 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.010 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.010 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.010 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.010 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.010 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.010 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.010 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.011 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.011 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.011 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.011 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.011 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.011 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.011 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.011 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.011 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.012 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.012 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.012 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.012 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.012 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.012 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.012 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.012 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.013 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.013 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.013 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.013 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.013 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.013 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.013 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.013 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.013 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.014 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.014 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.014 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.014 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.014 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.014 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.014 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.014 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.015 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.015 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.015 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.015 INFO:teuthology.orchestra.run.smithi093.stdout: "predict_interval": { 2023-11-16T16:54:58.015 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predict_interval", 2023-11-16T16:54:58.015 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.015 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.015 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.015 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-11-16T16:54:58.016 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.016 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.016 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.016 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.016 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.016 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.016 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.016 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.017 INFO:teuthology.orchestra.run.smithi093.stdout: "predictor_model": { 2023-11-16T16:54:58.017 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predictor_model", 2023-11-16T16:54:58.017 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.017 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.017 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.017 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "prophetstor", 2023-11-16T16:54:58.017 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.017 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.018 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.018 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.018 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.018 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.018 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.018 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.018 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:54:58.018 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:54:58.019 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.019 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.019 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.019 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:54:58.019 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.019 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.019 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.019 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.020 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.020 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.020 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.020 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.020 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.020 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.020 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.020 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "influx", 2023-11-16T16:54:58.021 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": false, 2023-11-16T16:54:58.021 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "influxdb python module not found", 2023-11-16T16:54:58.021 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.021 INFO:teuthology.orchestra.run.smithi093.stdout: "batch_size": { 2023-11-16T16:54:58.021 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "batch_size", 2023-11-16T16:54:58.021 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:58.021 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.021 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.021 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5000", 2023-11-16T16:54:58.022 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.022 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.022 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.022 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-16T16:54:58.022 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.022 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.022 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.022 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.023 INFO:teuthology.orchestra.run.smithi093.stdout: "database": { 2023-11-16T16:54:58.023 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "database", 2023-11-16T16:54:58.023 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.023 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.023 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.023 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "ceph", 2023-11-16T16:54:58.023 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.023 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.023 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.024 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-16T16:54:58.024 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.024 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.024 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.024 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.024 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": { 2023-11-16T16:54:58.024 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "hostname", 2023-11-16T16:54:58.024 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.024 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.025 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.025 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.025 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.025 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.025 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.025 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server hostname", 2023-11-16T16:54:58.025 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.025 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.025 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.026 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.026 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:54:58.026 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:54:58.026 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:54:58.026 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.026 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.026 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "30", 2023-11-16T16:54:58.026 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "5", 2023-11-16T16:54:58.026 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.027 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.027 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-16T16:54:58.027 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.027 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.027 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.027 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.027 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.027 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.027 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.028 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.028 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.028 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.028 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.028 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.028 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.028 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.028 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.028 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.029 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.029 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.029 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.029 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.029 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.029 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.029 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.029 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.030 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.030 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.030 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.030 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.030 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.030 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.030 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.030 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.031 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.031 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.031 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.031 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.031 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.031 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.031 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.031 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.031 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.032 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.032 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.032 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.032 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.032 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.032 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.032 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.032 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.032 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.033 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.033 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.033 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.033 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.033 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.033 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.033 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.033 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.033 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.034 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.034 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.034 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.034 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.034 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.034 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.034 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.034 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.034 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.035 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.035 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.035 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.035 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.035 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.035 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.035 INFO:teuthology.orchestra.run.smithi093.stdout: "password": { 2023-11-16T16:54:58.035 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "password", 2023-11-16T16:54:58.035 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.036 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.036 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.036 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.036 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.036 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.036 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.036 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "password of InfluxDB server user", 2023-11-16T16:54:58.036 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.037 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.037 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.037 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.037 INFO:teuthology.orchestra.run.smithi093.stdout: "port": { 2023-11-16T16:54:58.037 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "port", 2023-11-16T16:54:58.037 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:58.037 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.037 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.038 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8086", 2023-11-16T16:54:58.038 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.038 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.038 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.038 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server port", 2023-11-16T16:54:58.038 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.038 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.038 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.039 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.039 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-11-16T16:54:58.039 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-11-16T16:54:58.039 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.039 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.039 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.039 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "false", 2023-11-16T16:54:58.039 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.040 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.040 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.040 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-16T16:54:58.040 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.040 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.040 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.040 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.040 INFO:teuthology.orchestra.run.smithi093.stdout: "threads": { 2023-11-16T16:54:58.040 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threads", 2023-11-16T16:54:58.041 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:58.041 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.041 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.041 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-11-16T16:54:58.041 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-11-16T16:54:58.041 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "32", 2023-11-16T16:54:58.041 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.041 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-16T16:54:58.041 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.042 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.042 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.042 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.042 INFO:teuthology.orchestra.run.smithi093.stdout: "username": { 2023-11-16T16:54:58.042 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "username", 2023-11-16T16:54:58.042 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.042 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.042 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.043 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.043 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.043 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.043 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.043 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "username of InfluxDB server user", 2023-11-16T16:54:58.043 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.043 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.043 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.043 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.044 INFO:teuthology.orchestra.run.smithi093.stdout: "verify_ssl": { 2023-11-16T16:54:58.044 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "verify_ssl", 2023-11-16T16:54:58.044 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.044 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.044 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.044 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "true", 2023-11-16T16:54:58.044 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.044 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.045 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.045 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-16T16:54:58.045 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.045 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.045 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.045 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.045 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.045 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.045 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.046 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "insights", 2023-11-16T16:54:58.046 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:58.046 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:58.046 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.046 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.046 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.046 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.046 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.047 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.047 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.047 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.047 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.047 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.047 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.047 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.047 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.047 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.047 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.048 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.048 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.048 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.048 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.048 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.048 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.048 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.048 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.049 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.049 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.049 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.049 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.049 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.049 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.049 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.049 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.050 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.050 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.050 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.050 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.050 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.050 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.050 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.050 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.051 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.051 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.051 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.051 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.051 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.051 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.051 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.052 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.052 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.052 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.052 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.052 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.052 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.052 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.052 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.052 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.053 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.053 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.053 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.053 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.053 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.053 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.053 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.053 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.053 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.054 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.054 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.054 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.054 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.054 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.054 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.054 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.054 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.055 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.055 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.055 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "iostat", 2023-11-16T16:54:58.055 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:58.055 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:58.055 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.055 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.055 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.055 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.056 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.056 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.056 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.056 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.056 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.056 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.056 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.056 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.056 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.057 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.057 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.057 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.057 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.057 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.057 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.057 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.057 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.057 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.058 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.058 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.058 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.058 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.058 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.058 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.058 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.058 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.058 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.059 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.059 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.059 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.059 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.059 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.059 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.059 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.059 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.060 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.060 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.060 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.060 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.060 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.060 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.060 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.060 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.061 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.061 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.061 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.061 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.061 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.061 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.061 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.061 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.062 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.062 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.062 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.062 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.062 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.062 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.062 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.062 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.062 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.063 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.063 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.063 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.063 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.063 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.063 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.063 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.063 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.063 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.064 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.064 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "k8sevents", 2023-11-16T16:54:58.064 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:58.064 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:58.064 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.064 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_event_retention_days": { 2023-11-16T16:54:58.064 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ceph_event_retention_days", 2023-11-16T16:54:58.064 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:58.064 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.065 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.065 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7", 2023-11-16T16:54:58.065 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.065 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.065 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.065 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-16T16:54:58.065 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.065 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.065 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.066 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.066 INFO:teuthology.orchestra.run.smithi093.stdout: "config_check_secs": { 2023-11-16T16:54:58.066 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_check_secs", 2023-11-16T16:54:58.066 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:58.167 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.167 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.167 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:54:58.167 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "10", 2023-11-16T16:54:58.167 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.167 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.167 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-16T16:54:58.168 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.168 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.168 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.168 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.168 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.168 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.168 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.168 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.169 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.169 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.169 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.169 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.169 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.169 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.169 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.169 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.170 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.170 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.170 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.170 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.170 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.170 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.170 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.171 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.171 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.171 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.171 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.171 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.171 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.171 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.171 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.172 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.172 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.172 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.172 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.172 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.172 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.172 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.172 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.173 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.173 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.173 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.173 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.173 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.173 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.173 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.173 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.174 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.174 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.174 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.174 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.174 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.174 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.174 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.174 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.175 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.175 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.175 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.175 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.176 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.176 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.176 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.176 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.176 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.176 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.176 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.176 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.177 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.177 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.177 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.177 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.177 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.177 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.177 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.177 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.178 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.178 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.178 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "localpool", 2023-11-16T16:54:58.178 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:58.178 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:58.178 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.178 INFO:teuthology.orchestra.run.smithi093.stdout: "failure_domain": { 2023-11-16T16:54:58.178 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "failure_domain", 2023-11-16T16:54:58.179 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.179 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.179 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.179 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "host", 2023-11-16T16:54:58.179 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.179 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.179 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.179 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "failure domain for any created local pool", 2023-11-16T16:54:58.180 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-16T16:54:58.180 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.180 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.180 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.180 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.180 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.180 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.180 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.181 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.181 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.181 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.181 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.181 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.181 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.181 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.181 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.182 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.182 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.182 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.182 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.182 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.182 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.182 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.182 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.183 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.183 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.183 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.183 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.183 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.183 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.183 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.183 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.184 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.184 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.184 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.184 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.184 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.184 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.184 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.184 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.185 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.185 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.185 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.185 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.185 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.185 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.185 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.185 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.186 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.186 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.186 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.186 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.186 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.186 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.186 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.186 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.187 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.187 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.187 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.187 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.187 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.187 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.187 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.187 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.187 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.188 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.188 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.188 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.188 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.188 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.188 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.188 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.188 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.189 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.189 INFO:teuthology.orchestra.run.smithi093.stdout: "min_size": { 2023-11-16T16:54:58.189 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_size", 2023-11-16T16:54:58.189 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:58.189 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.189 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.189 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.189 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.190 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.190 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.190 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default min_size for any created local pool", 2023-11-16T16:54:58.190 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-16T16:54:58.190 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.190 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.190 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.190 INFO:teuthology.orchestra.run.smithi093.stdout: "num_rep": { 2023-11-16T16:54:58.191 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "num_rep", 2023-11-16T16:54:58.191 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:58.191 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.191 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.191 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3", 2023-11-16T16:54:58.191 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.191 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.191 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.192 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default replica count for any created local pool", 2023-11-16T16:54:58.192 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.192 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.192 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.192 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.192 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_num": { 2023-11-16T16:54:58.192 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_num", 2023-11-16T16:54:58.192 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:54:58.193 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.193 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.193 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "128", 2023-11-16T16:54:58.193 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.193 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.193 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.193 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default pg_num for any created local pool", 2023-11-16T16:54:58.193 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.193 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.194 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.194 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.194 INFO:teuthology.orchestra.run.smithi093.stdout: "prefix": { 2023-11-16T16:54:58.194 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prefix", 2023-11-16T16:54:58.194 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.194 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.194 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.194 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.195 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.195 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.195 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.195 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name prefix for any created local pool", 2023-11-16T16:54:58.195 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.195 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.195 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.195 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.196 INFO:teuthology.orchestra.run.smithi093.stdout: "subtree": { 2023-11-16T16:54:58.196 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "subtree", 2023-11-16T16:54:58.196 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.196 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.196 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.196 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "rack", 2023-11-16T16:54:58.196 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.196 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.197 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.197 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-16T16:54:58.197 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-16T16:54:58.197 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.197 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.197 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.197 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.197 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.198 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.198 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mds_autoscaler", 2023-11-16T16:54:58.198 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:58.198 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:58.198 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.198 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.198 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.198 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.199 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.199 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.199 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.199 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.199 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.199 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.199 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.200 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.200 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.200 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.200 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.200 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.200 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.200 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.200 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.201 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.201 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.201 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.201 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.201 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.201 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.201 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.201 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.202 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.202 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.202 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.202 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.202 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.202 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.202 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.202 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.203 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.203 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.203 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.203 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.203 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.203 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.203 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.203 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.203 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.204 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.204 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.204 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.204 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.204 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.204 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.204 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.204 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.205 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.205 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.205 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.205 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.205 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.205 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.205 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.205 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.206 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.206 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.206 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.206 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.206 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.206 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.206 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.206 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.207 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.207 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.207 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.207 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.207 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.208 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.208 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirroring", 2023-11-16T16:54:58.208 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:58.208 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:58.208 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.208 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.208 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.208 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.209 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.209 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.209 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.209 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.209 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.209 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.209 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.209 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.210 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.210 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.210 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.210 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.210 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.210 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.210 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.210 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.211 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.211 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.211 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.211 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.211 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.211 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.211 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.211 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.212 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.212 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.212 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.212 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.212 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.212 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.212 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.212 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.213 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.213 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.213 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.213 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.213 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.213 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.213 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.213 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.213 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.214 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.214 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.214 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.214 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.214 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.214 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.214 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.215 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.215 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.215 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.215 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.215 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.215 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.215 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.215 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.215 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.216 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.216 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.216 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.216 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.216 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.216 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.216 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.217 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.217 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.217 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.217 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.217 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.217 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.217 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "nfs", 2023-11-16T16:54:58.217 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:58.218 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:58.218 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.218 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.218 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.218 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.218 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.218 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.218 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.219 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.219 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.219 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.219 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.219 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.219 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.219 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.219 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.220 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.220 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.220 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.220 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.220 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.220 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.220 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.220 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.221 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.221 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.221 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.221 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.221 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.221 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.221 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.221 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.222 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.222 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.222 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.222 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.222 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.222 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.222 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.222 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.223 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.223 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.223 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.223 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.223 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.223 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.223 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.223 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.223 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.224 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.224 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.224 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.224 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.224 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.224 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.224 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.224 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.225 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.225 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.225 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.225 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.225 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.225 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.225 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.225 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.226 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.226 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.226 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.226 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.226 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.226 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.226 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.226 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.227 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.227 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.227 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-11-16T16:54:58.227 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:58.227 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:58.227 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.227 INFO:teuthology.orchestra.run.smithi093.stdout: "fail_fs": { 2023-11-16T16:54:58.227 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "fail_fs", 2023-11-16T16:54:58.228 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.228 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.228 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.228 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.228 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.228 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.228 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.228 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-16T16:54:58.229 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.229 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.229 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.229 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.229 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.229 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.229 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.230 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.230 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.230 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.230 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.230 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.230 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.230 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.230 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.231 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.231 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.231 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.231 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.231 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.231 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.231 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.231 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.232 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.232 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.232 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.232 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.232 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.232 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.232 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.232 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.233 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.233 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.233 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.233 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.233 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.233 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.233 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.234 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.234 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.234 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.234 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.234 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.234 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.234 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.234 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.235 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.235 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.235 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.235 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.235 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.235 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.235 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.236 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.236 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.236 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.236 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.236 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.236 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.236 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.236 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.237 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.237 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.237 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.237 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.237 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.237 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.237 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.238 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.238 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.238 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.238 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.238 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.238 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.238 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator": { 2023-11-16T16:54:58.238 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-11-16T16:54:58.239 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.239 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.239 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.239 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.239 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.239 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.239 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.240 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm", 2023-11-16T16:54:58.240 INFO:teuthology.orchestra.run.smithi093.stdout: "rook", 2023-11-16T16:54:58.240 INFO:teuthology.orchestra.run.smithi093.stdout: "test_orchestrator" 2023-11-16T16:54:58.240 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.240 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Orchestrator backend", 2023-11-16T16:54:58.240 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.240 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.241 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.241 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.241 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.241 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.241 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.241 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_perf_query", 2023-11-16T16:54:58.241 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:58.241 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:58.242 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.242 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.242 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.242 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.242 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.242 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.242 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.242 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.243 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.243 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.243 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.243 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.243 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.243 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.243 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.244 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.244 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.244 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.244 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.244 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.244 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.244 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.244 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.245 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.245 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.245 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.245 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.245 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.245 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.245 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.245 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.246 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.246 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.246 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.246 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.246 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.246 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.246 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.246 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.247 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.247 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.247 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.247 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.247 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.247 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.247 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.248 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.248 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.248 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.248 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.248 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.248 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.248 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.248 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.249 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.249 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.249 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.249 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.249 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.249 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.249 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.249 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.250 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.250 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.250 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.250 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.250 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.250 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.250 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.251 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.251 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.251 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.251 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.251 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.251 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_support", 2023-11-16T16:54:58.251 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:58.251 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:58.252 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.252 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.252 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.252 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.252 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.252 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.252 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.252 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.253 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.253 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.253 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.253 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.253 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.253 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.253 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.253 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.254 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.254 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.254 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.254 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.254 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.254 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.254 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.255 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.255 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.255 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.255 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.255 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.255 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.255 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.255 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.256 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.256 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.256 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.256 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.256 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.256 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.256 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.256 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.257 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.257 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.257 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.257 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.257 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.257 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.257 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.257 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.258 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.258 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.258 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.258 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.258 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.258 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.258 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.259 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.259 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.259 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.259 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.259 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.259 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.259 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.259 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.260 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.260 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.260 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.260 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.260 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.260 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.260 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.260 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.261 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.261 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:54:58.261 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.261 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:54:58.261 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_autoscaler", 2023-11-16T16:54:58.261 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:54:58.261 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:54:58.261 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:54:58.262 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:54:58.262 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:54:58.262 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.262 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.262 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.262 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:54:58.262 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.262 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.263 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.263 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.263 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.263 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.263 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.263 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.263 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.263 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.264 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.264 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.264 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.264 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.264 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.264 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:54:58.264 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:54:58.264 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.265 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.265 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.265 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.265 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.265 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.265 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.265 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.265 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.265 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.266 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.266 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.266 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:54:58.266 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:54:58.266 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:54:58.266 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.266 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.267 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:54:58.267 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.267 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.267 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:54:58.267 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:54:58.267 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:54:58.267 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:54:58.267 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:54:58.267 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:54:58.268 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:54:58.268 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:54:58.268 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.268 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.268 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.268 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.268 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.268 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:54:58.269 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:54:58.269 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.269 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.269 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:54:58.269 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.269 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.269 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:54:58.269 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:54:58.270 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:54:58.270 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:54:58.270 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:54:58.270 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:54:58.270 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:54:58.270 INFO:teuthology.orchestra.run.smithi093.stdout: "noautoscale": { 2023-11-16T16:54:58.270 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "noautoscale", 2023-11-16T16:54:58.271 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:54:58.271 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:54:58.271 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:54:58.271 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:54:58.271 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:54:58.271 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.399 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.400 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "global autoscale flag", 2023-11-16T16:55:05.400 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-16T16:55:05.400 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.400 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.400 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.400 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:55:05.400 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:55:05.401 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.401 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.401 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.401 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:55:05.401 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.401 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.401 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.402 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.402 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.402 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.402 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.402 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.402 INFO:teuthology.orchestra.run.smithi093.stdout: "threshold": { 2023-11-16T16:55:05.403 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threshold", 2023-11-16T16:55:05.403 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:55:05.403 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.403 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.403 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-11-16T16:55:05.403 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1.0", 2023-11-16T16:55:05.403 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.403 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.404 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "scaling threshold", 2023-11-16T16:55:05.404 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-16T16:55:05.404 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.404 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.404 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.404 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.404 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.405 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.405 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "progress", 2023-11-16T16:55:05.405 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.405 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.405 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.405 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_pg_recovery_event": { 2023-11-16T16:55:05.405 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_pg_recovery_event", 2023-11-16T16:55:05.405 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.406 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.406 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.406 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.406 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.406 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.406 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.406 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-16T16:55:05.407 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.407 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.407 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.407 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.407 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-11-16T16:55:05.407 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-11-16T16:55:05.407 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.408 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.408 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.408 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.408 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.408 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.408 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.408 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.409 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.409 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.409 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.409 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.409 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.409 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.409 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.409 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.410 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.410 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.410 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.410 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.410 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.410 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.410 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.411 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.411 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.411 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.411 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.411 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.411 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.411 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.412 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.412 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.412 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.412 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.412 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.412 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.412 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.413 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.413 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.413 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.413 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.413 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.413 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.413 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.413 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.414 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.414 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.414 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.414 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.414 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.414 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.414 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.415 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.415 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.415 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.415 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.415 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.415 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.415 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.415 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.416 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.416 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.416 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.416 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.416 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.416 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.416 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.417 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.417 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.438 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.438 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.438 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.438 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.438 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.438 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.438 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.439 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.439 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.439 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.439 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.439 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.439 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.439 INFO:teuthology.orchestra.run.smithi093.stdout: "max_completed_events": { 2023-11-16T16:55:05.439 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_completed_events", 2023-11-16T16:55:05.440 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.440 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.440 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.440 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "50", 2023-11-16T16:55:05.440 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.440 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.440 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.440 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "number of past completed events to remember", 2023-11-16T16:55:05.441 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.441 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.441 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.441 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.441 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:55:05.441 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:55:05.441 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.441 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.442 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.442 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-11-16T16:55:05.442 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.442 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.442 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.442 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long the module is going to sleep", 2023-11-16T16:55:05.442 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.442 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.443 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.443 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.443 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.443 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.443 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.443 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus", 2023-11-16T16:55:05.443 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.443 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.444 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.444 INFO:teuthology.orchestra.run.smithi093.stdout: "cache": { 2023-11-16T16:55:05.444 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cache", 2023-11-16T16:55:05.444 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.444 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.444 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.444 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.444 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.445 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.445 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.445 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.445 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.445 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.445 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.445 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.446 INFO:teuthology.orchestra.run.smithi093.stdout: "exclude_perf_counters": { 2023-11-16T16:55:05.446 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "exclude_perf_counters", 2023-11-16T16:55:05.446 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.446 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.446 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.446 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.446 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.446 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.447 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.447 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-16T16:55:05.447 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-16T16:55:05.447 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.447 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.447 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.447 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.447 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.448 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.448 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.448 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.448 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.448 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.448 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.448 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.448 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.449 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.449 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.468 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.469 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.469 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.469 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.469 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.469 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.469 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.469 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.469 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.470 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.470 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.470 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.470 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.470 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.470 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.470 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.470 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.471 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.471 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.471 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.471 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.471 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.471 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.471 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.471 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.472 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.472 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.472 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.472 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.472 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.472 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.472 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.472 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.472 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.473 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.473 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.473 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.473 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.473 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.473 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.473 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.473 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.474 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.474 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.474 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.474 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.474 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.474 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.474 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.475 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.475 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.475 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.475 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.475 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.475 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.475 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.475 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.476 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.476 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools": { 2023-11-16T16:55:05.476 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools", 2023-11-16T16:55:05.476 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.476 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.476 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.476 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.476 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.477 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.477 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.477 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.477 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.477 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.477 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.477 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.477 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-16T16:55:05.478 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-16T16:55:05.478 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.478 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.478 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.478 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300", 2023-11-16T16:55:05.478 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.478 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.478 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.479 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.479 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.479 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.479 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.479 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.479 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_interval": { 2023-11-16T16:55:05.479 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_interval", 2023-11-16T16:55:05.479 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:55:05.480 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.480 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.480 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15.0", 2023-11-16T16:55:05.480 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.480 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.480 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.480 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.480 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.480 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.481 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.481 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.481 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-11-16T16:55:05.481 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-11-16T16:55:05.481 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.481 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.481 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.481 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-11-16T16:55:05.482 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.482 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.482 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.482 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-16T16:55:05.482 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.482 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.482 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.482 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.483 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-11-16T16:55:05.483 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-11-16T16:55:05.483 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.483 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.483 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.483 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "9283", 2023-11-16T16:55:05.483 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.483 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.484 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.484 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-16T16:55:05.484 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.484 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.484 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.484 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.484 INFO:teuthology.orchestra.run.smithi093.stdout: "stale_cache_strategy": { 2023-11-16T16:55:05.484 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stale_cache_strategy", 2023-11-16T16:55:05.485 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.485 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.485 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.485 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "log", 2023-11-16T16:55:05.485 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.485 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.485 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.485 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.485 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.486 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.486 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.486 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.486 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-11-16T16:55:05.486 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-11-16T16:55:05.486 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.486 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.486 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.487 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "default", 2023-11-16T16:55:05.487 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.487 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.487 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.487 INFO:teuthology.orchestra.run.smithi093.stdout: "default", 2023-11-16T16:55:05.487 INFO:teuthology.orchestra.run.smithi093.stdout: "error" 2023-11-16T16:55:05.487 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.487 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.488 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.489 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.489 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.489 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.489 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-11-16T16:55:05.490 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-11-16T16:55:05.490 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.490 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.490 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.490 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-11-16T16:55:05.490 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-11-16T16:55:05.490 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-11-16T16:55:05.490 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.491 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.491 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.491 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.491 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.491 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.491 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.491 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.491 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.491 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_support", 2023-11-16T16:55:05.492 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.492 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.492 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.492 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.492 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.492 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.492 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.492 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.492 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.493 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.508 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.508 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.508 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.509 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.509 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.509 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.509 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.509 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.509 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.509 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.509 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.510 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.510 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.510 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.510 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.510 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.510 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.510 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.510 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.511 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.511 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.511 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.511 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.511 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.511 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.511 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.511 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.511 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.512 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.512 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.512 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.512 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.512 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.512 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.512 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.512 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.513 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.513 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.513 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.513 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.513 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.513 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.513 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.513 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.513 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.514 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.514 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.514 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.514 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.514 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.514 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.514 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.514 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.515 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.515 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.515 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.515 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.515 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.515 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.515 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.515 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.516 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.516 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.516 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_snap_create": { 2023-11-16T16:55:05.516 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_snap_create", 2023-11-16T16:55:05.516 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.516 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.516 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.516 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:55:05.517 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.517 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.517 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.517 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.517 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.517 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.517 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.517 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.517 INFO:teuthology.orchestra.run.smithi093.stdout: "mirror_snapshot_schedule": { 2023-11-16T16:55:05.518 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirror_snapshot_schedule", 2023-11-16T16:55:05.518 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.518 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.518 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.518 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.518 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.518 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.519 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.519 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.519 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.519 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.519 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.519 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.519 INFO:teuthology.orchestra.run.smithi093.stdout: "trash_purge_schedule": { 2023-11-16T16:55:05.519 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "trash_purge_schedule", 2023-11-16T16:55:05.520 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.520 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.520 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.520 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.520 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.520 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.520 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.520 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.521 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.521 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.521 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.521 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.521 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.521 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.521 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.521 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "restful", 2023-11-16T16:55:05.521 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.522 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.522 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.522 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_auth": { 2023-11-16T16:55:05.522 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_auth", 2023-11-16T16:55:05.522 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.522 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.522 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.522 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.523 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.523 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.523 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.523 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.523 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.523 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.523 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.523 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.524 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-11-16T16:55:05.524 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-11-16T16:55:05.524 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.524 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.524 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.524 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.524 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.524 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.524 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.525 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.525 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.525 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.525 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.525 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.525 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.525 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.525 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.526 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.526 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.526 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.526 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.526 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.526 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.526 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.526 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.527 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.527 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.527 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.527 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.527 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.527 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.527 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.527 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.527 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.528 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.528 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.528 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.528 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.528 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.528 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.528 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.528 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.529 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.529 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.529 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.529 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.529 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.529 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.529 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.529 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.530 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.530 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.530 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.530 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.530 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.530 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.530 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.531 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.531 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.531 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.531 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.531 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.531 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.531 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.531 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.532 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.532 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.532 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.532 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.532 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.532 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.532 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.532 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.532 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.533 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.533 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.533 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.533 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.533 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.533 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.533 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.533 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.534 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.534 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.534 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-11-16T16:55:05.534 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-11-16T16:55:05.534 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.534 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.534 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.534 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.535 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.535 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.535 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.535 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.535 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.535 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.535 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.535 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.536 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-11-16T16:55:05.536 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-11-16T16:55:05.536 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.536 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.536 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.536 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.536 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.536 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.537 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.537 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.537 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.537 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.537 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.537 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.537 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.537 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.538 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.538 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rgw", 2023-11-16T16:55:05.538 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.538 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.538 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.538 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.538 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.539 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.539 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.539 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.539 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.539 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.539 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.539 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.539 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.540 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.540 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.540 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.540 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.540 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.540 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.540 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.540 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.541 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.541 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.541 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.541 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.541 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.541 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.541 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.541 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.542 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.542 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.542 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.542 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.542 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.542 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.542 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.543 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.543 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.543 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.543 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.543 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.543 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.543 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.543 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.544 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.544 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.544 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.544 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.544 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.544 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.544 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.544 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.545 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.545 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.545 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.545 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.545 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.545 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.545 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.545 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.546 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.546 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.546 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.546 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.546 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.546 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.546 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.546 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.547 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.547 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.547 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.547 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.547 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.547 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.547 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.547 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.548 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rook", 2023-11-16T16:55:05.548 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.548 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.548 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.548 INFO:teuthology.orchestra.run.smithi093.stdout: "drive_group_interval": { 2023-11-16T16:55:05.548 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "drive_group_interval", 2023-11-16T16:55:05.548 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:55:05.548 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.549 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.549 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300.0", 2023-11-16T16:55:05.549 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.549 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.549 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.549 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-16T16:55:05.549 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.549 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.550 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.550 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.550 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.550 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.550 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.550 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.550 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.551 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.551 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.551 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.551 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.551 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.551 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.551 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.551 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.552 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.552 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.552 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.552 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.552 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.552 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.552 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.552 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.553 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.553 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.553 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.553 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.553 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.553 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.553 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.553 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.554 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.554 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.554 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.554 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.554 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.554 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.554 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.554 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.555 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.555 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.555 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.555 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.555 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.555 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.555 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.555 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.556 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.556 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.556 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.556 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.556 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.556 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.556 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.556 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.557 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.557 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.557 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.557 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.557 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.557 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.557 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.557 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.558 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.558 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.558 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.558 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.558 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.558 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.558 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.558 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.559 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.559 INFO:teuthology.orchestra.run.smithi093.stdout: "storage_class": { 2023-11-16T16:55:05.559 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "storage_class", 2023-11-16T16:55:05.559 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.559 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.559 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.559 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "local", 2023-11-16T16:55:05.559 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.560 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.560 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.560 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-16T16:55:05.560 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.560 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.560 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.560 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.560 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.561 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.561 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.561 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "selftest", 2023-11-16T16:55:05.561 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.561 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.561 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.561 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.561 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.562 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.562 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.562 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.562 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.562 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.562 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.562 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.563 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.563 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.563 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.563 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.563 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.563 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.563 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.563 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.564 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.564 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.564 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.564 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.564 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.564 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.564 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.564 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.565 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.565 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.565 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.565 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.565 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.565 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.565 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.565 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.566 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.566 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.566 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.566 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.566 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.566 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.566 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.567 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.567 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.567 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.567 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.567 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.567 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.567 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.567 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.568 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.568 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.568 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.568 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.568 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.568 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.568 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.568 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.569 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.569 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.569 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.569 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.569 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.569 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.569 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.569 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.570 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.570 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.570 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.570 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.570 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.570 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.570 INFO:teuthology.orchestra.run.smithi093.stdout: "roption1": { 2023-11-16T16:55:05.570 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption1", 2023-11-16T16:55:05.571 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.571 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.571 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.571 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.571 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.571 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.571 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.571 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.572 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.572 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.572 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.572 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.572 INFO:teuthology.orchestra.run.smithi093.stdout: "roption2": { 2023-11-16T16:55:05.572 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption2", 2023-11-16T16:55:05.572 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.572 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.573 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.573 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "xyz", 2023-11-16T16:55:05.573 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.573 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.573 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.573 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.573 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.573 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.574 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.574 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.574 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption1": { 2023-11-16T16:55:05.574 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption1", 2023-11-16T16:55:05.574 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.574 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.574 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.574 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.575 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.575 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.575 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.575 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.575 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.575 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.575 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.575 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.576 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption2": { 2023-11-16T16:55:05.576 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption2", 2023-11-16T16:55:05.576 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.576 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.576 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.576 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.576 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.576 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.577 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.577 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.577 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.577 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.577 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.577 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.577 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption3": { 2023-11-16T16:55:05.578 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption3", 2023-11-16T16:55:05.578 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:55:05.578 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.578 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.578 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.578 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.578 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.578 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.579 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.579 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.579 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.579 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.579 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.579 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption4": { 2023-11-16T16:55:05.579 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption4", 2023-11-16T16:55:05.580 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.580 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.580 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.580 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.580 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.580 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.580 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.580 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.581 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.581 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.581 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.581 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.581 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption5": { 2023-11-16T16:55:05.581 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption5", 2023-11-16T16:55:05.581 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.581 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.582 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.582 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.582 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.582 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.582 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.582 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.582 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.582 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.583 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.583 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.583 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption6": { 2023-11-16T16:55:05.583 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption6", 2023-11-16T16:55:05.583 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.583 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.583 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.583 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.584 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.584 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.584 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.584 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.584 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.584 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.584 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.585 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.585 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption7": { 2023-11-16T16:55:05.585 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption7", 2023-11-16T16:55:05.585 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.585 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.585 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.585 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.585 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-11-16T16:55:05.585 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "42", 2023-11-16T16:55:05.586 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.586 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.586 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.586 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.586 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.586 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.586 INFO:teuthology.orchestra.run.smithi093.stdout: "testkey": { 2023-11-16T16:55:05.587 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testkey", 2023-11-16T16:55:05.587 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.587 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.587 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.587 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.587 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.587 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.587 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.588 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.588 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.588 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.588 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.588 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.588 INFO:teuthology.orchestra.run.smithi093.stdout: "testlkey": { 2023-11-16T16:55:05.588 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testlkey", 2023-11-16T16:55:05.588 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.589 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.589 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.589 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.589 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.589 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.589 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.589 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.590 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.590 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.590 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.590 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.590 INFO:teuthology.orchestra.run.smithi093.stdout: "testnewline": { 2023-11-16T16:55:05.590 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testnewline", 2023-11-16T16:55:05.590 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.590 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.591 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.591 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.591 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.591 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.591 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.591 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.591 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.591 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.592 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.592 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.592 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.592 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.592 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.592 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-11-16T16:55:05.592 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.592 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.593 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.593 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_m_granularity": { 2023-11-16T16:55:05.593 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_m_granularity", 2023-11-16T16:55:05.593 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.593 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.593 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.593 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.593 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.594 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.594 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.594 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow minute scheduled snapshots", 2023-11-16T16:55:05.594 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.594 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.594 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.594 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.595 INFO:teuthology.orchestra.run.smithi093.stdout: "dump_on_update": { 2023-11-16T16:55:05.595 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dump_on_update", 2023-11-16T16:55:05.595 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.595 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.595 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.595 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.595 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.595 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.596 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.596 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "dump database to debug log on update", 2023-11-16T16:55:05.596 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.596 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.596 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.596 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.596 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.596 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.597 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.597 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.597 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.597 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.597 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.597 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.597 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.597 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.598 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.598 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.598 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.598 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.598 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.598 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.598 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.598 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.599 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.599 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.599 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.599 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.599 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.599 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.599 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.599 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.600 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.600 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.600 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.600 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.600 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.600 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.600 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.601 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.601 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.601 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.601 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.601 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.601 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.601 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.601 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.602 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.602 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.602 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.602 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.602 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.602 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.602 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.603 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.603 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.603 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.603 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.603 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.603 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.603 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.603 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.604 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.604 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.604 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.604 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.604 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.604 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.604 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.604 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.605 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.605 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.605 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.605 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.605 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.605 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.605 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.605 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.606 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.606 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stats", 2023-11-16T16:55:05.606 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.606 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.606 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.606 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.606 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.606 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.607 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.607 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.607 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.607 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.607 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.607 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.607 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.607 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.608 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.608 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.608 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.608 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.608 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.608 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.608 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.609 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.609 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.609 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.609 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.609 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.609 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.609 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.609 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.610 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.610 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.610 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.610 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.610 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.610 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.610 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.610 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.611 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.611 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.611 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.611 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.611 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.611 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.611 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.611 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.612 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.612 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.612 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.612 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.612 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.612 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.612 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.612 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.613 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.613 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.613 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.613 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.613 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.613 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.613 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.614 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.614 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.614 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.614 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.614 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.614 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.614 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.614 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.615 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.615 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.615 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.615 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.615 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.615 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.615 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.616 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.616 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "status", 2023-11-16T16:55:05.616 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.616 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.616 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.616 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.616 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.616 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.617 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.617 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.617 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.617 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.617 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.617 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.617 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.617 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.618 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.618 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.618 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.618 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.618 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.618 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.618 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.619 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.619 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.619 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.619 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.619 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.619 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.619 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.619 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.620 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.620 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.620 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.620 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.620 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.620 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.620 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.620 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.621 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.621 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.621 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.621 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.621 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.621 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.621 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.621 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.622 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.622 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.622 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.622 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.622 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.622 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.622 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.622 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.623 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.623 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.623 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.623 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.623 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.623 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.623 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.623 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.624 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.624 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.624 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.624 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.624 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.624 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.624 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.625 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.625 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.625 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.625 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.625 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.625 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.625 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.625 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.626 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telegraf", 2023-11-16T16:55:05.626 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.626 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.626 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.626 INFO:teuthology.orchestra.run.smithi093.stdout: "address": { 2023-11-16T16:55:05.626 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "address", 2023-11-16T16:55:05.626 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.626 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.627 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.627 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-16T16:55:05.627 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.627 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.627 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.627 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.627 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.628 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.628 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.628 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.628 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:55:05.628 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:55:05.628 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.628 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.628 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.629 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15", 2023-11-16T16:55:05.629 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.629 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.629 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.629 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.629 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.629 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.630 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.630 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.630 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.630 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.630 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.630 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.630 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.630 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.631 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.631 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.631 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.631 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.631 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.631 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.631 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.631 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.632 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.632 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.632 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.632 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.632 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.632 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.632 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.632 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.633 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.633 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.633 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.633 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.633 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.633 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.633 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.633 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.634 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.634 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.634 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.634 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.634 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.634 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.634 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.635 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.635 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.635 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.635 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.635 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.635 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.635 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.635 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.636 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.636 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.636 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.636 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.636 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.636 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.636 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.636 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.637 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.637 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.637 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.637 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.637 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.637 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.637 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.637 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.638 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.638 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.638 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.638 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.638 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.638 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.638 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.638 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.639 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.639 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.639 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.639 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.639 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telemetry", 2023-11-16T16:55:05.639 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.639 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.640 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.640 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_basic": { 2023-11-16T16:55:05.640 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_basic", 2023-11-16T16:55:05.640 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.640 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.640 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.640 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.640 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.641 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.641 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.641 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-16T16:55:05.641 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.641 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.641 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.641 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.641 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_crash": { 2023-11-16T16:55:05.642 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_crash", 2023-11-16T16:55:05.642 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.642 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.642 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.642 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.642 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.642 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.642 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.643 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-16T16:55:05.643 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.643 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.643 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.643 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.643 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_device": { 2023-11-16T16:55:05.643 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_device", 2023-11-16T16:55:05.644 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.644 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.644 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.644 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.644 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.644 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.644 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.644 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-16T16:55:05.645 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.645 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.645 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.645 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.645 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_ident": { 2023-11-16T16:55:05.645 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_ident", 2023-11-16T16:55:05.645 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.645 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.646 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.646 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.646 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.646 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.646 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.646 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-16T16:55:05.646 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.647 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.647 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.647 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.647 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_perf": { 2023-11-16T16:55:05.647 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_perf", 2023-11-16T16:55:05.647 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.647 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.647 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.648 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.648 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.648 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.648 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.648 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-16T16:55:05.648 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.648 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.648 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.649 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.649 INFO:teuthology.orchestra.run.smithi093.stdout: "contact": { 2023-11-16T16:55:05.649 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "contact", 2023-11-16T16:55:05.649 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.649 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.649 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.649 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.649 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.650 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.650 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.650 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.650 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.650 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.650 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.650 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.651 INFO:teuthology.orchestra.run.smithi093.stdout: "description": { 2023-11-16T16:55:05.651 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "description", 2023-11-16T16:55:05.651 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.651 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.651 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.651 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.651 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.651 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.652 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.652 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.652 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.652 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.652 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.652 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.652 INFO:teuthology.orchestra.run.smithi093.stdout: "device_url": { 2023-11-16T16:55:05.652 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_url", 2023-11-16T16:55:05.653 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.653 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.653 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.653 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-16T16:55:05.653 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.653 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.653 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.653 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.654 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.654 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.654 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.654 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.654 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-11-16T16:55:05.654 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-11-16T16:55:05.654 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.654 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.655 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.655 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.655 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.655 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.655 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.655 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.655 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.655 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.656 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.656 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.656 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:55:05.656 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:55:05.656 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.656 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.656 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.656 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "24", 2023-11-16T16:55:05.657 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "8", 2023-11-16T16:55:05.657 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.657 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.657 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.657 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.657 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.657 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.657 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.658 INFO:teuthology.orchestra.run.smithi093.stdout: "last_opt_revision": { 2023-11-16T16:55:05.658 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "last_opt_revision", 2023-11-16T16:55:05.658 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.658 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.658 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.658 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1", 2023-11-16T16:55:05.658 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.659 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.659 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.659 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.659 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.659 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.659 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.659 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.659 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard": { 2023-11-16T16:55:05.660 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard", 2023-11-16T16:55:05.660 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.660 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.660 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.660 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.660 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.660 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.660 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.661 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.661 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.661 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.661 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.661 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.661 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard_description": { 2023-11-16T16:55:05.661 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard_description", 2023-11-16T16:55:05.662 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.662 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.662 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.662 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.662 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.662 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.662 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.662 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.663 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.663 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.663 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.663 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.663 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.663 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.663 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.663 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.664 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.664 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.664 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.664 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.664 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.664 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.664 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.664 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.665 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.665 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.665 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.665 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.665 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.665 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.665 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.665 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.666 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.666 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.666 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.666 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.666 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.666 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.666 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.667 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.667 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.667 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.667 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.667 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.667 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.667 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.667 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.668 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.668 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.668 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.668 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.668 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.668 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.668 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.668 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.669 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.669 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.669 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.669 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.669 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.669 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.669 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.669 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.670 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.670 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.670 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.670 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.670 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.670 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.670 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.671 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.671 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.671 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.671 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.671 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.671 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.671 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.671 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.672 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.672 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.672 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.672 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.672 INFO:teuthology.orchestra.run.smithi093.stdout: "organization": { 2023-11-16T16:55:05.672 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "organization", 2023-11-16T16:55:05.672 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.672 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.673 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.673 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.673 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.673 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.673 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.673 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.673 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.673 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.674 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.674 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.674 INFO:teuthology.orchestra.run.smithi093.stdout: "proxy": { 2023-11-16T16:55:05.674 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "proxy", 2023-11-16T16:55:05.674 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.674 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.674 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.674 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.675 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.675 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.675 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.675 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.675 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.675 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.675 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.676 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.676 INFO:teuthology.orchestra.run.smithi093.stdout: "url": { 2023-11-16T16:55:05.676 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url", 2023-11-16T16:55:05.676 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.676 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.676 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.676 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-16T16:55:05.676 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.677 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.677 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.677 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.677 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.677 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.677 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.677 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.677 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.678 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.678 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.678 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "test_orchestrator", 2023-11-16T16:55:05.678 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.678 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.678 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.678 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.678 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.679 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.679 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.679 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.679 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.679 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.679 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.679 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.679 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.680 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.680 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.680 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.680 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.680 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.680 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.680 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.681 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.681 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.681 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.681 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.681 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.681 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.681 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.681 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.682 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.682 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.682 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.682 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.682 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.682 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.682 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.682 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.683 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.683 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.683 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.683 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.683 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.683 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.683 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.684 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.684 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.684 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.684 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.684 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.684 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.684 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.684 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.685 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.685 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.685 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.685 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.685 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.685 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.685 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.685 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.686 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.686 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.686 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.686 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.686 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.686 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.686 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.686 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.687 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.687 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.687 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.687 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.687 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.687 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.687 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.687 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.688 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.688 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "volumes", 2023-11-16T16:55:05.688 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.688 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.688 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.688 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.688 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.689 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.689 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.689 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.689 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.689 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.689 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.689 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.689 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.690 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.690 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.690 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.690 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.690 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.690 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.690 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.690 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.691 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.691 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.691 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.691 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.691 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.691 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.691 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.691 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.692 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.692 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.692 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.692 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.692 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.692 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.692 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.692 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.692 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.693 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.693 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.693 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.693 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.693 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.694 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.694 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.694 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.694 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.694 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.694 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.694 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.694 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.695 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.695 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.695 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.695 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.695 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.695 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.695 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.695 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.696 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.696 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.696 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.696 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.696 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.696 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.696 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.696 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.697 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.697 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.697 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.697 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.697 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.697 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.697 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_clones": { 2023-11-16T16:55:05.697 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_clones", 2023-11-16T16:55:05.698 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.698 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.698 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.698 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4", 2023-11-16T16:55:05.698 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.698 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.698 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.699 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-16T16:55:05.699 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.699 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.699 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.699 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.699 INFO:teuthology.orchestra.run.smithi093.stdout: "snapshot_clone_delay": { 2023-11-16T16:55:05.699 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snapshot_clone_delay", 2023-11-16T16:55:05.699 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.700 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.700 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.700 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:55:05.700 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.700 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.700 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.700 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-16T16:55:05.700 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.701 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.701 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.701 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.701 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.701 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.701 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.701 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix", 2023-11-16T16:55:05.701 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.702 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.702 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.702 INFO:teuthology.orchestra.run.smithi093.stdout: "discovery_interval": { 2023-11-16T16:55:05.702 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "discovery_interval", 2023-11-16T16:55:05.702 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:55:05.702 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.702 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.702 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "100", 2023-11-16T16:55:05.703 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.703 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.703 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.703 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.703 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.703 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.703 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.704 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.704 INFO:teuthology.orchestra.run.smithi093.stdout: "identifier": { 2023-11-16T16:55:05.704 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "identifier", 2023-11-16T16:55:05.704 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.704 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.704 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.704 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.704 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.705 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.705 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.705 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.705 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.705 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.705 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.705 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.705 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:55:05.706 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:55:05.706 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.706 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.706 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.706 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:55:05.706 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.706 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.706 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.707 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.707 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.707 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.707 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.707 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.707 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.707 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.707 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.708 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.708 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.708 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.708 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.708 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.708 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.708 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.708 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.709 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.709 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.709 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.709 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.709 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.709 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.709 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.710 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.710 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.710 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.710 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.710 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.710 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.710 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.710 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.711 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.711 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.711 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.711 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.711 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.711 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.711 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.711 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.712 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.712 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.712 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.712 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.712 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.712 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.712 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.712 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.713 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.713 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.713 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.713 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.713 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.713 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.713 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.714 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.714 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.714 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.714 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.714 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.714 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.714 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.714 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.715 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.715 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.715 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.715 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.715 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.715 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.715 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.715 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.716 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.716 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.716 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.716 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.716 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.716 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_host": { 2023-11-16T16:55:05.716 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_host", 2023-11-16T16:55:05.717 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.717 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.717 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.717 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.717 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.717 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.717 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.717 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.718 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.718 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.718 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.718 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.718 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_port": { 2023-11-16T16:55:05.718 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_port", 2023-11-16T16:55:05.718 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.719 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.719 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.719 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10051", 2023-11-16T16:55:05.719 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.719 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.719 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.719 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.719 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.720 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.720 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.720 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.720 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_sender": { 2023-11-16T16:55:05.720 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_sender", 2023-11-16T16:55:05.720 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.720 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.720 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.721 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-16T16:55:05.721 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.721 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.721 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.721 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.721 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.721 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.721 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.722 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.722 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.722 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.722 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-11-16T16:55:05.722 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.722 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.722 INFO:teuthology.orchestra.run.smithi093.stdout: "modules": [ 2023-11-16T16:55:05.723 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm", 2023-11-16T16:55:05.723 INFO:teuthology.orchestra.run.smithi093.stdout: "dashboard", 2023-11-16T16:55:05.723 INFO:teuthology.orchestra.run.smithi093.stdout: "iostat", 2023-11-16T16:55:05.723 INFO:teuthology.orchestra.run.smithi093.stdout: "nfs", 2023-11-16T16:55:05.723 INFO:teuthology.orchestra.run.smithi093.stdout: "restful" 2023-11-16T16:55:05.723 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.723 INFO:teuthology.orchestra.run.smithi093.stdout: "available_modules": [ 2023-11-16T16:55:05.723 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.724 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alerts", 2023-11-16T16:55:05.724 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.724 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.724 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.724 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:55:05.724 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:55:05.724 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.724 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.725 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.725 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:55:05.725 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.725 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.725 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.725 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How frequently to reexamine health status", 2023-11-16T16:55:05.725 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.725 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.726 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.726 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.726 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.726 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.726 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.726 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.726 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.727 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.727 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.727 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.727 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.727 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.727 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.727 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.727 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.728 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.728 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.728 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.728 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.728 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.728 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.728 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.728 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.729 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.729 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.729 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.729 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.729 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.729 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.729 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.729 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.730 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.730 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.730 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.730 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.730 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.730 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.730 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.731 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.731 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.731 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.731 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.731 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.731 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.731 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.731 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.732 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.732 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.732 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.732 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.732 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.732 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.732 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.732 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.733 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.733 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.733 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.733 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.733 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.733 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.733 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.733 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.734 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.734 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.734 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.734 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.734 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.734 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.734 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.734 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.735 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.735 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.735 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_destination": { 2023-11-16T16:55:05.735 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_destination", 2023-11-16T16:55:05.735 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.735 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.735 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.735 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.736 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.736 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.736 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.736 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email address to send alerts to", 2023-11-16T16:55:05.736 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.736 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.736 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.736 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.737 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_from_name": { 2023-11-16T16:55:05.737 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_from_name", 2023-11-16T16:55:05.737 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.737 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.737 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.737 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "Ceph", 2023-11-16T16:55:05.737 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.737 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.738 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.738 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email From: name", 2023-11-16T16:55:05.738 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.738 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.738 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.738 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.738 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_host": { 2023-11-16T16:55:05.739 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_host", 2023-11-16T16:55:05.739 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.739 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.739 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.739 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.739 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.739 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.739 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.740 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP server", 2023-11-16T16:55:05.740 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.740 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.740 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.740 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.740 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_password": { 2023-11-16T16:55:05.740 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_password", 2023-11-16T16:55:05.740 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.741 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.741 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.741 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.741 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.741 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.741 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.741 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Password to authenticate with", 2023-11-16T16:55:05.741 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.742 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.742 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.742 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.742 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_port": { 2023-11-16T16:55:05.742 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_port", 2023-11-16T16:55:05.742 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.742 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.742 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.743 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "465", 2023-11-16T16:55:05.743 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.743 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.743 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.743 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP port", 2023-11-16T16:55:05.743 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.743 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.744 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.744 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.744 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_sender": { 2023-11-16T16:55:05.744 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_sender", 2023-11-16T16:55:05.744 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.744 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.744 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.744 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.745 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.745 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.745 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.745 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP envelope sender", 2023-11-16T16:55:05.745 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.745 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.745 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.745 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.746 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_ssl": { 2023-11-16T16:55:05.746 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_ssl", 2023-11-16T16:55:05.746 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.746 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.746 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.746 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.746 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.746 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.747 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.747 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-16T16:55:05.747 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.747 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.747 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.747 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.747 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_user": { 2023-11-16T16:55:05.747 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_user", 2023-11-16T16:55:05.748 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.748 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.748 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.748 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.748 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.748 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.748 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.748 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "User to authenticate as", 2023-11-16T16:55:05.749 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.749 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.749 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.749 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.749 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.749 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.749 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.749 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "balancer", 2023-11-16T16:55:05.750 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.750 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.750 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.750 INFO:teuthology.orchestra.run.smithi093.stdout: "active": { 2023-11-16T16:55:05.750 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "active", 2023-11-16T16:55:05.750 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.750 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.751 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.751 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.751 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.751 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.751 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.751 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically balance PGs across cluster", 2023-11-16T16:55:05.751 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.751 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.752 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.752 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.752 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_time": { 2023-11-16T16:55:05.752 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_time", 2023-11-16T16:55:05.752 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.752 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.752 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.752 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0000", 2023-11-16T16:55:05.752 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.753 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.753 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.753 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "beginning time of day to automatically balance", 2023-11-16T16:55:05.753 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-16T16:55:05.753 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.753 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.753 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.753 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_weekday": { 2023-11-16T16:55:05.754 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_weekday", 2023-11-16T16:55:05.754 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:55:05.754 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.754 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.754 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:55:05.754 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-11-16T16:55:05.754 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-11-16T16:55:05.755 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.755 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-16T16:55:05.755 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-16T16:55:05.755 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.755 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.755 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.755 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_max_iterations": { 2023-11-16T16:55:05.755 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_max_iterations", 2023-11-16T16:55:05.756 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:55:05.756 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.756 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.756 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "25", 2023-11-16T16:55:05.756 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-11-16T16:55:05.756 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "250", 2023-11-16T16:55:05.756 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.756 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-16T16:55:05.757 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.757 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.757 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.757 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.757 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_metrics": { 2023-11-16T16:55:05.757 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_metrics", 2023-11-16T16:55:05.757 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.757 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.758 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.758 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "pgs,objects,bytes", 2023-11-16T16:55:05.758 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.758 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.758 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.758 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-16T16:55:05.758 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-16T16:55:05.758 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.759 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.759 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.759 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_step": { 2023-11-16T16:55:05.759 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_step", 2023-11-16T16:55:05.759 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:55:05.759 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.759 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.759 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.5", 2023-11-16T16:55:05.760 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0.001", 2023-11-16T16:55:05.760 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "0.999", 2023-11-16T16:55:05.760 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.760 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "aggressiveness of optimization", 2023-11-16T16:55:05.760 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-16T16:55:05.760 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.760 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.760 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.761 INFO:teuthology.orchestra.run.smithi093.stdout: "end_time": { 2023-11-16T16:55:05.761 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_time", 2023-11-16T16:55:05.761 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.761 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.761 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.761 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2359", 2023-11-16T16:55:05.761 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.761 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.762 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.762 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ending time of day to automatically balance", 2023-11-16T16:55:05.762 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-16T16:55:05.762 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.762 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.762 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.762 INFO:teuthology.orchestra.run.smithi093.stdout: "end_weekday": { 2023-11-16T16:55:05.762 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_weekday", 2023-11-16T16:55:05.763 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:55:05.763 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.763 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.763 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:55:05.763 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-11-16T16:55:05.763 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-11-16T16:55:05.763 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.763 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-16T16:55:05.764 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-16T16:55:05.764 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.764 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.764 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.764 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.764 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.764 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.765 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.765 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.765 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.765 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.765 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.765 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.765 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.765 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.766 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.766 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.766 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.766 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.766 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.766 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.766 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.766 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.767 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.767 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.767 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.767 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.767 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.767 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.767 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.767 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.768 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.768 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.768 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.768 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.768 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.768 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.768 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.769 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.769 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.769 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.769 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.769 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.769 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.769 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.769 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.770 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.770 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.770 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.770 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.770 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.770 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.770 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.770 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.771 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.771 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.771 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.771 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.771 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.771 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.771 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.772 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.772 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.772 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.772 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.772 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.772 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.772 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.772 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.773 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.773 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.773 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.773 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.773 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.773 INFO:teuthology.orchestra.run.smithi093.stdout: "min_score": { 2023-11-16T16:55:05.773 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_score", 2023-11-16T16:55:05.773 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:55:05.774 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.774 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.774 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:55:05.774 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.774 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.774 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.774 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-16T16:55:05.775 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.775 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.775 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.775 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.775 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-11-16T16:55:05.775 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-11-16T16:55:05.775 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.775 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.776 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.776 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "upmap", 2023-11-16T16:55:05.776 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.776 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.776 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.776 INFO:teuthology.orchestra.run.smithi093.stdout: "crush-compat", 2023-11-16T16:55:05.776 INFO:teuthology.orchestra.run.smithi093.stdout: "none", 2023-11-16T16:55:05.776 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap" 2023-11-16T16:55:05.777 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.777 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Balancer mode", 2023-11-16T16:55:05.777 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.777 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.777 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.777 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.777 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_ids": { 2023-11-16T16:55:05.777 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_ids", 2023-11-16T16:55:05.778 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.778 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.778 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.778 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.778 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.778 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.778 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.779 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-16T16:55:05.779 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.779 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.779 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.779 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.779 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:55:05.779 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:55:05.779 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.780 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.780 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.780 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:55:05.780 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.780 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.780 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.780 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-16T16:55:05.780 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.781 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.781 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.781 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.781 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_deviation": { 2023-11-16T16:55:05.781 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_deviation", 2023-11-16T16:55:05.781 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.781 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.781 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.782 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-11-16T16:55:05.782 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-11-16T16:55:05.782 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.782 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.782 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-16T16:55:05.782 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-16T16:55:05.782 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.782 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.783 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.783 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_optimizations": { 2023-11-16T16:55:05.783 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_optimizations", 2023-11-16T16:55:05.783 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:55:05.783 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.783 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.783 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:55:05.784 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.784 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.784 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.784 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-16T16:55:05.784 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.784 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.784 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.784 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.784 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.785 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.785 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.785 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cephadm", 2023-11-16T16:55:05.785 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.785 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.785 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.785 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_down_multiplier": { 2023-11-16T16:55:05.785 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_down_multiplier", 2023-11-16T16:55:05.786 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:55:05.786 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.786 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.786 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-11-16T16:55:05.786 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.786 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.786 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.786 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-16T16:55:05.787 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.787 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.787 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.787 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.787 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_refresh_rate": { 2023-11-16T16:55:05.787 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_refresh_rate", 2023-11-16T16:55:05.787 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.787 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.788 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.788 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "20", 2023-11-16T16:55:05.788 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.788 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.788 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.788 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-16T16:55:05.788 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.788 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.788 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.789 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.789 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_starting_port": { 2023-11-16T16:55:05.789 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_starting_port", 2023-11-16T16:55:05.789 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.789 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.789 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.789 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4721", 2023-11-16T16:55:05.789 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.790 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.790 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.790 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-16T16:55:05.790 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.790 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.790 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.790 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.790 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_password": { 2023-11-16T16:55:05.791 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_password", 2023-11-16T16:55:05.791 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.791 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.791 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.791 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:55:05.791 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.791 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.791 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.792 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web password", 2023-11-16T16:55:05.792 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.792 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.792 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.792 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.792 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_user": { 2023-11-16T16:55:05.792 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_user", 2023-11-16T16:55:05.792 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.792 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.793 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.793 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:55:05.793 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.793 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.793 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.793 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web user", 2023-11-16T16:55:05.793 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.793 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.794 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.794 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.794 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_ptrace": { 2023-11-16T16:55:05.794 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_ptrace", 2023-11-16T16:55:05.794 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.794 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.794 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.794 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.795 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.795 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.795 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.795 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-16T16:55:05.795 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-16T16:55:05.795 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.795 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.795 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.796 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_interval": { 2023-11-16T16:55:05.796 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_interval", 2023-11-16T16:55:05.796 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.796 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.796 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.796 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:55:05.796 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.797 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.797 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.797 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-16T16:55:05.797 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.797 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.797 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.797 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.797 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_memory_target_ratio": { 2023-11-16T16:55:05.798 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_memory_target_ratio", 2023-11-16T16:55:05.798 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:55:05.798 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.798 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.798 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.7", 2023-11-16T16:55:05.798 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.798 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.799 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.799 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-16T16:55:05.799 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.799 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.799 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.799 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.799 INFO:teuthology.orchestra.run.smithi093.stdout: "cgroups_split": { 2023-11-16T16:55:05.799 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cgroups_split", 2023-11-16T16:55:05.800 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.800 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.800 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.800 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.800 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.800 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.800 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.800 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-16T16:55:05.801 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.801 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.801 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.801 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.801 INFO:teuthology.orchestra.run.smithi093.stdout: "config_checks_enabled": { 2023-11-16T16:55:05.801 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_checks_enabled", 2023-11-16T16:55:05.801 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.801 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.802 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.802 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.802 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.802 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.802 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.802 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-16T16:55:05.802 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.802 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.803 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.803 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.803 INFO:teuthology.orchestra.run.smithi093.stdout: "config_dashboard": { 2023-11-16T16:55:05.803 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_dashboard", 2023-11-16T16:55:05.803 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.803 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.803 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.803 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.804 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.804 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.804 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.804 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-16T16:55:05.804 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.804 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.804 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.805 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.805 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_alertmanager": { 2023-11-16T16:55:05.805 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_alertmanager", 2023-11-16T16:55:05.805 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.805 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.805 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.805 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-16T16:55:05.805 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.806 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.806 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.806 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-11-16T16:55:05.806 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.806 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.806 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.806 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.806 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_base": { 2023-11-16T16:55:05.807 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_base", 2023-11-16T16:55:05.807 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.807 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.807 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.807 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-16T16:55:05.807 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.807 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.807 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.808 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Container image name, without the tag", 2023-11-16T16:55:05.808 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.808 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.808 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.808 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.808 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_elasticsearch": { 2023-11-16T16:55:05.808 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_elasticsearch", 2023-11-16T16:55:05.809 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.809 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.809 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.809 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-16T16:55:05.809 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.809 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.809 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.809 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "elasticsearch container image", 2023-11-16T16:55:05.810 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.810 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.810 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.810 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.810 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_grafana": { 2023-11-16T16:55:05.810 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_grafana", 2023-11-16T16:55:05.810 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.811 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.811 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.811 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-16T16:55:05.811 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.811 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.811 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.811 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-11-16T16:55:05.811 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.812 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.812 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.812 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.812 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_haproxy": { 2023-11-16T16:55:05.812 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_haproxy", 2023-11-16T16:55:05.812 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.812 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.812 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.813 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-16T16:55:05.813 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.813 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.813 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.813 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "HAproxy container image", 2023-11-16T16:55:05.813 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.813 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.813 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.814 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.814 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_agent": { 2023-11-16T16:55:05.814 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_agent", 2023-11-16T16:55:05.814 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.814 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.814 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.814 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-16T16:55:05.814 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.815 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.815 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.815 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger agent container image", 2023-11-16T16:55:05.815 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.815 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.815 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.815 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.815 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_collector": { 2023-11-16T16:55:05.816 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_collector", 2023-11-16T16:55:05.816 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.816 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.816 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.816 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-16T16:55:05.816 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.816 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.817 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.817 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger collector container image", 2023-11-16T16:55:05.817 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.817 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.817 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.817 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.817 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_query": { 2023-11-16T16:55:05.817 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_query", 2023-11-16T16:55:05.818 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.818 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.818 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.818 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-16T16:55:05.818 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.818 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.818 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.818 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger query container image", 2023-11-16T16:55:05.819 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.819 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.819 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.819 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.819 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_keepalived": { 2023-11-16T16:55:05.819 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_keepalived", 2023-11-16T16:55:05.819 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.820 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.820 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.820 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-16T16:55:05.820 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.820 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.820 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.820 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Keepalived container image", 2023-11-16T16:55:05.820 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.821 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.821 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.821 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.821 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_loki": { 2023-11-16T16:55:05.821 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_loki", 2023-11-16T16:55:05.821 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.821 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.821 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.822 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-16T16:55:05.822 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.822 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.822 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.822 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Loki container image", 2023-11-16T16:55:05.822 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.822 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.823 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.823 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.823 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_node_exporter": { 2023-11-16T16:55:05.823 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_node_exporter", 2023-11-16T16:55:05.823 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.823 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.823 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.823 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-16T16:55:05.824 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.824 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.824 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.824 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-11-16T16:55:05.824 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.824 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.824 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.824 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.825 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_prometheus": { 2023-11-16T16:55:05.825 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_prometheus", 2023-11-16T16:55:05.825 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.825 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.825 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.825 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-16T16:55:05.825 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.826 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.826 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.826 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-11-16T16:55:05.826 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.826 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.826 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.826 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.826 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_promtail": { 2023-11-16T16:55:05.827 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_promtail", 2023-11-16T16:55:05.827 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.827 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.827 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.827 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-16T16:55:05.827 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.827 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.827 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.828 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Promtail container image", 2023-11-16T16:55:05.828 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.828 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.828 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.828 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.828 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_snmp_gateway": { 2023-11-16T16:55:05.828 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_snmp_gateway", 2023-11-16T16:55:05.828 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.829 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.829 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.829 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-16T16:55:05.829 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.829 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.829 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.829 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SNMP Gateway container image", 2023-11-16T16:55:05.830 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.830 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.830 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.830 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.830 INFO:teuthology.orchestra.run.smithi093.stdout: "container_init": { 2023-11-16T16:55:05.830 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_init", 2023-11-16T16:55:05.830 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.830 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.831 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.831 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.831 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.831 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.831 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.831 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Run podman/docker with `--init`", 2023-11-16T16:55:05.831 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.831 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.832 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.832 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.832 INFO:teuthology.orchestra.run.smithi093.stdout: "daemon_cache_timeout": { 2023-11-16T16:55:05.832 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "daemon_cache_timeout", 2023-11-16T16:55:05.832 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.832 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.832 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.832 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:55:05.833 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.833 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.833 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.833 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-16T16:55:05.833 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.833 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.833 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.833 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.834 INFO:teuthology.orchestra.run.smithi093.stdout: "default_cephadm_command_timeout": { 2023-11-16T16:55:05.834 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_cephadm_command_timeout", 2023-11-16T16:55:05.834 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.834 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.834 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.834 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "900", 2023-11-16T16:55:05.834 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.835 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.835 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.835 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-16T16:55:05.835 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.835 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.835 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.835 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.835 INFO:teuthology.orchestra.run.smithi093.stdout: "default_registry": { 2023-11-16T16:55:05.836 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_registry", 2023-11-16T16:55:05.836 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.836 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.836 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.836 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io", 2023-11-16T16:55:05.836 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.836 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.836 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.837 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-16T16:55:05.837 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.837 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.837 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.837 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.837 INFO:teuthology.orchestra.run.smithi093.stdout: "device_cache_timeout": { 2023-11-16T16:55:05.837 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_cache_timeout", 2023-11-16T16:55:05.837 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.838 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.838 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.838 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1800", 2023-11-16T16:55:05.838 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.838 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.838 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.838 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache device inventory", 2023-11-16T16:55:05.838 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.839 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.839 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.839 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.839 INFO:teuthology.orchestra.run.smithi093.stdout: "device_enhanced_scan": { 2023-11-16T16:55:05.839 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_enhanced_scan", 2023-11-16T16:55:05.839 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.839 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.839 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.840 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.840 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.840 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.840 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.840 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-16T16:55:05.840 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.840 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.841 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.841 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.841 INFO:teuthology.orchestra.run.smithi093.stdout: "facts_cache_timeout": { 2023-11-16T16:55:05.841 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "facts_cache_timeout", 2023-11-16T16:55:05.841 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.841 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.841 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.841 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:55:05.842 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.842 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.842 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.842 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache host facts data", 2023-11-16T16:55:05.842 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.842 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.842 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.842 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.843 INFO:teuthology.orchestra.run.smithi093.stdout: "host_check_interval": { 2023-11-16T16:55:05.843 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "host_check_interval", 2023-11-16T16:55:05.843 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.843 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.843 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.843 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:55:05.843 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.843 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.844 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.844 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to perform a host check", 2023-11-16T16:55:05.844 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.844 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.844 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.844 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.844 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.844 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.845 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.845 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.845 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.845 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.845 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.845 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.845 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.845 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.846 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.846 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.846 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.846 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.846 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.846 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.846 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.847 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.847 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.847 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.847 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.847 INFO:teuthology.orchestra.run.smithi093.stdout: "log_refresh_metadata": { 2023-11-16T16:55:05.847 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_refresh_metadata", 2023-11-16T16:55:05.847 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.847 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.848 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.848 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.848 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.848 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.848 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.848 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-16T16:55:05.848 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.848 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.849 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.849 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.849 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.849 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.849 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.849 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.849 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.850 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.850 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.850 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.850 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.850 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-16T16:55:05.850 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.850 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.850 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.851 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.851 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.851 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.851 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.851 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.851 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.851 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.851 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.852 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.852 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.852 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.852 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.852 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.852 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.852 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.852 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.853 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.853 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.853 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.853 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.853 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.853 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.853 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.853 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.854 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.854 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.854 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.854 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.854 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.854 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.854 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.855 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.855 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.855 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.855 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.855 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.855 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-16T16:55:05.855 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-16T16:55:05.855 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.856 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.856 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.856 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.856 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.856 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.856 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.856 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-16T16:55:05.856 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.857 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.857 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.857 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.857 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-16T16:55:05.857 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-16T16:55:05.857 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.857 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.857 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.858 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "*", 2023-11-16T16:55:05.858 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.858 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.858 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.858 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-16T16:55:05.858 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.858 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.858 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.859 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.859 INFO:teuthology.orchestra.run.smithi093.stdout: "max_count_per_host": { 2023-11-16T16:55:05.859 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_count_per_host", 2023-11-16T16:55:05.859 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.859 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.859 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.859 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:55:05.859 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.860 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.860 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.860 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of daemons per service per host", 2023-11-16T16:55:05.860 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.860 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.860 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.860 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.861 INFO:teuthology.orchestra.run.smithi093.stdout: "max_osd_draining_count": { 2023-11-16T16:55:05.861 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_osd_draining_count", 2023-11-16T16:55:05.861 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.861 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.861 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.861 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:55:05.861 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.861 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.862 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.862 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-16T16:55:05.862 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.862 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.862 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.862 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.862 INFO:teuthology.orchestra.run.smithi093.stdout: "migration_current": { 2023-11-16T16:55:05.862 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "migration_current", 2023-11-16T16:55:05.863 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.863 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.863 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.863 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.863 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.863 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.863 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.863 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "internal - do not modify", 2023-11-16T16:55:05.864 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.864 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.864 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.864 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.864 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-11-16T16:55:05.864 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-11-16T16:55:05.864 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.864 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.865 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.865 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "root", 2023-11-16T16:55:05.865 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.865 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.865 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.865 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm-package", 2023-11-16T16:55:05.865 INFO:teuthology.orchestra.run.smithi093.stdout: "root" 2023-11-16T16:55:05.865 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.866 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "mode for remote execution of cephadm", 2023-11-16T16:55:05.866 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.866 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.866 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.866 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.866 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_alerts_path": { 2023-11-16T16:55:05.866 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_alerts_path", 2023-11-16T16:55:05.867 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.867 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.867 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.867 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-16T16:55:05.867 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.867 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.867 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.867 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-16T16:55:05.868 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.868 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.868 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.868 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.868 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_password": { 2023-11-16T16:55:05.868 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_password", 2023-11-16T16:55:05.868 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.868 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.869 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.869 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:55:05.869 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.869 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.869 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.869 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web password", 2023-11-16T16:55:05.869 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.869 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.870 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.870 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.870 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_user": { 2023-11-16T16:55:05.870 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_user", 2023-11-16T16:55:05.870 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.870 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.870 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.871 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:55:05.871 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.871 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.871 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.871 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web user", 2023-11-16T16:55:05.871 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.871 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.871 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.872 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.872 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_insecure": { 2023-11-16T16:55:05.872 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_insecure", 2023-11-16T16:55:05.872 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.872 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.872 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.872 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.872 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.873 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.873 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.873 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-16T16:55:05.873 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.873 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.873 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.873 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.873 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_password": { 2023-11-16T16:55:05.874 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_password", 2023-11-16T16:55:05.874 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.874 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.874 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.874 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.874 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.874 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.875 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.875 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-16T16:55:05.875 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.875 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.875 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.875 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.875 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_url": { 2023-11-16T16:55:05.876 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_url", 2023-11-16T16:55:05.876 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.876 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.876 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.876 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.876 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.876 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.876 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.877 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-16T16:55:05.877 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.877 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.877 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.877 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.877 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_username": { 2023-11-16T16:55:05.877 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_username", 2023-11-16T16:55:05.877 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.878 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.878 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.878 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.878 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.878 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.878 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.878 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-16T16:55:05.879 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.879 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.879 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.879 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.879 INFO:teuthology.orchestra.run.smithi093.stdout: "secure_monitoring_stack": { 2023-11-16T16:55:05.879 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "secure_monitoring_stack", 2023-11-16T16:55:05.879 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.879 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.880 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.880 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.880 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.880 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.880 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.880 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-16T16:55:05.880 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.880 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.881 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.881 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.881 INFO:teuthology.orchestra.run.smithi093.stdout: "service_discovery_port": { 2023-11-16T16:55:05.881 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "service_discovery_port", 2023-11-16T16:55:05.881 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.881 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.881 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.881 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8765", 2023-11-16T16:55:05.882 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.882 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.882 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.882 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "cephadm service discovery port", 2023-11-16T16:55:05.882 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.882 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.882 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.882 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.883 INFO:teuthology.orchestra.run.smithi093.stdout: "ssh_config_file": { 2023-11-16T16:55:05.883 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssh_config_file", 2023-11-16T16:55:05.883 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.883 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.883 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.883 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.883 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.884 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.884 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.884 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-16T16:55:05.884 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.884 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.884 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.884 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.884 INFO:teuthology.orchestra.run.smithi093.stdout: "use_agent": { 2023-11-16T16:55:05.885 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_agent", 2023-11-16T16:55:05.885 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.885 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.885 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.885 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.885 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.885 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.885 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.886 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-16T16:55:05.886 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.886 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.886 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.886 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.886 INFO:teuthology.orchestra.run.smithi093.stdout: "use_repo_digest": { 2023-11-16T16:55:05.886 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_repo_digest", 2023-11-16T16:55:05.886 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.887 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.887 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.887 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.887 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.887 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.887 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.887 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-16T16:55:05.887 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.888 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.888 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.888 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.888 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_failed_host_check": { 2023-11-16T16:55:05.888 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_failed_host_check", 2023-11-16T16:55:05.888 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.888 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.889 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.889 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.889 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.889 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.889 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.889 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if the host check fails", 2023-11-16T16:55:05.889 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.889 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.890 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.890 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.890 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_daemons": { 2023-11-16T16:55:05.890 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_daemons", 2023-11-16T16:55:05.890 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.890 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.890 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.891 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.891 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.891 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.891 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.891 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-16T16:55:05.891 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.891 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.891 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.892 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.892 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_hosts": { 2023-11-16T16:55:05.892 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_hosts", 2023-11-16T16:55:05.892 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.892 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.892 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.892 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.892 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.893 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.893 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.893 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-16T16:55:05.893 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.893 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.893 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.893 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.893 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.894 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.894 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.894 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crash", 2023-11-16T16:55:05.894 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.894 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.894 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.894 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:05.894 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:05.897 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.897 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.898 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.898 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.898 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.898 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.898 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.898 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.899 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.899 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.899 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.899 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.899 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.899 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.899 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.900 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.900 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.900 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.900 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.900 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:05.900 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:05.900 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.901 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.901 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.901 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.901 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.901 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.901 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.901 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.901 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.902 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.902 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.902 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.902 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:05.902 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:05.902 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.902 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.903 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.903 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:05.903 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.903 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.903 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:05.903 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:05.903 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:05.903 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:05.904 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:05.904 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:05.904 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:05.904 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:05.904 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.904 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.904 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.905 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.905 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.905 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:05.905 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:05.905 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.905 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.905 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.906 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.906 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.906 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.906 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.906 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.906 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.906 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.906 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.907 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.907 INFO:teuthology.orchestra.run.smithi093.stdout: "retain_interval": { 2023-11-16T16:55:05.907 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retain_interval", 2023-11-16T16:55:05.907 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.907 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.907 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.907 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "31536000", 2023-11-16T16:55:05.908 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.908 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.908 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.908 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-16T16:55:05.908 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.908 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.908 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.909 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.909 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_recent_interval": { 2023-11-16T16:55:05.909 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_recent_interval", 2023-11-16T16:55:05.909 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:05.909 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.909 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:05.909 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1209600", 2023-11-16T16:55:05.910 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.910 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.910 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.910 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-16T16:55:05.910 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.910 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.911 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.911 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.911 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:05.911 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.911 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:05.911 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dashboard", 2023-11-16T16:55:05.911 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:05.912 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:05.912 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:05.912 INFO:teuthology.orchestra.run.smithi093.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-16T16:55:05.912 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-16T16:55:05.912 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.912 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.912 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.913 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:55:05.913 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.913 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.913 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.913 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.913 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.913 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.914 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.914 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.914 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_HOST": { 2023-11-16T16:55:05.914 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-16T16:55:05.914 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.914 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.914 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.915 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.915 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.915 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.915 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.915 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.915 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.915 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.916 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.916 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.916 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-16T16:55:05.916 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-16T16:55:05.916 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.916 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.916 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.917 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.917 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.917 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.917 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.917 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.917 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.917 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.917 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.918 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.918 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_ENABLED": { 2023-11-16T16:55:05.918 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_ENABLED", 2023-11-16T16:55:05.918 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.918 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.918 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.918 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.919 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.919 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.919 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.919 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.919 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.919 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.919 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.920 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.920 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-16T16:55:05.920 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-16T16:55:05.920 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.920 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.920 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.920 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.921 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.921 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.921 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.921 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.921 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.921 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.921 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.921 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.922 INFO:teuthology.orchestra.run.smithi093.stdout: "ENABLE_BROWSABLE_API": { 2023-11-16T16:55:05.922 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-16T16:55:05.922 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.922 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.922 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.922 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.922 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.923 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.923 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.923 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.923 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.923 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.923 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.923 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.923 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-16T16:55:05.924 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-16T16:55:05.924 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.924 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.924 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.924 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.924 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.925 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.925 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.925 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.925 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.925 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.925 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.925 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.925 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-16T16:55:05.926 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-16T16:55:05.926 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.926 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.926 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.926 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.926 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.926 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.927 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.927 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.927 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.927 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.927 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.927 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.927 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-16T16:55:05.927 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-16T16:55:05.928 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.928 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.928 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.928 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.928 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.928 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.928 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.928 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.929 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.929 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.929 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.929 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.929 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-16T16:55:05.929 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-16T16:55:05.930 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.930 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.930 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.930 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.930 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.930 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.930 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.930 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.931 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.931 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.931 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.931 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.931 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-16T16:55:05.931 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-16T16:55:05.931 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.932 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.932 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.932 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.932 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.932 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.932 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.932 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.932 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.933 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.933 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.933 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.933 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-16T16:55:05.933 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-16T16:55:05.933 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.933 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.934 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.934 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.934 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.934 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.934 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.934 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.934 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.934 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.935 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.935 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.935 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-16T16:55:05.935 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-16T16:55:05.935 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.935 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.935 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.936 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.936 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.936 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.936 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.936 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.936 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.936 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.936 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.937 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.937 INFO:teuthology.orchestra.run.smithi093.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-16T16:55:05.937 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-16T16:55:05.937 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.937 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.937 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.937 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.938 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.938 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.938 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.938 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.938 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.938 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.938 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.939 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.939 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_PASSWORD": { 2023-11-16T16:55:05.939 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-16T16:55:05.939 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.939 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.939 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.939 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:55:05.940 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.940 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.940 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.940 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.940 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.940 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.940 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.940 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.941 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-16T16:55:05.941 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-16T16:55:05.941 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.941 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.941 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.941 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.941 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.942 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.942 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.942 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.942 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.942 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.942 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.942 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.943 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_URL": { 2023-11-16T16:55:05.943 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_URL", 2023-11-16T16:55:05.943 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.943 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.943 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.943 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.943 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.943 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.944 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.944 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.944 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.944 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.944 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.944 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.944 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_USERNAME": { 2023-11-16T16:55:05.945 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-16T16:55:05.945 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.945 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.945 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.945 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:55:05.945 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.945 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.945 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.946 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.946 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.946 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.946 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.946 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.946 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-16T16:55:05.946 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-16T16:55:05.947 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.947 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.947 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.947 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.947 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.947 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.947 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.947 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.948 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.948 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.948 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.948 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.948 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-16T16:55:05.948 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-16T16:55:05.948 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.949 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.949 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.949 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.949 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.949 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.949 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.949 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.950 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.950 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.950 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.950 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.950 INFO:teuthology.orchestra.run.smithi093.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-16T16:55:05.950 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-16T16:55:05.950 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.951 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.951 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.951 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.951 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.951 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.951 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.951 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.951 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.952 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.952 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.952 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.952 INFO:teuthology.orchestra.run.smithi093.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-16T16:55:05.952 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-16T16:55:05.952 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.952 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.953 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.953 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.953 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.953 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.953 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.953 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.953 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.953 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.954 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.954 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.954 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_HOST": { 2023-11-16T16:55:05.954 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-16T16:55:05.954 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.954 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.954 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.955 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.955 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.955 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.955 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.955 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.955 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.955 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.956 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.956 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.956 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-16T16:55:05.956 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-16T16:55:05.956 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.956 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.956 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.957 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.957 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.957 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.957 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.957 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.957 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.957 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.957 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.958 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.958 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-16T16:55:05.958 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-16T16:55:05.958 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.958 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.958 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.959 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.959 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.959 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.959 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.959 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.959 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.960 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.960 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.960 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.960 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-16T16:55:05.960 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-16T16:55:05.960 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.960 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.961 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.961 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.961 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.961 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.961 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.961 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.961 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.962 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.962 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.962 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.962 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-16T16:55:05.962 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-16T16:55:05.962 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.962 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.963 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.963 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.963 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.963 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.963 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.963 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.963 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.963 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.964 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.964 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.964 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-16T16:55:05.964 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-16T16:55:05.964 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.964 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.964 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.965 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.965 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.965 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.965 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.965 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.965 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.965 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.965 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.966 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.966 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-16T16:55:05.966 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-16T16:55:05.966 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.966 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.966 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.967 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.967 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.967 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.967 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.967 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.967 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.967 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.968 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.968 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.968 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-16T16:55:05.968 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-16T16:55:05.968 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.968 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.968 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.968 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.969 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.969 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.969 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.969 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.969 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.969 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.969 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.970 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.970 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-16T16:55:05.970 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-16T16:55:05.970 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.970 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.970 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.971 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:05.971 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.971 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.971 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.971 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.971 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.971 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.971 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.972 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.972 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_ENABLED": { 2023-11-16T16:55:05.972 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-16T16:55:05.972 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.972 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.972 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.972 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.973 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.973 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.973 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.973 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.973 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.973 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.973 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.973 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.974 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-16T16:55:05.974 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-16T16:55:05.974 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.974 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.974 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.974 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-16T16:55:05.974 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.975 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.975 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.975 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.975 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.975 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.975 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.975 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.975 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-16T16:55:05.976 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-16T16:55:05.976 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.976 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.976 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.976 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:55:05.976 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.976 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.977 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.977 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.977 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.977 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.977 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.977 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.977 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-16T16:55:05.977 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-16T16:55:05.978 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.978 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.978 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.978 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8", 2023-11-16T16:55:05.978 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.979 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.979 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.979 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.979 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.979 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.980 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.980 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.980 INFO:teuthology.orchestra.run.smithi093.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-16T16:55:05.980 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-16T16:55:05.980 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.981 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.981 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.981 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "45", 2023-11-16T16:55:05.981 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.981 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.981 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.982 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.982 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.982 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.982 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.982 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.983 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ACCESS_KEY": { 2023-11-16T16:55:05.983 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-16T16:55:05.983 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.983 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.983 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.983 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.984 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.984 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.984 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.984 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.984 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.984 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.985 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.985 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.985 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-16T16:55:05.985 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-16T16:55:05.985 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.986 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.986 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.986 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-11-16T16:55:05.986 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.986 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.987 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.987 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.987 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.987 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.987 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.987 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.988 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SECRET_KEY": { 2023-11-16T16:55:05.988 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-16T16:55:05.988 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.988 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.988 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.988 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.988 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.989 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.989 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.989 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.989 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.989 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.989 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.989 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.990 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SSL_VERIFY": { 2023-11-16T16:55:05.990 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-16T16:55:05.990 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:05.990 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.990 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.990 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:05.991 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.991 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.991 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.991 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.991 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.991 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.991 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.991 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.992 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-16T16:55:05.992 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-16T16:55:05.992 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.992 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.992 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.992 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:55:05.992 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.992 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.993 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.993 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.993 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.993 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.993 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.993 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.993 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-16T16:55:05.994 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-16T16:55:05.994 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.994 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.994 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.994 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:55:05.994 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.994 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.995 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.995 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.995 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.995 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.995 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.995 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.995 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-16T16:55:05.995 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-16T16:55:05.996 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:05.996 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.996 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.996 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-11-16T16:55:05.996 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.996 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.996 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.997 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.997 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.997 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.997 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.997 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.997 INFO:teuthology.orchestra.run.smithi093.stdout: "cross_origin_url": { 2023-11-16T16:55:05.997 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cross_origin_url", 2023-11-16T16:55:05.997 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:05.998 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:05.998 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:05.998 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:05.998 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:05.998 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:05.998 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:05.998 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:05.999 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:05.999 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:05.999 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:05.999 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:05.999 INFO:teuthology.orchestra.run.smithi093.stdout: "crt_file": { 2023-11-16T16:55:05.999 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crt_file", 2023-11-16T16:55:06.000 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.000 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.000 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.000 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.000 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.000 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.000 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.000 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.001 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.001 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.001 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.001 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.001 INFO:teuthology.orchestra.run.smithi093.stdout: "debug": { 2023-11-16T16:55:06.001 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "debug", 2023-11-16T16:55:06.001 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.002 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.002 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.002 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.002 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.002 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.002 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.002 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable/disable debug options", 2023-11-16T16:55:06.003 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.003 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.003 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.003 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.003 INFO:teuthology.orchestra.run.smithi093.stdout: "jwt_token_ttl": { 2023-11-16T16:55:06.003 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "jwt_token_ttl", 2023-11-16T16:55:06.003 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.004 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.004 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.004 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "28800", 2023-11-16T16:55:06.004 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.004 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.004 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.004 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.004 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.005 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.005 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.005 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.005 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-11-16T16:55:06.005 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-11-16T16:55:06.005 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.005 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.006 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.006 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.006 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.006 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.006 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.006 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.006 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.007 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.007 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.007 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.007 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.007 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.007 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.007 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.007 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.008 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.008 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.008 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.008 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.008 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.008 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.008 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.009 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.009 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.009 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.009 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.009 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.009 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.009 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.010 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.010 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.010 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.010 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.010 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.010 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.010 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.011 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.011 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.011 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.011 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.011 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.011 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.011 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.011 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.012 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.012 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.012 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.012 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.012 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.012 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.012 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.013 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.013 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.013 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.013 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.013 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.013 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.013 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.013 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.014 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.014 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.014 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.014 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.014 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.014 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.015 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.015 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.015 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.015 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.015 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.015 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.015 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.015 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.016 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.016 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.016 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.016 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.016 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.016 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.016 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.016 INFO:teuthology.orchestra.run.smithi093.stdout: "motd": { 2023-11-16T16:55:06.017 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "motd", 2023-11-16T16:55:06.017 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.017 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.017 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.017 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.017 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.017 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.018 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.018 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "The message of the day", 2023-11-16T16:55:06.018 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.018 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.018 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.018 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.018 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect_resolve_ip_addr": { 2023-11-16T16:55:06.019 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "redirect_resolve_ip_addr", 2023-11-16T16:55:06.019 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.019 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.019 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.019 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.019 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.019 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.019 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.020 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.020 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.020 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.020 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.020 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.020 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-11-16T16:55:06.020 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-11-16T16:55:06.021 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.021 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.021 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.021 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-11-16T16:55:06.021 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.021 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.021 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.022 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.022 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.022 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.022 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.022 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.022 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-11-16T16:55:06.022 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-11-16T16:55:06.022 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.023 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.023 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.023 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8080", 2023-11-16T16:55:06.023 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.023 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.023 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.023 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.024 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.024 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.024 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.024 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.024 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-11-16T16:55:06.024 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-11-16T16:55:06.024 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.024 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.025 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.025 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:06.025 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.025 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.025 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.025 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.025 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.026 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.026 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.026 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.026 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl_server_port": { 2023-11-16T16:55:06.026 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl_server_port", 2023-11-16T16:55:06.026 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.026 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.027 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.027 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8443", 2023-11-16T16:55:06.027 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.027 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.027 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.027 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.027 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.027 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.028 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.028 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.028 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-11-16T16:55:06.028 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-11-16T16:55:06.028 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.028 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.028 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.029 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "redirect", 2023-11-16T16:55:06.029 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.029 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.029 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.029 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.029 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect" 2023-11-16T16:55:06.029 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.029 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.030 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.030 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.030 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.030 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.030 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-11-16T16:55:06.030 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-11-16T16:55:06.030 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.031 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.031 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.031 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-11-16T16:55:06.031 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-11-16T16:55:06.031 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-11-16T16:55:06.031 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.031 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.032 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.032 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.032 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.032 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.032 INFO:teuthology.orchestra.run.smithi093.stdout: "url_prefix": { 2023-11-16T16:55:06.032 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url_prefix", 2023-11-16T16:55:06.032 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.032 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.033 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.033 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.033 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.033 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.033 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.033 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.033 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.034 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.034 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.034 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.034 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.034 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.034 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.034 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "devicehealth", 2023-11-16T16:55:06.035 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.035 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.035 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.035 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_monitoring": { 2023-11-16T16:55:06.035 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_monitoring", 2023-11-16T16:55:06.035 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.035 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.035 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.036 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:06.036 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.036 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.036 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.036 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "monitor device health metrics", 2023-11-16T16:55:06.036 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.036 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.036 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.037 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.037 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.037 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.037 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.037 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.037 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.037 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.038 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.038 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.038 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.038 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.038 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.038 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.038 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.038 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.039 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.039 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.039 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.039 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.039 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.039 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.039 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.040 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.040 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.040 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.040 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.040 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.040 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.040 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.041 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.041 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.041 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.041 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.041 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.041 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.041 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.041 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.042 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.042 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.042 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.042 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.042 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.042 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.042 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.043 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.043 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.043 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.043 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.043 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.043 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.043 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.044 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.044 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.044 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.044 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.044 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.044 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.044 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.044 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.045 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.045 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.045 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.045 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.045 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.045 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.045 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.046 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.046 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.046 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.046 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.046 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.046 INFO:teuthology.orchestra.run.smithi093.stdout: "mark_out_threshold": { 2023-11-16T16:55:06.046 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mark_out_threshold", 2023-11-16T16:55:06.046 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:06.047 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.047 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.047 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2419200", 2023-11-16T16:55:06.047 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.047 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.047 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.047 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-16T16:55:06.048 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.048 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.048 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.048 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.048 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_name": { 2023-11-16T16:55:06.048 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_name", 2023-11-16T16:55:06.048 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.048 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.049 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.049 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "device_health_metrics", 2023-11-16T16:55:06.049 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.049 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.049 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.049 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-16T16:55:06.049 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.050 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.050 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.050 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.050 INFO:teuthology.orchestra.run.smithi093.stdout: "retention_period": { 2023-11-16T16:55:06.050 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retention_period", 2023-11-16T16:55:06.050 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:06.050 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.051 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.051 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15552000", 2023-11-16T16:55:06.051 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.051 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.051 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.051 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain device health metrics", 2023-11-16T16:55:06.051 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.052 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.052 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.052 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.052 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_frequency": { 2023-11-16T16:55:06.052 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_frequency", 2023-11-16T16:55:06.052 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:06.052 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.052 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.053 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-11-16T16:55:06.053 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.053 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.053 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.053 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-16T16:55:06.053 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.053 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.054 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.054 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.054 INFO:teuthology.orchestra.run.smithi093.stdout: "self_heal": { 2023-11-16T16:55:06.054 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "self_heal", 2023-11-16T16:55:06.054 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.054 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.054 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.054 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:06.055 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.055 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.055 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.055 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-16T16:55:06.055 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.055 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.055 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.056 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.056 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:55:06.056 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:55:06.056 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:06.056 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.056 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.056 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:55:06.056 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.057 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.057 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.057 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and check device health", 2023-11-16T16:55:06.057 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.057 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.057 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.057 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.058 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_threshold": { 2023-11-16T16:55:06.058 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_threshold", 2023-11-16T16:55:06.058 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:06.058 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.058 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.058 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7257600", 2023-11-16T16:55:06.058 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.059 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.059 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.059 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-16T16:55:06.059 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.059 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.059 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.059 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.060 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.060 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.060 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.060 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "diskprediction_local", 2023-11-16T16:55:06.060 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.060 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.060 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.061 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.061 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.061 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.061 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.061 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.061 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.061 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.061 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.062 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.062 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.062 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.062 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.062 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.062 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.062 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.063 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.063 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.063 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.063 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.063 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.063 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.063 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.063 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.064 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.064 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.064 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.064 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.064 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.064 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.064 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.065 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.065 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.065 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.065 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.065 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.065 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.065 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.066 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.066 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.066 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.066 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.066 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.066 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.066 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.067 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.067 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.067 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.067 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.067 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.067 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.067 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.068 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.068 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.068 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.068 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.068 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.068 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.068 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.069 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.069 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.069 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.069 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.069 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.069 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.070 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.070 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.070 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.070 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.070 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.070 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.070 INFO:teuthology.orchestra.run.smithi093.stdout: "predict_interval": { 2023-11-16T16:55:06.071 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predict_interval", 2023-11-16T16:55:06.071 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.071 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.071 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.071 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-11-16T16:55:06.071 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.071 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.072 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.072 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.072 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.072 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.072 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.072 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.072 INFO:teuthology.orchestra.run.smithi093.stdout: "predictor_model": { 2023-11-16T16:55:06.073 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predictor_model", 2023-11-16T16:55:06.073 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.073 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.073 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.073 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "prophetstor", 2023-11-16T16:55:06.073 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.073 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.073 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.074 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.074 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.074 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.074 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.074 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.074 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:55:06.074 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:55:06.075 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.075 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.075 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.075 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-11-16T16:55:06.075 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.075 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.075 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.075 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.076 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.076 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.076 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.076 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.076 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.076 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.076 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.077 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "influx", 2023-11-16T16:55:06.077 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": false, 2023-11-16T16:55:06.077 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "influxdb python module not found", 2023-11-16T16:55:06.077 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.077 INFO:teuthology.orchestra.run.smithi093.stdout: "batch_size": { 2023-11-16T16:55:06.077 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "batch_size", 2023-11-16T16:55:06.077 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.077 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.078 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.078 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5000", 2023-11-16T16:55:06.078 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.078 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.078 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.078 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-16T16:55:06.079 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.079 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.079 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.079 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.079 INFO:teuthology.orchestra.run.smithi093.stdout: "database": { 2023-11-16T16:55:06.079 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "database", 2023-11-16T16:55:06.079 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.080 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.080 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.080 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "ceph", 2023-11-16T16:55:06.080 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.080 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.080 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.080 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-16T16:55:06.081 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.081 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.081 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.081 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.081 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": { 2023-11-16T16:55:06.081 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "hostname", 2023-11-16T16:55:06.081 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.081 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.082 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.082 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.082 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.082 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.082 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.082 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server hostname", 2023-11-16T16:55:06.082 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.083 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.083 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.083 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.083 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:55:06.083 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:55:06.083 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:06.083 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.083 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.084 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "30", 2023-11-16T16:55:06.084 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "5", 2023-11-16T16:55:06.084 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.084 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.084 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-16T16:55:06.084 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.084 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.084 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.085 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.085 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.085 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.085 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.085 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.085 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.085 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.086 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.086 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.086 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.086 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.086 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.086 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.086 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.086 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.087 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.087 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.087 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.087 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.087 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.087 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.087 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.088 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.088 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.088 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.088 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.088 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.088 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.088 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.088 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.089 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.089 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.089 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.089 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.089 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.089 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.090 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.090 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.090 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.090 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.090 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.090 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.090 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.091 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.091 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.091 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.091 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.091 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.091 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.091 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.092 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.092 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.092 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.092 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.092 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.092 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.093 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.093 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.093 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.093 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.093 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.093 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.093 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.094 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.094 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.094 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.094 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.094 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.094 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.095 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.095 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.095 INFO:teuthology.orchestra.run.smithi093.stdout: "password": { 2023-11-16T16:55:06.095 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "password", 2023-11-16T16:55:06.095 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.095 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.095 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.096 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.096 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.096 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.096 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.096 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "password of InfluxDB server user", 2023-11-16T16:55:06.096 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.096 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.097 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.097 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.097 INFO:teuthology.orchestra.run.smithi093.stdout: "port": { 2023-11-16T16:55:06.097 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "port", 2023-11-16T16:55:06.097 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.097 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.097 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.098 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8086", 2023-11-16T16:55:06.098 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.101 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.101 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.101 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server port", 2023-11-16T16:55:06.101 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.101 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.102 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.102 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.102 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-11-16T16:55:06.102 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-11-16T16:55:06.102 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.102 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.102 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.103 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "false", 2023-11-16T16:55:06.103 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.103 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.103 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.103 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-16T16:55:06.103 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.103 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.104 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.104 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.104 INFO:teuthology.orchestra.run.smithi093.stdout: "threads": { 2023-11-16T16:55:06.104 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threads", 2023-11-16T16:55:06.104 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.104 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.104 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.104 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-11-16T16:55:06.105 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-11-16T16:55:06.105 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "32", 2023-11-16T16:55:06.105 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.105 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-16T16:55:06.105 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.105 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.105 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.106 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.106 INFO:teuthology.orchestra.run.smithi093.stdout: "username": { 2023-11-16T16:55:06.106 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "username", 2023-11-16T16:55:06.106 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.106 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.106 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.106 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.107 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.107 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.107 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.107 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "username of InfluxDB server user", 2023-11-16T16:55:06.107 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.107 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.107 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.107 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.108 INFO:teuthology.orchestra.run.smithi093.stdout: "verify_ssl": { 2023-11-16T16:55:06.108 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "verify_ssl", 2023-11-16T16:55:06.108 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.108 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.108 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.108 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "true", 2023-11-16T16:55:06.108 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.109 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.109 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.109 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-16T16:55:06.109 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.109 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.109 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.110 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.110 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.110 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.110 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.110 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "insights", 2023-11-16T16:55:06.110 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.110 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.110 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.111 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.111 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.111 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.111 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.111 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.111 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.111 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.112 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.112 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.112 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.112 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.112 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.112 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.112 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.112 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.113 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.113 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.113 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.113 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.113 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.113 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.113 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.114 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.114 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.114 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.114 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.114 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.114 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.114 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.114 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.115 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.115 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.115 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.115 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.115 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.115 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.115 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.116 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.116 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.116 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.116 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.116 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.116 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.116 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.116 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.117 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.117 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.117 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.117 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.117 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.117 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.117 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.118 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.118 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.118 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.118 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.118 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.118 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.118 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.119 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.119 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.119 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.119 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.119 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.119 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.119 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.120 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.120 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.120 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.120 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.120 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.120 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.120 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.121 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "iostat", 2023-11-16T16:55:06.121 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.121 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.121 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.121 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.121 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.121 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.121 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.122 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.122 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.122 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.122 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.122 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.122 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.122 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.123 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.123 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.123 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.123 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.123 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.123 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.123 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.123 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.124 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.124 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.124 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.124 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.124 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.124 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.124 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.125 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.125 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.125 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.125 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.125 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.125 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.125 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.125 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.126 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.126 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.126 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.126 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.126 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.126 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.126 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.127 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.127 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.127 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.127 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.127 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.127 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.127 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.127 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.128 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.128 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.128 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.128 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.128 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.128 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.128 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.129 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.129 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.129 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.129 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.129 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.129 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.129 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.130 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.130 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.130 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.130 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.130 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.130 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.130 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.131 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.131 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.131 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.131 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "k8sevents", 2023-11-16T16:55:06.131 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.131 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.131 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.132 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_event_retention_days": { 2023-11-16T16:55:06.132 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ceph_event_retention_days", 2023-11-16T16:55:06.132 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.132 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.132 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.132 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7", 2023-11-16T16:55:06.132 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.133 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.133 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.133 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-16T16:55:06.133 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.133 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.133 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.133 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.133 INFO:teuthology.orchestra.run.smithi093.stdout: "config_check_secs": { 2023-11-16T16:55:06.134 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_check_secs", 2023-11-16T16:55:06.134 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.134 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.134 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.134 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:55:06.134 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "10", 2023-11-16T16:55:06.134 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.135 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.135 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-16T16:55:06.135 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.135 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.135 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.135 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.135 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.135 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.136 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.136 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.136 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.136 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.136 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.136 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.136 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.137 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.137 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.137 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.137 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.137 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.137 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.137 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.137 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.138 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.138 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.138 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.138 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.138 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.138 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.138 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.139 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.139 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.139 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.139 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.139 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.139 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.140 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.140 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.140 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.140 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.140 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.140 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.140 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.140 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.141 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.141 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.141 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.141 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.141 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.141 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.141 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.141 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.142 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.142 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.142 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.142 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.142 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.142 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.142 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.143 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.143 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.143 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.143 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.143 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.143 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.143 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.143 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.144 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.144 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.144 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.144 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.144 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.144 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.144 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.145 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.145 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.145 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.145 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.145 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.145 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "localpool", 2023-11-16T16:55:06.145 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.145 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.146 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.146 INFO:teuthology.orchestra.run.smithi093.stdout: "failure_domain": { 2023-11-16T16:55:06.146 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "failure_domain", 2023-11-16T16:55:06.146 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.146 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.146 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.146 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "host", 2023-11-16T16:55:06.147 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.147 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.147 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.147 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "failure domain for any created local pool", 2023-11-16T16:55:06.147 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-16T16:55:06.147 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.147 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.147 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.148 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.148 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.148 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.148 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.148 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.148 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.148 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.149 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.149 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.149 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.149 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.149 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.149 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.149 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.150 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.150 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.150 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.150 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.150 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.150 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.150 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.151 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.151 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.151 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.151 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.151 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.151 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.151 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.152 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.152 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.152 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.152 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.152 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.152 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.152 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.152 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.153 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.153 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.153 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.153 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.153 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.153 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.153 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.153 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.154 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.154 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.154 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.154 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.154 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.154 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.154 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.155 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.155 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.155 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.155 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.155 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.155 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.155 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.155 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.156 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.156 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.156 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.156 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.156 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.156 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.156 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.157 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.157 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.157 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.157 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.157 INFO:teuthology.orchestra.run.smithi093.stdout: "min_size": { 2023-11-16T16:55:06.157 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_size", 2023-11-16T16:55:06.157 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.157 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.158 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.158 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.158 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.158 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.158 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.158 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default min_size for any created local pool", 2023-11-16T16:55:06.158 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-16T16:55:06.159 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.159 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.159 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.159 INFO:teuthology.orchestra.run.smithi093.stdout: "num_rep": { 2023-11-16T16:55:06.159 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "num_rep", 2023-11-16T16:55:06.159 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.159 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.160 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.160 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3", 2023-11-16T16:55:06.160 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.160 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.160 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.160 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default replica count for any created local pool", 2023-11-16T16:55:06.160 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.161 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.161 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.161 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.161 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_num": { 2023-11-16T16:55:06.161 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_num", 2023-11-16T16:55:06.161 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.161 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.162 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.162 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "128", 2023-11-16T16:55:06.162 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.162 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.162 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.162 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default pg_num for any created local pool", 2023-11-16T16:55:06.162 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.162 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.163 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.163 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.163 INFO:teuthology.orchestra.run.smithi093.stdout: "prefix": { 2023-11-16T16:55:06.163 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prefix", 2023-11-16T16:55:06.163 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.163 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.163 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.164 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.164 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.164 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.164 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.164 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name prefix for any created local pool", 2023-11-16T16:55:06.164 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.164 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.165 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.165 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.165 INFO:teuthology.orchestra.run.smithi093.stdout: "subtree": { 2023-11-16T16:55:06.165 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "subtree", 2023-11-16T16:55:06.165 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.165 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.165 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.165 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "rack", 2023-11-16T16:55:06.166 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.166 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.166 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.166 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-16T16:55:06.166 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-16T16:55:06.166 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.166 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.167 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.167 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.167 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.167 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.167 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mds_autoscaler", 2023-11-16T16:55:06.167 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.167 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.167 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.168 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.168 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.168 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.168 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.168 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.168 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.168 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.168 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.169 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.169 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.169 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.169 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.169 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.169 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.169 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.170 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.170 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.170 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.170 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.170 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.170 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.170 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.171 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.171 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.171 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.171 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.171 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.171 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.171 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.172 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.172 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.172 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.172 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.172 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.172 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.172 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.172 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.173 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.173 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.173 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.173 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.173 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.173 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.173 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.174 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.174 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.174 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.174 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.174 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.174 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.174 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.175 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.175 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.175 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.175 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.175 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.175 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.175 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.175 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.176 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.176 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.176 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.176 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.176 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.176 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.176 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.177 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.177 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.177 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.177 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.177 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.177 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.177 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.177 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirroring", 2023-11-16T16:55:06.178 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.178 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.178 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.178 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.178 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.178 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.178 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.179 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.179 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.179 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.179 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.179 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.179 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.179 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.180 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.180 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.180 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.180 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.180 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.180 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.180 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.180 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.181 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.181 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.181 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.181 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.181 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.181 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.181 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.182 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.182 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.182 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.182 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.182 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.182 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.182 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.182 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.183 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.183 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.183 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.183 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.183 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.183 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.183 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.184 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.184 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.184 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.184 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.184 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.184 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.184 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.185 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.185 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.185 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.185 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.185 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.185 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.185 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.186 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.186 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.186 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.186 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.186 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.186 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.186 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.187 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.187 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.187 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.187 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.187 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.187 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.187 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.187 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.188 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.188 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.188 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.188 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "nfs", 2023-11-16T16:55:06.188 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.188 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.188 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.189 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.189 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.189 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.189 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.189 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.189 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.190 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.190 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.190 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.190 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.190 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.190 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.190 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.191 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.191 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.191 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.191 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.191 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.191 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.191 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.192 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.192 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.192 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.192 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.192 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.192 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.192 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.192 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.193 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.193 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.193 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.193 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.193 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.193 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.193 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.194 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.194 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.194 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.194 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.194 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.194 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.194 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.195 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.195 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.195 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.195 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.195 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.195 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.195 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.195 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.196 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.196 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.196 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.196 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.196 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.196 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.196 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.197 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.197 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.197 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.197 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.197 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.197 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.197 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.198 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.198 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.198 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.198 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.198 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.198 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.198 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.199 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.199 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.199 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-11-16T16:55:06.199 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.199 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.199 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.200 INFO:teuthology.orchestra.run.smithi093.stdout: "fail_fs": { 2023-11-16T16:55:06.200 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "fail_fs", 2023-11-16T16:55:06.200 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.200 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.200 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.200 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.200 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.201 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.201 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.201 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-16T16:55:06.201 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.201 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.201 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.201 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.202 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.202 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.202 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.202 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.202 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.202 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.202 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.203 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.203 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.203 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.203 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.203 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.203 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.203 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.204 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.204 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.204 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.204 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.204 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.204 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.204 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.205 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.205 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.205 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.205 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.205 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.205 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.205 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.206 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.206 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.206 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.206 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.206 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.206 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.206 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.207 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.207 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.207 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.207 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.207 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.207 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.207 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.208 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.208 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.208 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.208 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.208 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.208 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.208 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.209 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.209 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.209 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.209 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.209 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.209 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.210 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.210 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.210 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.210 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.210 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.210 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.210 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.211 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.211 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.211 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.211 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.211 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.211 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.211 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.211 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.212 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator": { 2023-11-16T16:55:06.212 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-11-16T16:55:06.212 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.212 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.212 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.212 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.212 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.213 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.213 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.213 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm", 2023-11-16T16:55:06.213 INFO:teuthology.orchestra.run.smithi093.stdout: "rook", 2023-11-16T16:55:06.213 INFO:teuthology.orchestra.run.smithi093.stdout: "test_orchestrator" 2023-11-16T16:55:06.213 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.213 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Orchestrator backend", 2023-11-16T16:55:06.213 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.214 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.214 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.214 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.214 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.214 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.214 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.214 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_perf_query", 2023-11-16T16:55:06.215 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.215 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.215 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.215 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.215 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.215 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.215 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.216 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.216 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.216 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.216 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.216 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.216 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.216 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.217 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.217 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.217 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.217 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.217 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.217 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.217 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.218 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.218 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.218 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.218 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.218 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.218 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.218 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.219 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.219 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.219 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.219 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.219 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.219 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.219 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.220 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.220 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.220 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.220 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.220 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.220 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.220 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.221 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.221 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.221 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.221 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.221 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.221 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.221 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.222 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.222 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.222 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.222 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.222 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.222 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.222 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.223 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.223 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.223 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.223 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.223 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.223 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.223 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.223 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.224 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.224 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.224 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.224 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.224 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.224 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.224 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.224 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.225 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.225 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.225 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.225 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.225 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_support", 2023-11-16T16:55:06.225 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.225 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.226 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.226 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.226 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.226 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.226 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.226 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.226 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.227 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.227 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.227 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.227 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.227 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.227 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.227 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.227 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.228 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.228 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.228 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.228 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.228 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.228 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.228 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.229 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.229 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.229 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.229 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.229 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.229 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.229 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.230 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.230 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.230 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.230 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.230 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.230 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.230 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.231 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.231 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.231 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.231 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.231 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.231 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.231 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.232 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.232 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.232 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.232 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.232 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.232 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.232 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.232 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.233 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.233 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.233 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.233 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.233 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.233 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.233 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.234 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.234 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.234 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.234 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.234 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.234 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.234 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.235 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.235 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.235 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.235 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.235 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.235 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.235 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.235 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.236 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.236 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_autoscaler", 2023-11-16T16:55:06.236 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.236 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.236 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.236 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.236 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.236 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.237 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.237 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.237 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.237 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.237 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.237 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.237 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.238 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.238 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.238 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.238 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.238 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.238 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.238 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.239 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.239 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.239 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.239 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.239 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.239 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.240 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.240 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.240 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.240 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.240 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.240 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.240 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.241 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.241 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.241 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.241 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.241 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.241 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.241 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.241 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.242 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.242 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.242 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.242 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.242 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.242 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.242 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.243 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.243 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.243 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.243 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.243 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.243 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.243 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.244 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.244 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.244 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.244 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.244 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.244 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.244 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.245 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.245 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.245 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.245 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.245 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.245 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.245 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.245 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.246 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.246 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.246 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.246 INFO:teuthology.orchestra.run.smithi093.stdout: "noautoscale": { 2023-11-16T16:55:06.246 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "noautoscale", 2023-11-16T16:55:06.246 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.246 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.247 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.247 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.247 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.247 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.247 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.247 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "global autoscale flag", 2023-11-16T16:55:06.247 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-16T16:55:06.248 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.248 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.248 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.248 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:55:06.248 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:55:06.248 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:06.249 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.249 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.249 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:55:06.249 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.249 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.249 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.249 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.250 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.250 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.250 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.250 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.250 INFO:teuthology.orchestra.run.smithi093.stdout: "threshold": { 2023-11-16T16:55:06.250 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threshold", 2023-11-16T16:55:06.250 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:55:06.251 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.251 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.251 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-11-16T16:55:06.251 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1.0", 2023-11-16T16:55:06.251 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.251 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.251 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "scaling threshold", 2023-11-16T16:55:06.251 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-16T16:55:06.252 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.252 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.252 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.252 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.252 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.252 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.252 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "progress", 2023-11-16T16:55:06.252 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.253 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.253 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.253 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_pg_recovery_event": { 2023-11-16T16:55:06.253 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_pg_recovery_event", 2023-11-16T16:55:06.253 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.253 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.253 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.254 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.254 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.254 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.254 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.254 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-16T16:55:06.254 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.254 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.255 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.255 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.255 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-11-16T16:55:06.255 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-11-16T16:55:06.255 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.255 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.255 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.256 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:06.256 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.256 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.256 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.256 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.256 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.256 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.257 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.257 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.257 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.257 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.257 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.257 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.257 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.257 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.258 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.258 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.258 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.258 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.258 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.258 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.258 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.259 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.259 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.259 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.259 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.259 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.259 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.259 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.260 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.260 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.260 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.260 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.260 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.260 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.260 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.261 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.261 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.261 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.261 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.261 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.261 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.261 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.262 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.262 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.262 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.262 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.262 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.262 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.262 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.262 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.263 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.263 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.263 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.263 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.263 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.263 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.264 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.264 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.264 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.264 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.264 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.264 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.264 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.264 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.265 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.265 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.265 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.265 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.265 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.265 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.265 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.265 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.266 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.266 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.266 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.266 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.266 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.266 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.266 INFO:teuthology.orchestra.run.smithi093.stdout: "max_completed_events": { 2023-11-16T16:55:06.267 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_completed_events", 2023-11-16T16:55:06.267 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.267 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.267 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.267 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "50", 2023-11-16T16:55:06.267 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.267 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.267 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.268 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "number of past completed events to remember", 2023-11-16T16:55:06.268 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.268 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.268 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.268 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.268 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-11-16T16:55:06.268 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-11-16T16:55:06.269 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:06.269 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.269 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.269 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-11-16T16:55:06.269 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.269 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.269 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.270 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long the module is going to sleep", 2023-11-16T16:55:06.270 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.270 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.270 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.270 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.270 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.270 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.271 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.271 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus", 2023-11-16T16:55:06.271 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.271 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.271 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.271 INFO:teuthology.orchestra.run.smithi093.stdout: "cache": { 2023-11-16T16:55:06.271 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cache", 2023-11-16T16:55:06.272 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.272 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.272 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.272 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:06.272 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.272 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.272 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.272 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.273 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.273 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.273 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.273 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.273 INFO:teuthology.orchestra.run.smithi093.stdout: "exclude_perf_counters": { 2023-11-16T16:55:06.273 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "exclude_perf_counters", 2023-11-16T16:55:06.273 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.273 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.274 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.274 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:06.274 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.274 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.274 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.274 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-16T16:55:06.274 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-16T16:55:06.275 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.275 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.275 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.275 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.275 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.275 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.275 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.275 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.276 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.276 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.276 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.276 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.276 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.276 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.276 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.277 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.277 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.277 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.277 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.277 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.277 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.277 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.277 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.278 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.278 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.278 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.278 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.278 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.278 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.278 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.279 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.279 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.279 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.279 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.279 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.279 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.279 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.280 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.280 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.280 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.280 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.280 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.280 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.280 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.281 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.281 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.281 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.281 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.281 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.281 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.281 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.281 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.282 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.282 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.282 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.282 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.282 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.282 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.282 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.282 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.283 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.283 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.283 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.283 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.283 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.283 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.283 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.284 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.284 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.284 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.284 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.284 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.284 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.284 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools": { 2023-11-16T16:55:06.284 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools", 2023-11-16T16:55:06.285 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.285 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.285 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.285 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.285 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.285 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.285 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.285 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.286 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.286 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.286 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.286 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.286 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-16T16:55:06.286 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-16T16:55:06.286 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.287 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.287 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.287 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300", 2023-11-16T16:55:06.287 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.287 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.287 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.287 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.287 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.288 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.288 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.288 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.288 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_interval": { 2023-11-16T16:55:06.288 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_interval", 2023-11-16T16:55:06.288 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:55:06.289 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.289 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.289 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15.0", 2023-11-16T16:55:06.289 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.289 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.289 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.289 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.290 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.290 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.290 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.290 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.290 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-11-16T16:55:06.290 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-11-16T16:55:06.290 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.291 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.291 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.291 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-11-16T16:55:06.291 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.291 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.291 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.291 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-16T16:55:06.292 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.292 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.292 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.292 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.292 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-11-16T16:55:06.292 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-11-16T16:55:06.292 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.293 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.293 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.293 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "9283", 2023-11-16T16:55:06.293 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.293 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.293 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.293 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-16T16:55:06.293 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.294 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.294 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.294 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.294 INFO:teuthology.orchestra.run.smithi093.stdout: "stale_cache_strategy": { 2023-11-16T16:55:06.294 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stale_cache_strategy", 2023-11-16T16:55:06.294 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.294 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.295 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.295 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "log", 2023-11-16T16:55:06.295 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.295 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.295 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.295 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.296 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.296 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.296 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.296 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.296 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-11-16T16:55:06.296 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-11-16T16:55:06.296 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.296 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.297 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.297 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "default", 2023-11-16T16:55:06.297 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.297 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.297 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.297 INFO:teuthology.orchestra.run.smithi093.stdout: "default", 2023-11-16T16:55:06.297 INFO:teuthology.orchestra.run.smithi093.stdout: "error" 2023-11-16T16:55:06.298 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.298 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.298 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.298 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.298 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.298 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.298 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-11-16T16:55:06.299 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-11-16T16:55:06.299 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.299 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.299 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.299 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-11-16T16:55:06.299 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-11-16T16:55:06.300 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-11-16T16:55:06.300 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.300 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.300 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.300 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.300 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.300 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.301 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.301 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.301 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.301 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_support", 2023-11-16T16:55:06.301 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.301 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.301 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.302 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.302 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.302 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.302 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.302 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.302 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.302 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.303 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.303 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.303 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.303 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.303 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.303 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.303 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.304 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.304 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.304 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.304 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.304 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.304 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.304 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.305 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.305 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.305 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.305 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.305 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.305 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.305 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.305 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.306 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.306 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.306 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.306 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.306 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.306 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.306 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.307 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.307 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.307 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.307 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.307 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.307 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.307 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.308 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.308 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.308 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.308 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.308 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.308 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.308 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.308 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.309 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.309 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.309 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.309 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.309 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.309 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.309 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.310 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.310 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.310 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.310 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.310 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.310 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.311 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.311 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.311 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.311 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.311 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.311 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.311 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_snap_create": { 2023-11-16T16:55:06.311 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_snap_create", 2023-11-16T16:55:06.312 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.312 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.312 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.312 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-11-16T16:55:06.312 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.312 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.312 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.312 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.313 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.313 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.313 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.313 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.313 INFO:teuthology.orchestra.run.smithi093.stdout: "mirror_snapshot_schedule": { 2023-11-16T16:55:06.313 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirror_snapshot_schedule", 2023-11-16T16:55:06.313 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.314 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.314 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.314 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.314 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.314 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.314 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.314 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.315 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.315 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.315 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.315 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.315 INFO:teuthology.orchestra.run.smithi093.stdout: "trash_purge_schedule": { 2023-11-16T16:55:06.315 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "trash_purge_schedule", 2023-11-16T16:55:06.315 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.316 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.316 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.316 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.316 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.316 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.316 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.316 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.317 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.317 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.317 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.317 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.317 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.317 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.317 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.318 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "restful", 2023-11-16T16:55:06.318 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.318 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.318 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.318 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_auth": { 2023-11-16T16:55:06.318 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_auth", 2023-11-16T16:55:06.319 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.319 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.319 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.319 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:06.319 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.319 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.319 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.320 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.320 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.320 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.320 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.320 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.320 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-11-16T16:55:06.321 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-11-16T16:55:06.321 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.321 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.321 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.321 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.321 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.321 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.322 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.322 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.322 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.322 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.322 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.322 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.322 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.323 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.323 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.323 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.323 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.323 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.323 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.323 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.324 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.324 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.324 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.324 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.324 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.324 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.324 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.325 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.325 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.325 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.325 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.325 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.325 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.325 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.326 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.326 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.326 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.326 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.326 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.326 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.326 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.327 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.327 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.327 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.327 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.327 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.327 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.327 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.328 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.328 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.328 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.328 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.328 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.328 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.328 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.329 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.329 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.329 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.329 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.329 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.329 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.330 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.330 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.330 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.330 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.330 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.330 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.331 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.331 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.331 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.331 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.331 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.331 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.331 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.332 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.332 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.332 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.332 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.332 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.332 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.332 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.333 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.333 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-11-16T16:55:06.333 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-11-16T16:55:06.333 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.333 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.333 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.333 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.334 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.334 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.334 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.334 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.334 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.334 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.334 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.335 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.335 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-11-16T16:55:06.335 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-11-16T16:55:06.335 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.335 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.335 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.335 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.335 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.336 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.336 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.336 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.336 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.336 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.336 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.336 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.337 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.337 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.337 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.337 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rgw", 2023-11-16T16:55:06.337 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.337 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.337 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.338 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.338 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.338 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.338 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.338 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.338 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.338 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.339 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.339 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.339 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.339 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.339 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.339 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.340 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.340 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.340 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.340 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.340 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.340 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.340 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.341 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.341 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.341 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.341 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.341 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.341 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.341 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.341 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.342 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.342 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.342 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.342 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.342 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.342 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.342 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.343 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.343 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.343 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.343 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.343 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.343 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.343 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.344 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.344 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.344 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.344 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.344 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.344 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.344 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.345 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.345 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.345 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.345 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.345 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.345 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.345 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.346 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.346 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.346 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.346 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.346 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.346 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.346 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.347 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.347 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.347 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.347 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.347 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.347 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.347 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.347 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.348 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.348 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.348 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rook", 2023-11-16T16:55:06.348 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.348 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.348 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.348 INFO:teuthology.orchestra.run.smithi093.stdout: "drive_group_interval": { 2023-11-16T16:55:06.349 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "drive_group_interval", 2023-11-16T16:55:06.349 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:55:06.349 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.349 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.349 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300.0", 2023-11-16T16:55:06.349 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.350 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.350 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.350 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-16T16:55:06.350 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.350 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.350 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.350 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.351 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.351 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.351 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.351 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.351 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.351 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.351 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.351 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.352 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.352 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.352 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.352 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.352 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.352 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.353 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.353 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.353 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.353 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.353 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.353 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.353 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.354 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.354 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.354 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.354 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.354 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.354 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.354 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.354 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.355 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.355 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.355 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.355 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.355 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.355 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.355 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.356 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.356 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.356 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.356 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.356 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.356 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.356 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.356 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.357 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.357 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.357 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.357 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.357 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.357 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.357 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.358 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.358 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.358 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.358 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.358 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.358 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.358 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.359 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.359 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.359 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.359 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.359 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.359 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.359 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.360 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.360 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.360 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.360 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.360 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.360 INFO:teuthology.orchestra.run.smithi093.stdout: "storage_class": { 2023-11-16T16:55:06.360 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "storage_class", 2023-11-16T16:55:06.361 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.361 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.361 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.361 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "local", 2023-11-16T16:55:06.361 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.361 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.361 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.361 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-16T16:55:06.362 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.362 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.362 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.362 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.362 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.362 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.362 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.363 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "selftest", 2023-11-16T16:55:06.363 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.363 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.363 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.363 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.363 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.363 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.364 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.364 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.364 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.364 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.364 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.364 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.364 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.364 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.365 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.365 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.365 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.365 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.365 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.365 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.365 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.365 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.366 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.366 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.366 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.366 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.366 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.366 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.366 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.367 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.367 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.367 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.367 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.367 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.367 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.367 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.368 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.368 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.368 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.368 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.368 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.368 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.368 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.368 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.369 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.369 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.369 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.369 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.369 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.369 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.369 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.370 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.370 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.370 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.370 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.370 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.370 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.370 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.371 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.371 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.371 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.371 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.371 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.371 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.371 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.372 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.372 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.372 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.372 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.372 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.372 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.372 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.372 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.373 INFO:teuthology.orchestra.run.smithi093.stdout: "roption1": { 2023-11-16T16:55:06.373 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption1", 2023-11-16T16:55:06.373 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.373 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.373 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.373 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.373 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.374 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.374 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.374 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.374 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.374 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.374 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.374 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.375 INFO:teuthology.orchestra.run.smithi093.stdout: "roption2": { 2023-11-16T16:55:06.375 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption2", 2023-11-16T16:55:06.375 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.375 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.375 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.375 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "xyz", 2023-11-16T16:55:06.375 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.376 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.376 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.376 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.376 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.376 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.376 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.376 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.377 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption1": { 2023-11-16T16:55:06.377 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption1", 2023-11-16T16:55:06.377 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.377 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.377 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.377 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.377 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.377 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.378 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.378 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.378 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.378 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.378 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.379 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.379 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption2": { 2023-11-16T16:55:06.379 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption2", 2023-11-16T16:55:06.379 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.380 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.380 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.380 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.380 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.381 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.381 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.381 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.381 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.382 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.382 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.382 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.382 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption3": { 2023-11-16T16:55:06.383 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption3", 2023-11-16T16:55:06.383 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-11-16T16:55:06.383 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.383 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.383 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.383 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.383 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.383 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.384 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.384 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.384 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.384 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.384 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.384 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption4": { 2023-11-16T16:55:06.384 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption4", 2023-11-16T16:55:06.385 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.385 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.385 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.385 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.385 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.385 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.385 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.385 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.386 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.386 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.386 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.386 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.386 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption5": { 2023-11-16T16:55:06.386 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption5", 2023-11-16T16:55:06.386 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.387 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.387 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.387 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.387 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.387 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.387 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.387 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.387 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.388 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.388 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.388 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.388 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption6": { 2023-11-16T16:55:06.388 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption6", 2023-11-16T16:55:06.388 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.388 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.389 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.389 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:06.389 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.389 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.389 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.389 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.389 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.390 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.390 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.390 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.390 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption7": { 2023-11-16T16:55:06.390 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption7", 2023-11-16T16:55:06.390 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.390 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.391 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.391 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.391 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-11-16T16:55:06.391 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "42", 2023-11-16T16:55:06.391 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.391 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.391 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.392 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.392 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.392 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.392 INFO:teuthology.orchestra.run.smithi093.stdout: "testkey": { 2023-11-16T16:55:06.392 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testkey", 2023-11-16T16:55:06.392 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.392 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.392 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.393 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.393 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.393 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.393 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.393 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.393 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.393 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.394 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.394 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.394 INFO:teuthology.orchestra.run.smithi093.stdout: "testlkey": { 2023-11-16T16:55:06.394 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testlkey", 2023-11-16T16:55:06.394 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.394 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.394 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.395 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.395 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.395 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.395 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.395 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.395 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.395 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.395 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.396 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.396 INFO:teuthology.orchestra.run.smithi093.stdout: "testnewline": { 2023-11-16T16:55:06.396 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testnewline", 2023-11-16T16:55:06.396 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.396 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.396 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.396 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.397 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.397 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.397 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.397 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.397 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.397 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.397 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.397 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.398 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.398 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.398 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.398 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-11-16T16:55:06.398 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.398 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.399 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.399 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_m_granularity": { 2023-11-16T16:55:06.399 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_m_granularity", 2023-11-16T16:55:06.399 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.399 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.399 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.399 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.400 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.400 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.400 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.400 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow minute scheduled snapshots", 2023-11-16T16:55:06.400 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.400 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.400 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.401 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.401 INFO:teuthology.orchestra.run.smithi093.stdout: "dump_on_update": { 2023-11-16T16:55:06.401 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dump_on_update", 2023-11-16T16:55:06.401 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.401 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.401 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.401 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.402 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.402 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.402 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.402 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "dump database to debug log on update", 2023-11-16T16:55:06.402 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.402 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.402 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.403 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.403 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.403 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.403 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.403 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.403 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.403 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.403 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.404 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.404 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.404 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.404 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.404 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.404 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.404 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.404 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.405 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.405 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.405 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.405 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.405 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.405 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.405 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.405 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.406 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.406 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.406 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.406 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.406 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.406 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.406 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.407 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.407 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.407 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.407 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.407 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.407 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.407 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.407 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.408 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.408 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.408 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.408 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.408 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.408 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.408 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.409 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.409 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.409 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.409 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.409 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.409 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.409 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.410 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.410 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.410 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.410 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.410 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.410 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.410 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.411 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.411 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.411 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.411 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.411 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.411 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.411 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.412 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.412 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.412 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.412 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.412 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.412 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.412 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.412 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stats", 2023-11-16T16:55:06.413 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.413 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.413 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.413 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.413 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.413 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.413 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.414 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.414 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.414 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.414 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.414 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.414 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.414 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.415 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.415 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.415 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.415 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.415 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.415 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.415 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.416 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.416 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.416 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.416 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.416 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.416 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.416 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.416 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.417 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.417 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.417 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.417 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.417 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.417 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.417 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.418 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.418 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.418 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.418 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.418 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.418 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.418 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.419 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.419 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.419 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.419 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.419 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.419 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.419 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.420 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.420 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.420 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.420 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.420 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.420 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.420 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.421 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.421 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.421 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.421 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.421 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.421 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.421 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.421 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.422 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.422 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.422 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.422 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.422 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.422 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.422 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.423 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.423 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.423 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.423 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.423 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "status", 2023-11-16T16:55:06.423 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.423 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.423 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.424 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.424 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.424 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.424 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.424 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.424 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.424 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.425 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.425 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.425 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.425 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.425 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.425 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.425 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.425 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.426 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.426 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.426 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.426 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.426 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.426 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.426 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.427 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.427 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.427 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.427 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.427 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.427 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.427 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.428 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.428 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.428 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.428 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.428 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.428 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.428 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.429 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.429 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.429 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.429 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.429 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.429 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.429 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.430 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.430 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.430 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.430 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.430 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.430 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.430 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.431 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.431 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.431 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.431 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.431 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.431 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.431 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.432 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.432 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.432 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.432 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.432 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.432 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.432 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.432 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.433 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.433 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.433 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.433 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.433 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.433 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.433 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.434 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.434 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telegraf", 2023-11-16T16:55:06.434 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.434 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.434 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.434 INFO:teuthology.orchestra.run.smithi093.stdout: "address": { 2023-11-16T16:55:06.434 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "address", 2023-11-16T16:55:06.435 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.435 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.435 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.435 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-16T16:55:06.435 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.435 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.435 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.436 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.436 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.436 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.436 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.436 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.436 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:55:06.436 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:55:06.436 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:06.437 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.437 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.437 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15", 2023-11-16T16:55:06.437 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.437 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.437 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.437 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.438 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.438 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.438 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.438 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.438 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.438 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.438 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.439 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.439 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.439 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.439 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.439 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.439 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.439 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.440 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.440 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.440 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.440 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.440 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.440 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.440 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.441 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.441 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.441 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.441 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.441 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.441 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.441 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.442 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.442 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.442 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.442 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.442 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.442 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.442 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.442 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.443 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.443 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.443 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.443 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.443 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.443 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.443 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.444 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.444 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.444 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.444 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.444 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.444 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.444 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.445 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.445 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.445 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.445 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.445 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.445 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.445 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.445 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.446 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.446 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.446 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.446 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.446 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.446 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.446 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.447 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.447 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.447 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.447 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.447 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.447 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.447 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.447 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.448 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.448 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.448 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.448 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.448 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telemetry", 2023-11-16T16:55:06.448 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.448 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.449 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.449 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_basic": { 2023-11-16T16:55:06.449 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_basic", 2023-11-16T16:55:06.449 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.449 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.449 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.449 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:06.450 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.450 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.450 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.450 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-16T16:55:06.450 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.450 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.451 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.451 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.451 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_crash": { 2023-11-16T16:55:06.451 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_crash", 2023-11-16T16:55:06.451 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.451 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.451 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.451 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:06.452 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.452 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.452 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.452 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-16T16:55:06.452 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.452 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.452 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.453 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.453 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_device": { 2023-11-16T16:55:06.453 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_device", 2023-11-16T16:55:06.453 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.453 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.453 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.453 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-11-16T16:55:06.453 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.454 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.454 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.454 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-16T16:55:06.454 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.454 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.454 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.454 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.455 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_ident": { 2023-11-16T16:55:06.455 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_ident", 2023-11-16T16:55:06.455 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.455 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.455 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.455 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.455 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.456 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.456 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.456 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-16T16:55:06.456 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.456 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.456 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.456 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.456 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_perf": { 2023-11-16T16:55:06.457 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_perf", 2023-11-16T16:55:06.457 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.457 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.457 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.457 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.457 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.457 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.458 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.458 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-16T16:55:06.458 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.458 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.458 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.458 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.458 INFO:teuthology.orchestra.run.smithi093.stdout: "contact": { 2023-11-16T16:55:06.458 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "contact", 2023-11-16T16:55:06.459 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.459 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.459 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.459 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.459 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.459 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.459 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.460 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.460 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.460 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.460 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.460 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.460 INFO:teuthology.orchestra.run.smithi093.stdout: "description": { 2023-11-16T16:55:06.460 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "description", 2023-11-16T16:55:06.461 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.461 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.461 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.461 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.461 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.461 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.461 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.461 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.462 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.462 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.462 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.462 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.462 INFO:teuthology.orchestra.run.smithi093.stdout: "device_url": { 2023-11-16T16:55:06.462 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_url", 2023-11-16T16:55:06.462 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.463 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.463 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.463 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-16T16:55:06.463 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.463 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.463 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.463 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.464 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.464 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.464 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.464 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.464 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-11-16T16:55:06.464 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-11-16T16:55:06.464 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.465 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.465 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.465 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.465 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.465 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.465 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.465 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.465 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.466 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.466 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.466 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.466 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:55:06.466 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:55:06.466 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.466 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.467 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.467 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "24", 2023-11-16T16:55:06.467 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "8", 2023-11-16T16:55:06.467 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.467 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.467 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.467 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.467 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.468 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.468 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.468 INFO:teuthology.orchestra.run.smithi093.stdout: "last_opt_revision": { 2023-11-16T16:55:06.468 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "last_opt_revision", 2023-11-16T16:55:06.468 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.468 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.468 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.469 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1", 2023-11-16T16:55:06.469 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.469 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.469 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.469 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.469 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.469 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.470 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.470 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.470 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard": { 2023-11-16T16:55:06.470 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard", 2023-11-16T16:55:06.470 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.470 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.470 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.471 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.471 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.471 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.471 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.471 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.471 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.471 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.472 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.472 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.472 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard_description": { 2023-11-16T16:55:06.472 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard_description", 2023-11-16T16:55:06.472 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.472 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.472 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.472 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.473 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.473 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.473 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.473 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.473 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.473 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.473 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.474 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.474 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.474 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.474 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.474 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.474 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.474 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.475 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.475 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.475 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.475 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.475 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.475 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.475 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.475 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.476 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.476 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.476 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.476 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.476 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.476 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.476 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.476 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.477 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.477 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.477 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.477 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.477 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.477 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.477 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.478 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.478 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.478 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.478 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.478 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.478 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.478 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.479 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.479 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.479 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.479 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.479 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.479 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.480 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.480 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.480 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.480 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.480 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.480 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.480 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.480 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.481 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.481 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.481 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.481 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.481 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.481 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.481 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.481 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.482 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.482 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.482 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.482 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.482 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.482 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.482 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.483 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.483 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.483 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.483 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.483 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.483 INFO:teuthology.orchestra.run.smithi093.stdout: "organization": { 2023-11-16T16:55:06.483 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "organization", 2023-11-16T16:55:06.483 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.484 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.484 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.484 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.484 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.484 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.484 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.484 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.485 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.485 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.485 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.485 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.485 INFO:teuthology.orchestra.run.smithi093.stdout: "proxy": { 2023-11-16T16:55:06.485 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "proxy", 2023-11-16T16:55:06.485 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.485 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.486 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.486 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.486 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.486 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.486 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.486 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.486 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.487 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.487 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.487 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.487 INFO:teuthology.orchestra.run.smithi093.stdout: "url": { 2023-11-16T16:55:06.487 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url", 2023-11-16T16:55:06.487 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.487 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.487 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.488 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-16T16:55:06.488 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.488 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.488 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.488 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.488 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.488 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.489 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.489 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.489 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.489 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.489 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.489 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "test_orchestrator", 2023-11-16T16:55:06.490 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.490 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.490 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.490 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.490 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.490 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.490 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.491 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.491 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.491 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.491 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.491 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.491 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.491 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.491 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.492 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.492 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.492 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.492 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.492 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.492 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.492 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.493 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.493 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.493 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.493 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.493 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.493 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.493 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.494 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.494 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.494 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.494 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.494 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.494 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.494 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.495 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.495 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.495 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.495 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.495 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.495 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.495 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.495 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.496 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.496 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.496 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.496 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.496 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.496 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.496 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.497 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.497 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.497 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.497 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.497 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.497 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.497 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.497 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.498 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.498 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.498 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.498 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.498 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.498 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.498 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.499 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.499 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.499 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.499 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.499 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.499 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.500 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.500 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.500 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.500 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.500 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "volumes", 2023-11-16T16:55:06.500 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.500 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.501 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.501 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.501 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.501 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.501 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.501 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.501 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.501 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.502 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.502 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.502 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.502 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.502 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.502 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.502 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.503 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.503 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.503 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.503 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.503 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.503 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.503 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.503 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.504 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.504 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.504 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.504 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.504 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.504 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.504 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.505 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.505 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.505 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.505 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.505 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.505 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.505 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.506 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.506 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.506 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.506 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.506 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.506 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.506 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.506 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.507 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.507 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.507 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.507 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.507 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.507 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.507 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.508 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.509 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.509 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.509 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.510 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.510 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.510 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.510 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.510 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.510 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.510 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.511 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.511 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.511 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.511 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.511 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.511 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.511 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.512 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.512 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_clones": { 2023-11-16T16:55:06.512 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_clones", 2023-11-16T16:55:06.512 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.512 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.512 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.512 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4", 2023-11-16T16:55:06.512 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.513 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.513 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.513 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-16T16:55:06.513 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.513 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.513 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.513 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.514 INFO:teuthology.orchestra.run.smithi093.stdout: "snapshot_clone_delay": { 2023-11-16T16:55:06.514 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snapshot_clone_delay", 2023-11-16T16:55:06.514 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.514 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.514 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.514 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-11-16T16:55:06.514 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.515 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.515 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.515 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-16T16:55:06.515 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.515 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.515 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.515 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.515 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.516 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.516 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.516 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix", 2023-11-16T16:55:06.516 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-11-16T16:55:06.516 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-11-16T16:55:06.516 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-11-16T16:55:06.516 INFO:teuthology.orchestra.run.smithi093.stdout: "discovery_interval": { 2023-11-16T16:55:06.517 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "discovery_interval", 2023-11-16T16:55:06.517 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-11-16T16:55:06.517 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.517 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.517 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "100", 2023-11-16T16:55:06.517 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.517 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.517 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.518 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.518 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.518 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.518 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.518 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.518 INFO:teuthology.orchestra.run.smithi093.stdout: "identifier": { 2023-11-16T16:55:06.518 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "identifier", 2023-11-16T16:55:06.519 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.519 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.519 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.519 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.519 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.519 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.520 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.520 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.520 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.520 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.520 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.520 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.520 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-11-16T16:55:06.520 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-11-16T16:55:06.521 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-11-16T16:55:06.521 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.521 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.521 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-11-16T16:55:06.521 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.521 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.521 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.522 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.522 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.522 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.522 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.522 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.522 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-11-16T16:55:06.522 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-11-16T16:55:06.522 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.523 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.523 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.523 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.523 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.523 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.523 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.523 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.524 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.524 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.524 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.524 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.524 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.524 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.524 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.524 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.525 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.525 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.525 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.525 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-11-16T16:55:06.525 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-11-16T16:55:06.525 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.525 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.526 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.526 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.526 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.526 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.526 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.526 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.526 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.526 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.527 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.527 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.527 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-11-16T16:55:06.527 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-11-16T16:55:06.527 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.527 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.527 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.528 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-11-16T16:55:06.528 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.528 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.528 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-11-16T16:55:06.528 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-11-16T16:55:06.528 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-11-16T16:55:06.528 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-11-16T16:55:06.528 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-11-16T16:55:06.529 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-11-16T16:55:06.529 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-11-16T16:55:06.529 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.529 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.529 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.529 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.529 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.530 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.530 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-11-16T16:55:06.530 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-11-16T16:55:06.530 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-11-16T16:55:06.530 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.530 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-11-16T16:55:06.530 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-11-16T16:55:06.531 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.531 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.531 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.531 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.531 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.531 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.531 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.532 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.532 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_host": { 2023-11-16T16:55:06.532 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_host", 2023-11-16T16:55:06.532 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.532 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.532 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.532 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-11-16T16:55:06.532 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.533 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.533 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.533 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.533 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.533 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.533 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.533 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.534 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_port": { 2023-11-16T16:55:06.534 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_port", 2023-11-16T16:55:06.534 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-11-16T16:55:06.534 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.534 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.534 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10051", 2023-11-16T16:55:06.534 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.534 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.535 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.535 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.535 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.535 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.535 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.535 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.535 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_sender": { 2023-11-16T16:55:06.536 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_sender", 2023-11-16T16:55:06.536 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-11-16T16:55:06.536 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-11-16T16:55:06.536 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-11-16T16:55:06.536 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-16T16:55:06.536 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-11-16T16:55:06.536 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-11-16T16:55:06.536 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-11-16T16:55:06.537 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-11-16T16:55:06.537 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-11-16T16:55:06.537 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-11-16T16:55:06.537 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-11-16T16:55:06.537 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.537 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.537 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.537 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.538 INFO:teuthology.orchestra.run.smithi093.stdout: "services": { 2023-11-16T16:55:06.538 INFO:teuthology.orchestra.run.smithi093.stdout: "dashboard": "https://172.21.15.93:8443/" 2023-11-16T16:55:06.538 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.538 INFO:teuthology.orchestra.run.smithi093.stdout: "always_on_modules": { 2023-11-16T16:55:06.538 INFO:teuthology.orchestra.run.smithi093.stdout: "octopus": [ 2023-11-16T16:55:06.538 INFO:teuthology.orchestra.run.smithi093.stdout: "balancer", 2023-11-16T16:55:06.539 INFO:teuthology.orchestra.run.smithi093.stdout: "crash", 2023-11-16T16:55:06.539 INFO:teuthology.orchestra.run.smithi093.stdout: "devicehealth", 2023-11-16T16:55:06.539 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator", 2023-11-16T16:55:06.539 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_autoscaler", 2023-11-16T16:55:06.539 INFO:teuthology.orchestra.run.smithi093.stdout: "progress", 2023-11-16T16:55:06.539 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_support", 2023-11-16T16:55:06.539 INFO:teuthology.orchestra.run.smithi093.stdout: "status", 2023-11-16T16:55:06.539 INFO:teuthology.orchestra.run.smithi093.stdout: "telemetry", 2023-11-16T16:55:06.540 INFO:teuthology.orchestra.run.smithi093.stdout: "volumes" 2023-11-16T16:55:06.540 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.540 INFO:teuthology.orchestra.run.smithi093.stdout: "pacific": [ 2023-11-16T16:55:06.540 INFO:teuthology.orchestra.run.smithi093.stdout: "balancer", 2023-11-16T16:55:06.540 INFO:teuthology.orchestra.run.smithi093.stdout: "crash", 2023-11-16T16:55:06.540 INFO:teuthology.orchestra.run.smithi093.stdout: "devicehealth", 2023-11-16T16:55:06.540 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator", 2023-11-16T16:55:06.541 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_autoscaler", 2023-11-16T16:55:06.541 INFO:teuthology.orchestra.run.smithi093.stdout: "progress", 2023-11-16T16:55:06.541 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_support", 2023-11-16T16:55:06.541 INFO:teuthology.orchestra.run.smithi093.stdout: "status", 2023-11-16T16:55:06.541 INFO:teuthology.orchestra.run.smithi093.stdout: "telemetry", 2023-11-16T16:55:06.541 INFO:teuthology.orchestra.run.smithi093.stdout: "volumes" 2023-11-16T16:55:06.541 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.542 INFO:teuthology.orchestra.run.smithi093.stdout: "quincy": [ 2023-11-16T16:55:06.542 INFO:teuthology.orchestra.run.smithi093.stdout: "balancer", 2023-11-16T16:55:06.542 INFO:teuthology.orchestra.run.smithi093.stdout: "crash", 2023-11-16T16:55:06.542 INFO:teuthology.orchestra.run.smithi093.stdout: "devicehealth", 2023-11-16T16:55:06.542 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator", 2023-11-16T16:55:06.542 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_autoscaler", 2023-11-16T16:55:06.543 INFO:teuthology.orchestra.run.smithi093.stdout: "progress", 2023-11-16T16:55:06.543 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_support", 2023-11-16T16:55:06.543 INFO:teuthology.orchestra.run.smithi093.stdout: "status", 2023-11-16T16:55:06.543 INFO:teuthology.orchestra.run.smithi093.stdout: "telemetry", 2023-11-16T16:55:06.543 INFO:teuthology.orchestra.run.smithi093.stdout: "volumes" 2023-11-16T16:55:06.543 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-11-16T16:55:06.543 INFO:teuthology.orchestra.run.smithi093.stdout: "reef": [ 2023-11-16T16:55:06.543 INFO:teuthology.orchestra.run.smithi093.stdout: "balancer", 2023-11-16T16:55:06.544 INFO:teuthology.orchestra.run.smithi093.stdout: "crash", 2023-11-16T16:55:06.544 INFO:teuthology.orchestra.run.smithi093.stdout: "devicehealth", 2023-11-16T16:55:06.544 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator", 2023-11-16T16:55:06.544 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_autoscaler", 2023-11-16T16:55:06.544 INFO:teuthology.orchestra.run.smithi093.stdout: "progress", 2023-11-16T16:55:06.544 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_support", 2023-11-16T16:55:06.544 INFO:teuthology.orchestra.run.smithi093.stdout: "status", 2023-11-16T16:55:06.545 INFO:teuthology.orchestra.run.smithi093.stdout: "telemetry", 2023-11-16T16:55:06.545 INFO:teuthology.orchestra.run.smithi093.stdout: "volumes" 2023-11-16T16:55:06.545 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-11-16T16:55:06.545 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.545 INFO:teuthology.orchestra.run.smithi093.stdout: "last_failure_osd_epoch": 3, 2023-11-16T16:55:06.545 INFO:teuthology.orchestra.run.smithi093.stdout: "active_clients": [ 2023-11-16T16:55:06.545 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.546 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "devicehealth", 2023-11-16T16:55:06.546 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-11-16T16:55:06.546 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.546 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-11-16T16:55:06.546 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-11-16T16:55:06.546 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 3600267289 2023-11-16T16:55:06.546 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.547 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-11-16T16:55:06.547 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.547 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.547 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "libcephsqlite", 2023-11-16T16:55:06.547 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-11-16T16:55:06.547 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.547 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-11-16T16:55:06.548 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-11-16T16:55:06.548 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 3165408888 2023-11-16T16:55:06.548 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.548 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-11-16T16:55:06.548 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.548 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.548 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_support", 2023-11-16T16:55:06.549 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-11-16T16:55:06.549 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.549 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-11-16T16:55:06.549 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-11-16T16:55:06.549 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 4047308833 2023-11-16T16:55:06.549 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.550 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-11-16T16:55:06.550 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:06.550 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.550 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "volumes", 2023-11-16T16:55:06.550 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-11-16T16:55:06.550 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-11-16T16:55:06.550 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-11-16T16:55:06.551 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-11-16T16:55:06.551 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 1038570028 2023-11-16T16:55:06.551 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.551 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-11-16T16:55:06.551 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:06.551 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-11-16T16:55:06.551 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:55:06.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.930+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/743550330 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3e800a1150 msgr2=0x7f3e800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:06.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.930+0000 7f3e9b9ff700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3e800a1150 0x7f3e800a3600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3e94074850 tx=0x7f3e84009310 comp rx=0 tx=0).stop 2023-11-16T16:55:06.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.930+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/743550330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e94073550 msgr2=0x7f3e9419c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:06.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.930+0000 7f3e9b9ff700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e94073550 0x7f3e9419c6b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e970 tx=0x7f3e9000ec80 comp rx=0 tx=0).stop 2023-11-16T16:55:06.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.930+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/743550330 shutdown_connections 2023-11-16T16:55:06.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.930+0000 7f3e9b9ff700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f3e800a1150 0x7f3e800a3600 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:06.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.930+0000 7f3e9b9ff700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e94073550 0x7f3e9419c6b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:06.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.930+0000 7f3e9b9ff700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e9410c860 0x7f3e941a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:06.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.930+0000 7f3e9b9ff700 1 --2- 172.21.15.93:0/743550330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e94074b30 0x7f3e9419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:06.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.930+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/743550330 >> 172.21.15.93:0/743550330 conn(0x7f3e940fb870 msgr2=0x7f3e940fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:06.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.930+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/743550330 shutdown_connections 2023-11-16T16:55:06.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:54:56.930+0000 7f3e9b9ff700 1 -- 172.21.15.93:0/743550330 wait complete. 2023-11-16T16:55:06.556 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:56 smithi125 ceph-mon[131076]: pgmap v536: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2023-11-16T16:55:06.557 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:56 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3566245238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:55:06.557 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:56 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2008944368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:55:06.557 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:57 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/743550330' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-16T16:55:06.557 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:54:58 smithi125 ceph-mon[131076]: pgmap v537: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 12 op/s 2023-11-16T16:55:06.557 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:00 smithi125 ceph-mon[131076]: pgmap v538: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-11-16T16:55:06.557 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:02 smithi125 ceph-mon[131076]: pgmap v539: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2023-11-16T16:55:06.557 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:04 smithi125 ceph-mon[131076]: pgmap v540: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2023-11-16T16:55:06.558 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:56 smithi159 ceph-mon[131528]: pgmap v536: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2023-11-16T16:55:06.558 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:56 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3566245238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:55:06.558 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:56 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2008944368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:55:06.558 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:57 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/743550330' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-16T16:55:06.558 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:54:58 smithi159 ceph-mon[131528]: pgmap v537: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 12 op/s 2023-11-16T16:55:06.558 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:00 smithi159 ceph-mon[131528]: pgmap v538: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-11-16T16:55:06.559 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:02 smithi159 ceph-mon[131528]: pgmap v539: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2023-11-16T16:55:06.559 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:04 smithi159 ceph-mon[131528]: pgmap v540: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2023-11-16T16:55:06.564 DEBUG:tasks.check_counter:Getting stats from a 2023-11-16T16:55:06.564 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-11-16T16:55:06.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:57 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/743550330' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-16T16:55:06.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:54:58 smithi093 ceph-mon[97863]: pgmap v537: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 12 op/s 2023-11-16T16:55:06.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:00 smithi093 ceph-mon[97863]: pgmap v538: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-11-16T16:55:06.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:02 smithi093 ceph-mon[97863]: pgmap v539: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2023-11-16T16:55:06.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:04 smithi093 ceph-mon[97863]: pgmap v540: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2023-11-16T16:55:06.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:06 smithi093 ceph-mon[97863]: pgmap v541: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-11-16T16:55:06.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:06 smithi159 ceph-mon[131528]: pgmap v541: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-11-16T16:55:07.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:06 smithi125 ceph-mon[131076]: pgmap v541: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-11-16T16:55:07.433 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:55:08.874 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:08 smithi093 ceph-mon[97863]: pgmap v542: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-11-16T16:55:08.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:08 smithi159 ceph-mon[131528]: pgmap v542: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-11-16T16:55:09.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:08 smithi125 ceph-mon[131076]: pgmap v542: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-11-16T16:55:10.375 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:55:10.376 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:55:10.376 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 0, 2023-11-16T16:55:10.376 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 2, 2023-11-16T16:55:10.376 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 0, 2023-11-16T16:55:10.376 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 0, 2023-11-16T16:55:10.376 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 4, 2023-11-16T16:55:10.377 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 0, 2023-11-16T16:55:10.377 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.003491021, 2023-11-16T16:55:10.377 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.000000000, 2023-11-16T16:55:10.377 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.000679029, 2023-11-16T16:55:10.377 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:55:10.378 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:10.378 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:10.378 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:10.378 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:10.378 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:10.378 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:10.379 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:10.379 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:10.379 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:10.379 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:10.379 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:55:10.379 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:55:10.380 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:10.380 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:55:10.380 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 149, 2023-11-16T16:55:10.380 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 144, 2023-11-16T16:55:10.380 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 453312, 2023-11-16T16:55:10.381 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 80896, 2023-11-16T16:55:10.381 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 2, 2023-11-16T16:55:10.381 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 1, 2023-11-16T16:55:10.381 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.030476055, 2023-11-16T16:55:10.381 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.013427153, 2023-11-16T16:55:10.381 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.015978959, 2023-11-16T16:55:10.382 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.003066183, 2023-11-16T16:55:10.382 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:10.382 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 144, 2023-11-16T16:55:10.382 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.011727220, 2023-11-16T16:55:10.382 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000081439 2023-11-16T16:55:10.383 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:10.383 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:10.383 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:10.383 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:10.383 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:10.383 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:10.384 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 453312, 2023-11-16T16:55:10.384 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 80896 2023-11-16T16:55:10.384 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:10.384 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:55:10.384 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 1, 2023-11-16T16:55:10.384 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 108, 2023-11-16T16:55:10.385 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 160, 2023-11-16T16:55:10.385 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 94784, 2023-11-16T16:55:10.385 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 3, 2023-11-16T16:55:10.385 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 2, 2023-11-16T16:55:10.385 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.012154109, 2023-11-16T16:55:10.385 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.009508755, 2023-11-16T16:55:10.386 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.001181158, 2023-11-16T16:55:10.386 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:55:10.386 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:10.386 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 106, 2023-11-16T16:55:10.386 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.007620639, 2023-11-16T16:55:10.387 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000071892 2023-11-16T16:55:10.387 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:10.387 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:10.387 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:10.387 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:10.387 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:10.388 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:10.388 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-16T16:55:10.388 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 94784 2023-11-16T16:55:10.388 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:10.388 INFO:teuthology.orchestra.run.smithi093.stdout: "cct": { 2023-11-16T16:55:10.388 INFO:teuthology.orchestra.run.smithi093.stdout: "total_workers": 0, 2023-11-16T16:55:10.389 INFO:teuthology.orchestra.run.smithi093.stdout: "unhealthy_workers": 0 2023-11-16T16:55:10.389 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:10.389 INFO:teuthology.orchestra.run.smithi093.stdout: "mempool": { 2023-11-16T16:55:10.389 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:55:10.389 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_items": 0, 2023-11-16T16:55:10.390 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:55:10.390 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:55:10.390 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:55:10.390 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:55:10.390 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:55:10.390 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:55:10.391 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:55:10.391 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:55:10.391 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:55:10.391 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:55:10.391 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:55:10.391 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:55:10.392 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:55:10.392 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_items": 0, 2023-11-16T16:55:10.392 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:55:10.392 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_items": 0, 2023-11-16T16:55:10.392 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:55:10.392 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:55:10.393 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:55:10.393 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:55:10.393 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:55:10.393 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:55:10.393 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:55:10.393 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_items": 0, 2023-11-16T16:55:10.394 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:55:10.394 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:55:10.394 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:55:10.394 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_items": 0, 2023-11-16T16:55:10.394 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_bytes": 0, 2023-11-16T16:55:10.394 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_items": 0, 2023-11-16T16:55:10.395 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:55:10.395 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:55:10.395 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:55:10.395 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:55:10.395 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_bytes": 148984, 2023-11-16T16:55:10.395 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_items": 30, 2023-11-16T16:55:10.396 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:55:10.396 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_items": 0, 2023-11-16T16:55:10.396 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_bytes": 0, 2023-11-16T16:55:10.396 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_items": 0, 2023-11-16T16:55:10.396 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:55:10.396 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_items": 0, 2023-11-16T16:55:10.397 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:55:10.397 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_items": 0, 2023-11-16T16:55:10.397 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_bytes": 0, 2023-11-16T16:55:10.397 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_items": 0, 2023-11-16T16:55:10.397 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:55:10.397 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:55:10.398 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_bytes": 0, 2023-11-16T16:55:10.398 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_items": 0, 2023-11-16T16:55:10.398 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_bytes": 928, 2023-11-16T16:55:10.398 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_items": 2, 2023-11-16T16:55:10.398 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_bytes": 0, 2023-11-16T16:55:10.398 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_items": 0, 2023-11-16T16:55:10.399 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_bytes": 0, 2023-11-16T16:55:10.399 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_items": 0 2023-11-16T16:55:10.399 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:10.399 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:55:10.399 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-11-16T16:55:10.399 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-11-16T16:55:10.400 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-11-16T16:55:10.400 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 146, 2023-11-16T16:55:10.400 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 429836, 2023-11-16T16:55:10.400 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:10.400 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 146, 2023-11-16T16:55:10.400 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-11-16T16:55:10.401 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-11-16T16:55:10.401 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 146, 2023-11-16T16:55:10.401 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 429836, 2023-11-16T16:55:10.401 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-11-16T16:55:10.401 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:10.401 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:10.402 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:10.402 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:10.402 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:10.402 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:55:10.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:10 smithi093 ceph-mon[97863]: pgmap v543: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-11-16T16:55:10.928 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-11-16T16:55:10.928 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-11-16T16:55:10.929 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2023-11-16T16:55:10.929 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2023-11-16T16:55:10.929 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-11-16T16:55:10.929 DEBUG:tasks.check_counter:Getting stats from d 2023-11-16T16:55:10.929 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-11-16T16:55:10.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:10 smithi159 ceph-mon[131528]: pgmap v543: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-11-16T16:55:11.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:10 smithi125 ceph-mon[131076]: pgmap v543: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-11-16T16:55:11.778 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:55:12.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:12 smithi093 ceph-mon[97863]: pgmap v544: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-11-16T16:55:12.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:12 smithi159 ceph-mon[131528]: pgmap v544: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-11-16T16:55:13.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:12 smithi125 ceph-mon[131076]: pgmap v544: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-11-16T16:55:13.390 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:55:13.390 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:55:13.390 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 0, 2023-11-16T16:55:13.390 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 2, 2023-11-16T16:55:13.390 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 0, 2023-11-16T16:55:13.391 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 0, 2023-11-16T16:55:13.391 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 4, 2023-11-16T16:55:13.391 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 0, 2023-11-16T16:55:13.391 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.003433037, 2023-11-16T16:55:13.391 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.000000000, 2023-11-16T16:55:13.391 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.000652193, 2023-11-16T16:55:13.391 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:55:13.391 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:13.392 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:13.392 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:13.392 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:13.392 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:13.392 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:13.392 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:13.392 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:13.392 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:13.392 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:13.393 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:55:13.393 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:55:13.393 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:13.393 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:55:13.393 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 5, 2023-11-16T16:55:13.393 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 114, 2023-11-16T16:55:13.393 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 3152, 2023-11-16T16:55:13.393 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 95936, 2023-11-16T16:55:13.394 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 3, 2023-11-16T16:55:13.394 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 2, 2023-11-16T16:55:13.394 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.012984210, 2023-11-16T16:55:13.394 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.009655557, 2023-11-16T16:55:13.394 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.001907982, 2023-11-16T16:55:13.394 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:55:13.394 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:13.395 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 113, 2023-11-16T16:55:13.395 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.008821987, 2023-11-16T16:55:13.395 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000078070 2023-11-16T16:55:13.395 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:13.395 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:13.395 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:13.395 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:13.396 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:13.396 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:13.396 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-16T16:55:13.396 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 95936 2023-11-16T16:55:13.396 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:13.396 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:55:13.396 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 152, 2023-11-16T16:55:13.396 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 149, 2023-11-16T16:55:13.397 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 452848, 2023-11-16T16:55:13.397 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 84336, 2023-11-16T16:55:13.397 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 2, 2023-11-16T16:55:13.397 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 1, 2023-11-16T16:55:13.397 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.029499365, 2023-11-16T16:55:13.397 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.012209206, 2023-11-16T16:55:13.397 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.014922969, 2023-11-16T16:55:13.397 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.003561497, 2023-11-16T16:55:13.398 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:13.398 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 148, 2023-11-16T16:55:13.398 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.014699627, 2023-11-16T16:55:13.398 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000099321 2023-11-16T16:55:13.398 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:13.398 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:13.398 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:13.398 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:13.399 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:13.399 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:13.399 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 452848, 2023-11-16T16:55:13.399 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 84336 2023-11-16T16:55:13.399 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:13.399 INFO:teuthology.orchestra.run.smithi093.stdout: "cct": { 2023-11-16T16:55:13.399 INFO:teuthology.orchestra.run.smithi093.stdout: "total_workers": 0, 2023-11-16T16:55:13.399 INFO:teuthology.orchestra.run.smithi093.stdout: "unhealthy_workers": 0 2023-11-16T16:55:13.400 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:13.400 INFO:teuthology.orchestra.run.smithi093.stdout: "mempool": { 2023-11-16T16:55:13.400 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:55:13.400 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_items": 0, 2023-11-16T16:55:13.400 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:55:13.400 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:55:13.400 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:55:13.400 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:55:13.400 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:55:13.401 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:55:13.401 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:55:13.401 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:55:13.401 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:55:13.401 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:55:13.401 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:55:13.401 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:55:13.401 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:55:13.402 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_items": 0, 2023-11-16T16:55:13.402 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:55:13.402 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_items": 0, 2023-11-16T16:55:13.402 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:55:13.402 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:55:13.402 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:55:13.402 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:55:13.402 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:55:13.403 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:55:13.403 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:55:13.403 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_items": 0, 2023-11-16T16:55:13.403 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:55:13.403 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:55:13.403 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:55:13.403 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_items": 0, 2023-11-16T16:55:13.403 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_bytes": 0, 2023-11-16T16:55:13.404 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_items": 0, 2023-11-16T16:55:13.404 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:55:13.404 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:55:13.404 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:55:13.404 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:55:13.404 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_bytes": 148967, 2023-11-16T16:55:13.404 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_items": 29, 2023-11-16T16:55:13.404 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:55:13.405 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_items": 0, 2023-11-16T16:55:13.405 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_bytes": 0, 2023-11-16T16:55:13.405 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_items": 0, 2023-11-16T16:55:13.405 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:55:13.405 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_items": 0, 2023-11-16T16:55:13.405 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:55:13.405 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_items": 0, 2023-11-16T16:55:13.405 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_bytes": 0, 2023-11-16T16:55:13.406 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_items": 0, 2023-11-16T16:55:13.406 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:55:13.406 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:55:13.406 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_bytes": 0, 2023-11-16T16:55:13.406 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_items": 0, 2023-11-16T16:55:13.406 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_bytes": 928, 2023-11-16T16:55:13.406 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_items": 2, 2023-11-16T16:55:13.406 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_bytes": 0, 2023-11-16T16:55:13.407 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_items": 0, 2023-11-16T16:55:13.407 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_bytes": 0, 2023-11-16T16:55:13.407 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_items": 0 2023-11-16T16:55:13.407 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:13.407 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:55:13.407 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-11-16T16:55:13.407 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-11-16T16:55:13.407 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-11-16T16:55:13.408 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 153, 2023-11-16T16:55:13.408 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 431383, 2023-11-16T16:55:13.408 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:13.408 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 153, 2023-11-16T16:55:13.408 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-11-16T16:55:13.408 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-11-16T16:55:13.408 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 153, 2023-11-16T16:55:13.408 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 431383, 2023-11-16T16:55:13.409 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-11-16T16:55:13.409 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:13.409 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:13.409 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:13.409 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:13.409 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:13.409 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:55:13.884 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-11-16T16:55:13.885 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-11-16T16:55:13.885 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-11-16T16:55:13.885 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-11-16T16:55:13.885 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-11-16T16:55:13.885 DEBUG:tasks.check_counter:Getting stats from g 2023-11-16T16:55:13.886 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-11-16T16:55:14.729 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:55:14.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:14 smithi093 ceph-mon[97863]: pgmap v545: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-16T16:55:14.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:14 smithi159 ceph-mon[131528]: pgmap v545: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-16T16:55:15.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:14 smithi125 ceph-mon[131076]: pgmap v545: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-16T16:55:16.296 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:55:16.297 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:55:16.297 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 148, 2023-11-16T16:55:16.297 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 145, 2023-11-16T16:55:16.297 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 451136, 2023-11-16T16:55:16.297 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 82224, 2023-11-16T16:55:16.297 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 4, 2023-11-16T16:55:16.298 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 0, 2023-11-16T16:55:16.298 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.033939700, 2023-11-16T16:55:16.298 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.013510531, 2023-11-16T16:55:16.298 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.014986181, 2023-11-16T16:55:16.298 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.004181312, 2023-11-16T16:55:16.298 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:16.298 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 144, 2023-11-16T16:55:16.299 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.012603313, 2023-11-16T16:55:16.299 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000087523 2023-11-16T16:55:16.299 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:16.299 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:16.299 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:16.299 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:16.299 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:16.299 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:16.300 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 451136, 2023-11-16T16:55:16.300 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 82224 2023-11-16T16:55:16.300 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:16.300 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:55:16.300 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 1, 2023-11-16T16:55:16.300 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 110, 2023-11-16T16:55:16.300 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 160, 2023-11-16T16:55:16.300 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 95136, 2023-11-16T16:55:16.301 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 3, 2023-11-16T16:55:16.301 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 2, 2023-11-16T16:55:16.301 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.012614210, 2023-11-16T16:55:16.301 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.009039353, 2023-11-16T16:55:16.301 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.001995608, 2023-11-16T16:55:16.301 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:55:16.301 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:16.301 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 108, 2023-11-16T16:55:16.302 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.008463835, 2023-11-16T16:55:16.302 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000078368 2023-11-16T16:55:16.302 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:16.302 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:16.302 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:16.302 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:16.302 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:16.303 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:16.303 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-16T16:55:16.303 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 95136 2023-11-16T16:55:16.303 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:16.303 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:55:16.303 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 4, 2023-11-16T16:55:16.303 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 4, 2023-11-16T16:55:16.303 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 2992, 2023-11-16T16:55:16.304 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 448, 2023-11-16T16:55:16.304 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 2, 2023-11-16T16:55:16.304 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 1, 2023-11-16T16:55:16.304 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.001364348, 2023-11-16T16:55:16.304 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.000076851, 2023-11-16T16:55:16.304 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.001213688, 2023-11-16T16:55:16.304 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:55:16.304 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:16.305 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 3, 2023-11-16T16:55:16.305 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.001232208, 2023-11-16T16:55:16.305 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000410736 2023-11-16T16:55:16.305 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:16.305 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:16.305 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:16.305 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:16.305 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:16.306 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:16.306 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-16T16:55:16.306 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 448 2023-11-16T16:55:16.306 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:16.306 INFO:teuthology.orchestra.run.smithi093.stdout: "cct": { 2023-11-16T16:55:16.306 INFO:teuthology.orchestra.run.smithi093.stdout: "total_workers": 0, 2023-11-16T16:55:16.306 INFO:teuthology.orchestra.run.smithi093.stdout: "unhealthy_workers": 0 2023-11-16T16:55:16.306 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:16.307 INFO:teuthology.orchestra.run.smithi093.stdout: "mempool": { 2023-11-16T16:55:16.307 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:55:16.307 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_items": 0, 2023-11-16T16:55:16.307 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:55:16.307 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:55:16.307 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:55:16.307 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:55:16.307 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:55:16.308 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:55:16.308 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:55:16.308 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:55:16.308 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:55:16.308 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:55:16.308 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:55:16.308 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:55:16.308 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:55:16.309 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_items": 0, 2023-11-16T16:55:16.309 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:55:16.309 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_items": 0, 2023-11-16T16:55:16.309 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:55:16.309 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:55:16.309 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:55:16.309 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:55:16.309 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:55:16.310 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:55:16.310 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:55:16.310 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_items": 0, 2023-11-16T16:55:16.310 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:55:16.310 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:55:16.310 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:55:16.310 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_items": 0, 2023-11-16T16:55:16.310 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_bytes": 0, 2023-11-16T16:55:16.311 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_items": 0, 2023-11-16T16:55:16.311 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:55:16.311 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:55:16.311 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:55:16.311 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:55:16.311 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_bytes": 148967, 2023-11-16T16:55:16.311 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_items": 29, 2023-11-16T16:55:16.312 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:55:16.312 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_items": 0, 2023-11-16T16:55:16.312 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_bytes": 0, 2023-11-16T16:55:16.312 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_items": 0, 2023-11-16T16:55:16.312 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:55:16.312 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_items": 0, 2023-11-16T16:55:16.312 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:55:16.312 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_items": 0, 2023-11-16T16:55:16.313 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_bytes": 0, 2023-11-16T16:55:16.313 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_items": 0, 2023-11-16T16:55:16.313 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:55:16.313 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:55:16.313 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_bytes": 0, 2023-11-16T16:55:16.313 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_items": 0, 2023-11-16T16:55:16.313 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_bytes": 928, 2023-11-16T16:55:16.313 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_items": 2, 2023-11-16T16:55:16.313 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_bytes": 0, 2023-11-16T16:55:16.314 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_items": 0, 2023-11-16T16:55:16.314 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_bytes": 0, 2023-11-16T16:55:16.314 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_items": 0 2023-11-16T16:55:16.314 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:16.314 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:55:16.314 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-11-16T16:55:16.314 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-11-16T16:55:16.315 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-11-16T16:55:16.315 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 149, 2023-11-16T16:55:16.315 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 430226, 2023-11-16T16:55:16.315 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:16.315 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 149, 2023-11-16T16:55:16.315 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-11-16T16:55:16.315 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-11-16T16:55:16.315 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 149, 2023-11-16T16:55:16.316 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 430226, 2023-11-16T16:55:16.316 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-11-16T16:55:16.316 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:16.316 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:16.316 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:16.316 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:16.316 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:16.316 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:55:16.826 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-11-16T16:55:16.826 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-11-16T16:55:16.826 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-11-16T16:55:16.827 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-11-16T16:55:16.827 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-11-16T16:55:16.827 DEBUG:tasks.check_counter:Getting stats from j 2023-11-16T16:55:16.827 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-11-16T16:55:16.890 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:16 smithi093 ceph-mon[97863]: pgmap v546: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:55:16.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:16 smithi159 ceph-mon[131528]: pgmap v546: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:55:17.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:16 smithi125 ceph-mon[131076]: pgmap v546: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:55:17.676 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:55:18.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:18 smithi093 ceph-mon[97863]: pgmap v547: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-16T16:55:18.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:18 smithi159 ceph-mon[131528]: pgmap v547: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-16T16:55:19.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:18 smithi125 ceph-mon[131076]: pgmap v547: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-16T16:55:19.245 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-16T16:55:19.245 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:55:19.245 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 0, 2023-11-16T16:55:19.245 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 2, 2023-11-16T16:55:19.246 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 0, 2023-11-16T16:55:19.246 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 0, 2023-11-16T16:55:19.246 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 4, 2023-11-16T16:55:19.246 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 0, 2023-11-16T16:55:19.247 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.002942165, 2023-11-16T16:55:19.247 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.000000000, 2023-11-16T16:55:19.247 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.000504103, 2023-11-16T16:55:19.247 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:55:19.247 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:19.248 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:19.248 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:19.248 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:19.248 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:19.249 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:19.249 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:19.249 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:19.249 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:19.249 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:19.250 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:55:19.250 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:55:19.250 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:19.250 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:55:19.251 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 154, 2023-11-16T16:55:19.251 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 151, 2023-11-16T16:55:19.251 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 453392, 2023-11-16T16:55:19.251 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 85520, 2023-11-16T16:55:19.251 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 2, 2023-11-16T16:55:19.252 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 1, 2023-11-16T16:55:19.252 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.032483000, 2023-11-16T16:55:19.252 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.013990910, 2023-11-16T16:55:19.252 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.018473644, 2023-11-16T16:55:19.252 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.003289054, 2023-11-16T16:55:19.252 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:19.253 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 150, 2023-11-16T16:55:19.253 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.009383072, 2023-11-16T16:55:19.253 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000062553 2023-11-16T16:55:19.253 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:19.253 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:19.254 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:19.254 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:19.254 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:19.254 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:19.254 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 453392, 2023-11-16T16:55:19.255 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 85520 2023-11-16T16:55:19.255 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:19.255 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:55:19.255 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 5, 2023-11-16T16:55:19.255 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 116, 2023-11-16T16:55:19.256 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 3152, 2023-11-16T16:55:19.256 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 96288, 2023-11-16T16:55:19.256 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 3, 2023-11-16T16:55:19.256 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 2, 2023-11-16T16:55:19.257 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.011870515, 2023-11-16T16:55:19.257 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.008573688, 2023-11-16T16:55:19.257 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.001837479, 2023-11-16T16:55:19.257 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:55:19.257 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:19.258 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 115, 2023-11-16T16:55:19.258 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.009508303, 2023-11-16T16:55:19.258 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000082680 2023-11-16T16:55:19.258 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:19.258 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:19.259 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:19.259 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:19.259 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:19.259 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:19.259 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-16T16:55:19.260 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 96288 2023-11-16T16:55:19.260 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:19.260 INFO:teuthology.orchestra.run.smithi093.stdout: "cct": { 2023-11-16T16:55:19.260 INFO:teuthology.orchestra.run.smithi093.stdout: "total_workers": 0, 2023-11-16T16:55:19.260 INFO:teuthology.orchestra.run.smithi093.stdout: "unhealthy_workers": 0 2023-11-16T16:55:19.260 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:19.261 INFO:teuthology.orchestra.run.smithi093.stdout: "mempool": { 2023-11-16T16:55:19.261 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:55:19.261 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_items": 0, 2023-11-16T16:55:19.261 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:55:19.261 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:55:19.262 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:55:19.262 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:55:19.262 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:55:19.262 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:55:19.262 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:55:19.263 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:55:19.263 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:55:19.263 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:55:19.263 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:55:19.263 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:55:19.263 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:55:19.264 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_items": 0, 2023-11-16T16:55:19.264 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:55:19.264 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_items": 0, 2023-11-16T16:55:19.264 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:55:19.264 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:55:19.265 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:55:19.265 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:55:19.265 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:55:19.265 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:55:19.265 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:55:19.265 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_items": 0, 2023-11-16T16:55:19.266 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:55:19.266 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:55:19.266 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:55:19.266 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_items": 0, 2023-11-16T16:55:19.266 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_bytes": 0, 2023-11-16T16:55:19.267 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_items": 0, 2023-11-16T16:55:19.267 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:55:19.267 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:55:19.267 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:55:19.267 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:55:19.267 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_bytes": 148984, 2023-11-16T16:55:19.268 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_items": 30, 2023-11-16T16:55:19.268 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:55:19.268 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_items": 0, 2023-11-16T16:55:19.268 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_bytes": 0, 2023-11-16T16:55:19.268 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_items": 0, 2023-11-16T16:55:19.269 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:55:19.269 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_items": 0, 2023-11-16T16:55:19.269 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:55:19.269 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_items": 0, 2023-11-16T16:55:19.269 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_bytes": 0, 2023-11-16T16:55:19.269 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_items": 0, 2023-11-16T16:55:19.270 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:55:19.270 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:55:19.270 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_bytes": 0, 2023-11-16T16:55:19.270 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_items": 0, 2023-11-16T16:55:19.270 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_bytes": 928, 2023-11-16T16:55:19.271 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_items": 2, 2023-11-16T16:55:19.271 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_bytes": 0, 2023-11-16T16:55:19.271 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_items": 0, 2023-11-16T16:55:19.271 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_bytes": 0, 2023-11-16T16:55:19.271 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_items": 0 2023-11-16T16:55:19.272 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-16T16:55:19.272 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:55:19.272 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-11-16T16:55:19.272 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-11-16T16:55:19.272 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-11-16T16:55:19.272 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 155, 2023-11-16T16:55:19.273 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 431643, 2023-11-16T16:55:19.273 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:19.273 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 155, 2023-11-16T16:55:19.273 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-11-16T16:55:19.273 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-11-16T16:55:19.273 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 155, 2023-11-16T16:55:19.274 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 431643, 2023-11-16T16:55:19.274 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-11-16T16:55:19.274 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-11-16T16:55:19.274 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-11-16T16:55:19.274 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-11-16T16:55:19.275 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:19.275 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-16T16:55:19.275 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-16T16:55:19.771 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-11-16T16:55:19.771 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-11-16T16:55:19.771 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-11-16T16:55:19.772 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-11-16T16:55:19.772 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-11-16T16:55:19.772 DEBUG:tasks.check_counter:Getting stats from b 2023-11-16T16:55:19.772 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-11-16T16:55:20.430 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:55:20.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:20 smithi093 ceph-mon[97863]: pgmap v548: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-16T16:55:20.931 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:20 smithi125 ceph-mon[131076]: pgmap v548: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-16T16:55:20.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:20 smithi159 ceph-mon[131528]: pgmap v548: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-16T16:55:21.700 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-11-16T16:55:21.700 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:55:21.700 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 36489, 2023-11-16T16:55:21.700 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 33149, 2023-11-16T16:55:21.700 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 16055335, 2023-11-16T16:55:21.700 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 130116154, 2023-11-16T16:55:21.701 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 56, 2023-11-16T16:55:21.701 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 8, 2023-11-16T16:55:21.701 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 2.104264408, 2023-11-16T16:55:21.701 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.748545292, 2023-11-16T16:55:21.701 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.699305562, 2023-11-16T16:55:21.701 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.712215342, 2023-11-16T16:55:21.701 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:21.701 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 33149, 2023-11-16T16:55:21.702 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 43.673535587, 2023-11-16T16:55:21.702 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.001317491 2023-11-16T16:55:21.702 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.702 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:21.702 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 21718, 2023-11-16T16:55:21.702 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.003238421, 2023-11-16T16:55:21.702 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000149 2023-11-16T16:55:21.703 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.703 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 525328, 2023-11-16T16:55:21.703 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 91040 2023-11-16T16:55:21.703 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.703 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:55:21.703 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 102357, 2023-11-16T16:55:21.703 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 196908, 2023-11-16T16:55:21.704 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 40018916, 2023-11-16T16:55:21.704 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 167201092, 2023-11-16T16:55:21.704 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 12, 2023-11-16T16:55:21.704 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 5, 2023-11-16T16:55:21.704 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 7.137147578, 2023-11-16T16:55:21.704 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 2.968211027, 2023-11-16T16:55:21.704 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 3.970591198, 2023-11-16T16:55:21.704 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 1.119613379, 2023-11-16T16:55:21.705 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:21.705 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 196906, 2023-11-16T16:55:21.705 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 103.000361410, 2023-11-16T16:55:21.705 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000523094 2023-11-16T16:55:21.705 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.705 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:21.705 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 186627, 2023-11-16T16:55:21.705 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.032006002, 2023-11-16T16:55:21.706 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000171 2023-11-16T16:55:21.706 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.706 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 29135600, 2023-11-16T16:55:21.706 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 93460416 2023-11-16T16:55:21.706 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.706 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:55:21.706 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 42274, 2023-11-16T16:55:21.706 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 46945, 2023-11-16T16:55:21.707 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 22719335, 2023-11-16T16:55:21.707 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 123008072, 2023-11-16T16:55:21.707 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 13, 2023-11-16T16:55:21.707 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 6, 2023-11-16T16:55:21.707 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 2.457893475, 2023-11-16T16:55:21.707 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.786530172, 2023-11-16T16:55:21.707 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.718740200, 2023-11-16T16:55:21.708 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.942654934, 2023-11-16T16:55:21.708 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:21.708 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 46943, 2023-11-16T16:55:21.708 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 55.898493551, 2023-11-16T16:55:21.708 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.001190773 2023-11-16T16:55:21.708 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.708 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:21.708 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 21665, 2023-11-16T16:55:21.709 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.004416526, 2023-11-16T16:55:21.709 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000203 2023-11-16T16:55:21.709 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.709 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:55:21.709 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:55:21.709 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.709 INFO:teuthology.orchestra.run.smithi125.stdout: "cct": { 2023-11-16T16:55:21.710 INFO:teuthology.orchestra.run.smithi125.stdout: "total_workers": 1, 2023-11-16T16:55:21.710 INFO:teuthology.orchestra.run.smithi125.stdout: "unhealthy_workers": 0 2023-11-16T16:55:21.710 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.710 INFO:teuthology.orchestra.run.smithi125.stdout: "finisher-MDSRank": { 2023-11-16T16:55:21.710 INFO:teuthology.orchestra.run.smithi125.stdout: "queue_len": 0, 2023-11-16T16:55:21.710 INFO:teuthology.orchestra.run.smithi125.stdout: "complete_latency": { 2023-11-16T16:55:21.710 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 4105, 2023-11-16T16:55:21.710 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 52.137023289, 2023-11-16T16:55:21.711 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.012700858 2023-11-16T16:55:21.711 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:21.711 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.711 INFO:teuthology.orchestra.run.smithi125.stdout: "finisher-PurgeQueue": { 2023-11-16T16:55:21.711 INFO:teuthology.orchestra.run.smithi125.stdout: "queue_len": 0, 2023-11-16T16:55:21.711 INFO:teuthology.orchestra.run.smithi125.stdout: "complete_latency": { 2023-11-16T16:55:21.711 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 23976, 2023-11-16T16:55:21.712 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 5.676831073, 2023-11-16T16:55:21.712 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000236771 2023-11-16T16:55:21.712 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:21.712 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.712 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": { 2023-11-16T16:55:21.712 INFO:teuthology.orchestra.run.smithi125.stdout: "request": 57484, 2023-11-16T16:55:21.712 INFO:teuthology.orchestra.run.smithi125.stdout: "reply": 48111, 2023-11-16T16:55:21.712 INFO:teuthology.orchestra.run.smithi125.stdout: "reply_latency": { 2023-11-16T16:55:21.713 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 48111, 2023-11-16T16:55:21.713 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 803.878068720, 2023-11-16T16:55:21.713 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.016708820 2023-11-16T16:55:21.713 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.713 INFO:teuthology.orchestra.run.smithi125.stdout: "slow_reply": 0, 2023-11-16T16:55:21.713 INFO:teuthology.orchestra.run.smithi125.stdout: "forward": 9373, 2023-11-16T16:55:21.713 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_fetch_complete": 336, 2023-11-16T16:55:21.713 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_fetch_keys": 0, 2023-11-16T16:55:21.714 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_commit": 6070, 2023-11-16T16:55:21.714 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_split": 127, 2023-11-16T16:55:21.714 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_merge": 41, 2023-11-16T16:55:21.714 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes": 65, 2023-11-16T16:55:21.714 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_top": 28, 2023-11-16T16:55:21.714 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_bottom": 22, 2023-11-16T16:55:21.714 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_pin_tail": 15, 2023-11-16T16:55:21.714 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_pinned": 24, 2023-11-16T16:55:21.714 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_expired": 31852, 2023-11-16T16:55:21.715 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_with_caps": 4, 2023-11-16T16:55:21.715 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": 4, 2023-11-16T16:55:21.715 INFO:teuthology.orchestra.run.smithi125.stdout: "subtrees": 4, 2023-11-16T16:55:21.715 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse": 67836, 2023-11-16T16:55:21.715 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_hit": 57539, 2023-11-16T16:55:21.715 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_forward": 8054, 2023-11-16T16:55:21.715 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_discover": 0, 2023-11-16T16:55:21.716 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_dir_fetch": 294, 2023-11-16T16:55:21.716 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_remote_ino": 0, 2023-11-16T16:55:21.716 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_lock": 0, 2023-11-16T16:55:21.716 INFO:teuthology.orchestra.run.smithi125.stdout: "load_cent": 245, 2023-11-16T16:55:21.716 INFO:teuthology.orchestra.run.smithi125.stdout: "q": 0, 2023-11-16T16:55:21.716 INFO:teuthology.orchestra.run.smithi125.stdout: "exported": 446, 2023-11-16T16:55:21.716 INFO:teuthology.orchestra.run.smithi125.stdout: "exported_inodes": 25594, 2023-11-16T16:55:21.716 INFO:teuthology.orchestra.run.smithi125.stdout: "imported": 768, 2023-11-16T16:55:21.717 INFO:teuthology.orchestra.run.smithi125.stdout: "imported_inodes": 0, 2023-11-16T16:55:21.717 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_dir_fetch": 0, 2023-11-16T16:55:21.717 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_backtrace_fetch": 0, 2023-11-16T16:55:21.717 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_peer_discover": 0, 2023-11-16T16:55:21.717 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rfiles": 4, 2023-11-16T16:55:21.717 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rbytes": 246, 2023-11-16T16:55:21.717 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rsnaps": 0, 2023-11-16T16:55:21.717 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_backtrace_fetch": 0, 2023-11-16T16:55:21.717 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_set_tag": 0, 2023-11-16T16:55:21.718 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_backtrace_repaired": 0, 2023-11-16T16:55:21.718 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_inotable_repaired": 0, 2023-11-16T16:55:21.718 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dir_inodes": 0, 2023-11-16T16:55:21.718 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dir_base_inodes": 0, 2023-11-16T16:55:21.718 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dirfrag_rstats": 0, 2023-11-16T16:55:21.718 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_file_inodes": 0, 2023-11-16T16:55:21.718 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_inode_file_caps": 6, 2023-11-16T16:55:21.718 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_revoke": 145, 2023-11-16T16:55:21.719 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_grant": 37533, 2023-11-16T16:55:21.719 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_trunc": 1, 2023-11-16T16:55:21.719 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-16T16:55:21.719 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-16T16:55:21.719 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_caps": 15045, 2023-11-16T16:55:21.719 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_caps_dirty": 14661, 2023-11-16T16:55:21.719 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_cap_release": 380, 2023-11-16T16:55:21.719 INFO:teuthology.orchestra.run.smithi125.stdout: "process_request_cap_release": 23891 2023-11-16T16:55:21.720 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.720 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_cache": { 2023-11-16T16:55:21.720 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_update": 1191, 2023-11-16T16:55:21.720 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_update_receipt": 0, 2023-11-16T16:55:21.720 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_try_discover": 0, 2023-11-16T16:55:21.720 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_send_discover": 0, 2023-11-16T16:55:21.720 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_handle_discover": 1737, 2023-11-16T16:55:21.720 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays": 1, 2023-11-16T16:55:21.721 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays_delayed": 0, 2023-11-16T16:55:21.721 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays_enqueuing": 0, 2023-11-16T16:55:21.721 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_created": 13570, 2023-11-16T16:55:21.721 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_enqueued": 13569, 2023-11-16T16:55:21.721 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_reintegrated": 0, 2023-11-16T16:55:21.721 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_migrated": 0, 2023-11-16T16:55:21.721 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_processing": 0, 2023-11-16T16:55:21.722 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_enqueued": 0, 2023-11-16T16:55:21.722 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_prioritized": 0, 2023-11-16T16:55:21.722 INFO:teuthology.orchestra.run.smithi125.stdout: "recovery_started": 0, 2023-11-16T16:55:21.722 INFO:teuthology.orchestra.run.smithi125.stdout: "recovery_completed": 0, 2023-11-16T16:55:21.722 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_enqueue_scrub": 0, 2023-11-16T16:55:21.722 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_exportdir": 693, 2023-11-16T16:55:21.722 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_flush": 0, 2023-11-16T16:55:21.722 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_fragmentdir": 172, 2023-11-16T16:55:21.723 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_fragstats": 0, 2023-11-16T16:55:21.723 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_inodestats": 0 2023-11-16T16:55:21.723 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.723 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_log": { 2023-11-16T16:55:21.723 INFO:teuthology.orchestra.run.smithi125.stdout: "evadd": 71891, 2023-11-16T16:55:21.723 INFO:teuthology.orchestra.run.smithi125.stdout: "evex": 71850, 2023-11-16T16:55:21.723 INFO:teuthology.orchestra.run.smithi125.stdout: "evtrm": 71850, 2023-11-16T16:55:21.724 INFO:teuthology.orchestra.run.smithi125.stdout: "ev": 41, 2023-11-16T16:55:21.724 INFO:teuthology.orchestra.run.smithi125.stdout: "evexg": 0, 2023-11-16T16:55:21.724 INFO:teuthology.orchestra.run.smithi125.stdout: "evexd": 127, 2023-11-16T16:55:21.724 INFO:teuthology.orchestra.run.smithi125.stdout: "segadd": 114, 2023-11-16T16:55:21.724 INFO:teuthology.orchestra.run.smithi125.stdout: "segex": 113, 2023-11-16T16:55:21.724 INFO:teuthology.orchestra.run.smithi125.stdout: "segtrm": 113, 2023-11-16T16:55:21.724 INFO:teuthology.orchestra.run.smithi125.stdout: "seg": 2, 2023-11-16T16:55:21.724 INFO:teuthology.orchestra.run.smithi125.stdout: "segexg": 0, 2023-11-16T16:55:21.724 INFO:teuthology.orchestra.run.smithi125.stdout: "segexd": 1, 2023-11-16T16:55:21.725 INFO:teuthology.orchestra.run.smithi125.stdout: "expos": 196069747, 2023-11-16T16:55:21.725 INFO:teuthology.orchestra.run.smithi125.stdout: "wrpos": 196145290, 2023-11-16T16:55:21.725 INFO:teuthology.orchestra.run.smithi125.stdout: "rdpos": 0, 2023-11-16T16:55:21.725 INFO:teuthology.orchestra.run.smithi125.stdout: "jlat": { 2023-11-16T16:55:21.725 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 2351, 2023-11-16T16:55:21.725 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 148.486535140, 2023-11-16T16:55:21.725 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.063158883 2023-11-16T16:55:21.725 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.726 INFO:teuthology.orchestra.run.smithi125.stdout: "replayed": 0 2023-11-16T16:55:21.726 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.726 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_mem": { 2023-11-16T16:55:21.726 INFO:teuthology.orchestra.run.smithi125.stdout: "ino": 26, 2023-11-16T16:55:21.726 INFO:teuthology.orchestra.run.smithi125.stdout: "ino+": 35028, 2023-11-16T16:55:21.726 INFO:teuthology.orchestra.run.smithi125.stdout: "ino-": 35002, 2023-11-16T16:55:21.726 INFO:teuthology.orchestra.run.smithi125.stdout: "dir": 88, 2023-11-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi125.stdout: "dir+": 3224, 2023-11-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi125.stdout: "dir-": 3136, 2023-11-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi125.stdout: "dn": 65, 2023-11-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi125.stdout: "dn+": 62228, 2023-11-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi125.stdout: "dn-": 62163, 2023-11-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi125.stdout: "cap": 4, 2023-11-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi125.stdout: "cap+": 47487, 2023-11-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi125.stdout: "cap-": 47483, 2023-11-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi125.stdout: "rss": 179600, 2023-11-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi125.stdout: "heap": 207156 2023-11-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_server": { 2023-11-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi125.stdout: "dispatch_client_request": 59656, 2023-11-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi125.stdout: "dispatch_server_request": 203, 2023-11-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_request": 57484, 2023-11-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_session": 252, 2023-11-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_peer_request": 460, 2023-11-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi125.stdout: "req_create_latency": { 2023-11-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 23971, 2023-11-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 736.297809837, 2023-11-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.030716190 2023-11-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getattr_latency": { 2023-11-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 52, 2023-11-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.026949399, 2023-11-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000518257 2023-11-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getfilelock_latency": { 2023-11-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi125.stdout: "req_link_latency": { 2023-11-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookup_latency": { 2023-11-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 9577, 2023-11-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 12.507108408, 2023-11-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.001305952 2023-11-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookuphash_latency": { 2023-11-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupino_latency": { 2023-11-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.733 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.733 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupname_latency": { 2023-11-16T16:55:21.733 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.733 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.733 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.733 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.733 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupparent_latency": { 2023-11-16T16:55:21.733 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.733 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.734 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.734 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.734 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupsnap_latency": { 2023-11-16T16:55:21.734 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.734 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.734 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.734 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.734 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lssnap_latency": { 2023-11-16T16:55:21.735 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.735 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.735 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.735 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.735 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mkdir_latency": { 2023-11-16T16:55:21.735 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 125, 2023-11-16T16:55:21.735 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.119745263, 2023-11-16T16:55:21.735 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000957962 2023-11-16T16:55:21.736 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.736 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mknod_latency": { 2023-11-16T16:55:21.736 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.736 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.736 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.736 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.736 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mksnap_latency": { 2023-11-16T16:55:21.736 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.736 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.737 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.737 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.737 INFO:teuthology.orchestra.run.smithi125.stdout: "req_open_latency": { 2023-11-16T16:55:21.737 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.737 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.737 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.737 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.738 INFO:teuthology.orchestra.run.smithi125.stdout: "req_readdir_latency": { 2023-11-16T16:55:21.738 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 769, 2023-11-16T16:55:21.738 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 7.524873266, 2023-11-16T16:55:21.738 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.009785270 2023-11-16T16:55:21.738 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.738 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rename_latency": { 2023-11-16T16:55:21.738 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 2, 2023-11-16T16:55:21.738 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.002587867, 2023-11-16T16:55:21.739 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.001293933 2023-11-16T16:55:21.739 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.739 INFO:teuthology.orchestra.run.smithi125.stdout: "req_renamesnap_latency": { 2023-11-16T16:55:21.739 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.739 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.739 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.739 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.739 INFO:teuthology.orchestra.run.smithi125.stdout: "req_snapdiff_latency": { 2023-11-16T16:55:21.740 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.740 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.740 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.740 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.740 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmdir_latency": { 2023-11-16T16:55:21.740 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 113, 2023-11-16T16:55:21.740 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.324431049, 2023-11-16T16:55:21.740 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.002871071 2023-11-16T16:55:21.741 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.741 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmsnap_latency": { 2023-11-16T16:55:21.741 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.741 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.741 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.741 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.741 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmxattr_latency": { 2023-11-16T16:55:21.741 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 3, 2023-11-16T16:55:21.742 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.002012131, 2023-11-16T16:55:21.742 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000670710 2023-11-16T16:55:21.742 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.742 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setattr_latency": { 2023-11-16T16:55:21.742 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 4, 2023-11-16T16:55:21.742 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.013905793, 2023-11-16T16:55:21.742 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.003476448 2023-11-16T16:55:21.742 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.742 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setdirlayout_latency": { 2023-11-16T16:55:21.743 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.743 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.743 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.743 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.743 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setfilelock_latency": { 2023-11-16T16:55:21.743 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 22, 2023-11-16T16:55:21.743 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 10.016841844, 2023-11-16T16:55:21.743 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.455310992 2023-11-16T16:55:21.744 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.744 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setlayout_latency": { 2023-11-16T16:55:21.744 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.744 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.744 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.744 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.744 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setxattr_latency": { 2023-11-16T16:55:21.744 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 16, 2023-11-16T16:55:21.744 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.033221977, 2023-11-16T16:55:21.745 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.002076373 2023-11-16T16:55:21.745 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.745 INFO:teuthology.orchestra.run.smithi125.stdout: "req_symlink_latency": { 2023-11-16T16:55:21.745 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.745 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.745 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.745 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.745 INFO:teuthology.orchestra.run.smithi125.stdout: "req_unlink_latency": { 2023-11-16T16:55:21.746 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 13457, 2023-11-16T16:55:21.746 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 37.008581886, 2023-11-16T16:55:21.746 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.002750136 2023-11-16T16:55:21.746 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.746 INFO:teuthology.orchestra.run.smithi125.stdout: "cap_revoke_eviction": 0, 2023-11-16T16:55:21.746 INFO:teuthology.orchestra.run.smithi125.stdout: "cap_acquisition_throttle": 0, 2023-11-16T16:55:21.746 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getvxattr_latency": { 2023-11-16T16:55:21.746 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.747 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.747 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.747 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:21.747 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.747 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_sessions": { 2023-11-16T16:55:21.747 INFO:teuthology.orchestra.run.smithi125.stdout: "session_count": 1, 2023-11-16T16:55:21.747 INFO:teuthology.orchestra.run.smithi125.stdout: "session_add": 3, 2023-11-16T16:55:21.747 INFO:teuthology.orchestra.run.smithi125.stdout: "session_remove": 2, 2023-11-16T16:55:21.747 INFO:teuthology.orchestra.run.smithi125.stdout: "sessions_open": 1, 2023-11-16T16:55:21.748 INFO:teuthology.orchestra.run.smithi125.stdout: "sessions_stale": 0, 2023-11-16T16:55:21.748 INFO:teuthology.orchestra.run.smithi125.stdout: "total_load": 11787, 2023-11-16T16:55:21.748 INFO:teuthology.orchestra.run.smithi125.stdout: "average_load": 11787, 2023-11-16T16:55:21.748 INFO:teuthology.orchestra.run.smithi125.stdout: "avg_session_uptime": 391 2023-11-16T16:55:21.748 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.748 INFO:teuthology.orchestra.run.smithi125.stdout: "mempool": { 2023-11-16T16:55:21.748 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_bytes": 360, 2023-11-16T16:55:21.748 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_items": 360, 2023-11-16T16:55:21.749 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:55:21.749 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:55:21.749 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:55:21.749 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:55:21.749 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:55:21.749 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:55:21.749 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:55:21.749 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:55:21.750 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:55:21.750 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:55:21.750 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:55:21.750 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:55:21.750 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:55:21.750 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_items": 0, 2023-11-16T16:55:21.750 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:55:21.750 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_items": 0, 2023-11-16T16:55:21.751 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:55:21.751 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:55:21.751 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:55:21.751 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:55:21.751 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:55:21.751 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:55:21.751 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:55:21.751 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_items": 0, 2023-11-16T16:55:21.751 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:55:21.752 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:55:21.752 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:55:21.752 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_items": 0, 2023-11-16T16:55:21.752 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_bytes": 0, 2023-11-16T16:55:21.752 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_items": 0, 2023-11-16T16:55:21.752 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:55:21.752 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:55:21.752 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:55:21.753 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:55:21.753 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_bytes": 292351, 2023-11-16T16:55:21.753 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_items": 131, 2023-11-16T16:55:21.753 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_bytes": 616, 2023-11-16T16:55:21.753 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_items": 7, 2023-11-16T16:55:21.753 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_bytes": 0, 2023-11-16T16:55:21.753 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_items": 0, 2023-11-16T16:55:21.753 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:55:21.754 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_items": 0, 2023-11-16T16:55:21.754 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:55:21.754 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_items": 0, 2023-11-16T16:55:21.754 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_bytes": 9280, 2023-11-16T16:55:21.754 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_items": 179, 2023-11-16T16:55:21.754 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:55:21.754 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:55:21.754 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_bytes": 0, 2023-11-16T16:55:21.755 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_items": 0, 2023-11-16T16:55:21.755 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_bytes": 235116, 2023-11-16T16:55:21.755 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_items": 1853, 2023-11-16T16:55:21.755 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_bytes": 0, 2023-11-16T16:55:21.755 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_items": 0, 2023-11-16T16:55:21.755 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_bytes": 0, 2023-11-16T16:55:21.755 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_items": 0 2023-11-16T16:55:21.755 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.756 INFO:teuthology.orchestra.run.smithi125.stdout: "objecter": { 2023-11-16T16:55:21.756 INFO:teuthology.orchestra.run.smithi125.stdout: "op_active": 0, 2023-11-16T16:55:21.756 INFO:teuthology.orchestra.run.smithi125.stdout: "op_laggy": 0, 2023-11-16T16:55:21.756 INFO:teuthology.orchestra.run.smithi125.stdout: "op_send": 84716, 2023-11-16T16:55:21.756 INFO:teuthology.orchestra.run.smithi125.stdout: "op_send_bytes": 244286182, 2023-11-16T16:55:21.756 INFO:teuthology.orchestra.run.smithi125.stdout: "op_resend": 37, 2023-11-16T16:55:21.756 INFO:teuthology.orchestra.run.smithi125.stdout: "op_reply": 84716, 2023-11-16T16:55:21.756 INFO:teuthology.orchestra.run.smithi125.stdout: "op_latency": { 2023-11-16T16:55:21.756 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 84716, 2023-11-16T16:55:21.757 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 38161.908391743, 2023-11-16T16:55:21.757 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.450468723 2023-11-16T16:55:21.757 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.757 INFO:teuthology.orchestra.run.smithi125.stdout: "op_inflight": 0, 2023-11-16T16:55:21.757 INFO:teuthology.orchestra.run.smithi125.stdout: "oplen_avg": { 2023-11-16T16:55:21.757 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 84716, 2023-11-16T16:55:21.757 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 164290 2023-11-16T16:55:21.757 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.758 INFO:teuthology.orchestra.run.smithi125.stdout: "op": 84716, 2023-11-16T16:55:21.758 INFO:teuthology.orchestra.run.smithi125.stdout: "op_r": 675, 2023-11-16T16:55:21.758 INFO:teuthology.orchestra.run.smithi125.stdout: "op_w": 84041, 2023-11-16T16:55:21.758 INFO:teuthology.orchestra.run.smithi125.stdout: "op_rmw": 0, 2023-11-16T16:55:21.758 INFO:teuthology.orchestra.run.smithi125.stdout: "op_pg": 0, 2023-11-16T16:55:21.758 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_stat": 4979, 2023-11-16T16:55:21.758 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_create": 46162, 2023-11-16T16:55:21.758 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_read": 339, 2023-11-16T16:55:21.758 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_write": 2690, 2023-11-16T16:55:21.759 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_writefull": 258, 2023-11-16T16:55:21.759 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_writesame": 0, 2023-11-16T16:55:21.759 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_append": 0, 2023-11-16T16:55:21.759 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_zero": 0, 2023-11-16T16:55:21.759 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_truncate": 5, 2023-11-16T16:55:21.759 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_delete": 28750, 2023-11-16T16:55:21.759 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_mapext": 0, 2023-11-16T16:55:21.760 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_sparse_read": 0, 2023-11-16T16:55:21.760 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_clonerange": 0, 2023-11-16T16:55:21.760 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_getxattr": 0, 2023-11-16T16:55:21.760 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_setxattr": 69715, 2023-11-16T16:55:21.760 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_cmpxattr": 0, 2023-11-16T16:55:21.760 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_rmxattr": 0, 2023-11-16T16:55:21.760 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_resetxattrs": 0, 2023-11-16T16:55:21.760 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_call": 0, 2023-11-16T16:55:21.761 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_watch": 0, 2023-11-16T16:55:21.761 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_notify": 0, 2023-11-16T16:55:21.761 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_src_cmpxattr": 0, 2023-11-16T16:55:21.761 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_pgls": 0, 2023-11-16T16:55:21.761 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_pgls_filter": 0, 2023-11-16T16:55:21.761 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_other": 1, 2023-11-16T16:55:21.761 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_active": 0, 2023-11-16T16:55:21.761 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_send": 0, 2023-11-16T16:55:21.761 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_resend": 0, 2023-11-16T16:55:21.762 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_ping": 0, 2023-11-16T16:55:21.762 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_active": 0, 2023-11-16T16:55:21.762 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_send": 0, 2023-11-16T16:55:21.762 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_resend": 0, 2023-11-16T16:55:21.762 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_active": 0, 2023-11-16T16:55:21.762 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_send": 0, 2023-11-16T16:55:21.762 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_resend": 0, 2023-11-16T16:55:21.762 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_active": 0, 2023-11-16T16:55:21.763 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_send": 0, 2023-11-16T16:55:21.763 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_resend": 0, 2023-11-16T16:55:21.763 INFO:teuthology.orchestra.run.smithi125.stdout: "command_active": 0, 2023-11-16T16:55:21.763 INFO:teuthology.orchestra.run.smithi125.stdout: "command_send": 0, 2023-11-16T16:55:21.763 INFO:teuthology.orchestra.run.smithi125.stdout: "command_resend": 0, 2023-11-16T16:55:21.763 INFO:teuthology.orchestra.run.smithi125.stdout: "map_epoch": 94, 2023-11-16T16:55:21.763 INFO:teuthology.orchestra.run.smithi125.stdout: "map_full": 0, 2023-11-16T16:55:21.763 INFO:teuthology.orchestra.run.smithi125.stdout: "map_inc": 11, 2023-11-16T16:55:21.764 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_sessions": 12, 2023-11-16T16:55:21.764 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_session_open": 12, 2023-11-16T16:55:21.764 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_session_close": 0, 2023-11-16T16:55:21.764 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_laggy": 0, 2023-11-16T16:55:21.764 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_wr": 10273, 2023-11-16T16:55:21.764 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_rd": 672, 2023-11-16T16:55:21.764 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_del": 446 2023-11-16T16:55:21.764 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.765 INFO:teuthology.orchestra.run.smithi125.stdout: "oft": { 2023-11-16T16:55:21.765 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_objs": 1, 2023-11-16T16:55:21.765 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_kv_pairs": 0, 2023-11-16T16:55:21.765 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_updates": 0, 2023-11-16T16:55:21.765 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_removes": 0 2023-11-16T16:55:21.765 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.765 INFO:teuthology.orchestra.run.smithi125.stdout: "osd-slow-ops": { 2023-11-16T16:55:21.765 INFO:teuthology.orchestra.run.smithi125.stdout: "slow_ops_count": 58553 2023-11-16T16:55:21.765 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.766 INFO:teuthology.orchestra.run.smithi125.stdout: "purge_queue": { 2023-11-16T16:55:21.766 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_ops": 0, 2023-11-16T16:55:21.766 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_ops_high_water": 16, 2023-11-16T16:55:21.766 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing": 0, 2023-11-16T16:55:21.766 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_high_water": 13, 2023-11-16T16:55:21.766 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executed": 13569, 2023-11-16T16:55:21.767 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_item_in_journal": 0 2023-11-16T16:55:21.767 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.767 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:55:21.767 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:21.767 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2023-11-16T16:55:21.767 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:21.767 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 181116, 2023-11-16T16:55:21.767 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 58921759, 2023-11-16T16:55:21.768 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:21.768 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 181116, 2023-11-16T16:55:21.768 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2023-11-16T16:55:21.768 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2023-11-16T16:55:21.768 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 181116, 2023-11-16T16:55:21.768 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 58921759, 2023-11-16T16:55:21.768 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:21.768 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.768 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.769 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.769 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:21.769 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.769 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-objecter_bytes": { 2023-11-16T16:55:21.769 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:21.769 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2023-11-16T16:55:21.769 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:21.769 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2023-11-16T16:55:21.770 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2023-11-16T16:55:21.770 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:21.770 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:21.770 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 84716, 2023-11-16T16:55:21.770 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 245758931, 2023-11-16T16:55:21.770 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 84716, 2023-11-16T16:55:21.770 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 245758931, 2023-11-16T16:55:21.770 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:21.771 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.771 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.771 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.771 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:21.771 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.771 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-objecter_ops": { 2023-11-16T16:55:21.771 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:21.771 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 1024, 2023-11-16T16:55:21.772 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:21.772 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2023-11-16T16:55:21.772 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2023-11-16T16:55:21.772 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:21.772 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:21.772 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 84716, 2023-11-16T16:55:21.772 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 84716, 2023-11-16T16:55:21.772 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 84716, 2023-11-16T16:55:21.772 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 84716, 2023-11-16T16:55:21.773 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:21.773 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.773 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.773 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.773 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:21.773 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.773 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-write_buf_throttle": { 2023-11-16T16:55:21.773 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:21.774 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 3758096384, 2023-11-16T16:55:21.774 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:21.774 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 13569, 2023-11-16T16:55:21.774 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 1478125, 2023-11-16T16:55:21.774 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:21.774 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 13569, 2023-11-16T16:55:21.774 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2023-11-16T16:55:21.774 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2023-11-16T16:55:21.775 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 339, 2023-11-16T16:55:21.775 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 1478125, 2023-11-16T16:55:21.775 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:21.775 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.775 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.775 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.775 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:21.775 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:21.776 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-write_buf_throttle-0x5610dee0c0a0": { 2023-11-16T16:55:21.776 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:21.776 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 3758096384, 2023-11-16T16:55:21.776 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:21.776 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 71891, 2023-11-16T16:55:21.776 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 191950986, 2023-11-16T16:55:21.776 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:21.776 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 71891, 2023-11-16T16:55:21.776 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2023-11-16T16:55:21.777 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2023-11-16T16:55:21.777 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 2351, 2023-11-16T16:55:21.777 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 191950986, 2023-11-16T16:55:21.777 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:21.777 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:21.777 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:21.777 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:21.777 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:21.778 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:21.778 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-11-16T16:55:21.778 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:21 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:55:21.899 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:21 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:55:21.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:21 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:55:22.094 INFO:tasks.check_counter:Daemon mds.b mds.exported=446 2023-11-16T16:55:22.094 INFO:tasks.check_counter:Daemon mds.b mds.imported=768 2023-11-16T16:55:22.095 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1191 2023-11-16T16:55:22.095 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2023-11-16T16:55:22.095 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=127 2023-11-16T16:55:22.095 DEBUG:tasks.check_counter:Getting stats from e 2023-11-16T16:55:22.095 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-11-16T16:55:22.736 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:55:22.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:22 smithi093 ceph-mon[97863]: pgmap v549: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-16T16:55:22.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:22 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:55:22.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:22 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:55:22.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:22 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:55:22.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:22 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:55:22.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:22 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:55:22.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:22 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:55:22.901 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:22 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:55:22.953 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:22 smithi125 ceph-mon[131076]: pgmap v549: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-16T16:55:22.953 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:22 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:55:22.953 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:22 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:55:22.953 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:22 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:55:22.954 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:22 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:55:22.954 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:22 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:55:22.954 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:22 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:55:22.954 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:22 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:55:22.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:22 smithi159 ceph-mon[131528]: pgmap v549: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-16T16:55:22.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:22 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:55:22.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:22 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:55:22.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:22 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:55:22.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:22 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:55:22.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:22 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:55:22.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:22 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:55:22.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:22 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' 2023-11-16T16:55:23.898 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-11-16T16:55:23.898 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:55:23.899 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 414, 2023-11-16T16:55:23.899 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 414, 2023-11-16T16:55:23.899 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 33664732, 2023-11-16T16:55:23.899 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 128021, 2023-11-16T16:55:23.899 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 8, 2023-11-16T16:55:23.899 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 4, 2023-11-16T16:55:23.899 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.053013537, 2023-11-16T16:55:23.899 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.013559517, 2023-11-16T16:55:23.900 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.024248399, 2023-11-16T16:55:23.900 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.010960718, 2023-11-16T16:55:23.900 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:23.900 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 413, 2023-11-16T16:55:23.900 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.024256189, 2023-11-16T16:55:23.900 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000058731 2023-11-16T16:55:23.900 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.900 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:23.901 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.901 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.901 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.901 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.901 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-16T16:55:23.901 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 448 2023-11-16T16:55:23.901 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.901 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:55:23.901 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 256, 2023-11-16T16:55:23.902 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 369, 2023-11-16T16:55:23.902 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 78653156, 2023-11-16T16:55:23.902 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 337599, 2023-11-16T16:55:23.902 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 6, 2023-11-16T16:55:23.902 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 5, 2023-11-16T16:55:23.902 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.067173877, 2023-11-16T16:55:23.902 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.007819663, 2023-11-16T16:55:23.902 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.050174195, 2023-11-16T16:55:23.903 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.006344460, 2023-11-16T16:55:23.903 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:23.903 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 367, 2023-11-16T16:55:23.903 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.025123349, 2023-11-16T16:55:23.903 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000068455 2023-11-16T16:55:23.903 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.903 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:23.903 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.904 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.904 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.904 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.904 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-16T16:55:23.904 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 258288 2023-11-16T16:55:23.904 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.904 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:55:23.904 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 1223, 2023-11-16T16:55:23.904 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 1212, 2023-11-16T16:55:23.905 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 74350790, 2023-11-16T16:55:23.905 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 414212, 2023-11-16T16:55:23.905 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 5, 2023-11-16T16:55:23.905 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 4, 2023-11-16T16:55:23.905 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.161971901, 2023-11-16T16:55:23.905 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.033743856, 2023-11-16T16:55:23.905 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.079638626, 2023-11-16T16:55:23.905 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.041251554, 2023-11-16T16:55:23.906 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:23.906 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 1211, 2023-11-16T16:55:23.906 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.054632397, 2023-11-16T16:55:23.906 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000045113 2023-11-16T16:55:23.906 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.906 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:23.906 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.906 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.907 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.907 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.907 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 501344, 2023-11-16T16:55:23.907 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 87040 2023-11-16T16:55:23.907 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.907 INFO:teuthology.orchestra.run.smithi125.stdout: "cct": { 2023-11-16T16:55:23.907 INFO:teuthology.orchestra.run.smithi125.stdout: "total_workers": 1, 2023-11-16T16:55:23.907 INFO:teuthology.orchestra.run.smithi125.stdout: "unhealthy_workers": 0 2023-11-16T16:55:23.908 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.908 INFO:teuthology.orchestra.run.smithi125.stdout: "finisher-MDSRank": { 2023-11-16T16:55:23.908 INFO:teuthology.orchestra.run.smithi125.stdout: "queue_len": 0, 2023-11-16T16:55:23.908 INFO:teuthology.orchestra.run.smithi125.stdout: "complete_latency": { 2023-11-16T16:55:23.908 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 2774, 2023-11-16T16:55:23.908 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.080730101, 2023-11-16T16:55:23.908 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000029102 2023-11-16T16:55:23.908 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:23.909 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.909 INFO:teuthology.orchestra.run.smithi125.stdout: "finisher-PurgeQueue": { 2023-11-16T16:55:23.909 INFO:teuthology.orchestra.run.smithi125.stdout: "queue_len": 0, 2023-11-16T16:55:23.909 INFO:teuthology.orchestra.run.smithi125.stdout: "complete_latency": { 2023-11-16T16:55:23.909 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.909 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.909 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.909 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:23.909 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.910 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": { 2023-11-16T16:55:23.910 INFO:teuthology.orchestra.run.smithi125.stdout: "request": 0, 2023-11-16T16:55:23.910 INFO:teuthology.orchestra.run.smithi125.stdout: "reply": 0, 2023-11-16T16:55:23.910 INFO:teuthology.orchestra.run.smithi125.stdout: "reply_latency": { 2023-11-16T16:55:23.910 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.910 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.910 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.911 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.911 INFO:teuthology.orchestra.run.smithi125.stdout: "slow_reply": 0, 2023-11-16T16:55:23.911 INFO:teuthology.orchestra.run.smithi125.stdout: "forward": 0, 2023-11-16T16:55:23.911 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_fetch_complete": 0, 2023-11-16T16:55:23.911 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_fetch_keys": 0, 2023-11-16T16:55:23.911 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_commit": 0, 2023-11-16T16:55:23.911 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_split": 0, 2023-11-16T16:55:23.911 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_merge": 0, 2023-11-16T16:55:23.912 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes": 20697, 2023-11-16T16:55:23.912 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_top": 0, 2023-11-16T16:55:23.912 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_bottom": 0, 2023-11-16T16:55:23.912 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_pin_tail": 20697, 2023-11-16T16:55:23.912 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_pinned": 20697, 2023-11-16T16:55:23.912 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_expired": 0, 2023-11-16T16:55:23.912 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_with_caps": 0, 2023-11-16T16:55:23.912 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": 0, 2023-11-16T16:55:23.913 INFO:teuthology.orchestra.run.smithi125.stdout: "subtrees": 1, 2023-11-16T16:55:23.913 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse": 0, 2023-11-16T16:55:23.913 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_hit": 0, 2023-11-16T16:55:23.913 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_forward": 0, 2023-11-16T16:55:23.913 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_discover": 0, 2023-11-16T16:55:23.913 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_dir_fetch": 0, 2023-11-16T16:55:23.913 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_remote_ino": 0, 2023-11-16T16:55:23.913 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_lock": 0, 2023-11-16T16:55:23.914 INFO:teuthology.orchestra.run.smithi125.stdout: "load_cent": 0, 2023-11-16T16:55:23.914 INFO:teuthology.orchestra.run.smithi125.stdout: "q": 0, 2023-11-16T16:55:23.914 INFO:teuthology.orchestra.run.smithi125.stdout: "exported": 0, 2023-11-16T16:55:23.914 INFO:teuthology.orchestra.run.smithi125.stdout: "exported_inodes": 0, 2023-11-16T16:55:23.914 INFO:teuthology.orchestra.run.smithi125.stdout: "imported": 0, 2023-11-16T16:55:23.914 INFO:teuthology.orchestra.run.smithi125.stdout: "imported_inodes": 0, 2023-11-16T16:55:23.914 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_dir_fetch": 0, 2023-11-16T16:55:23.915 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_backtrace_fetch": 0, 2023-11-16T16:55:23.915 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_peer_discover": 0, 2023-11-16T16:55:23.915 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rfiles": 0, 2023-11-16T16:55:23.915 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rbytes": 0, 2023-11-16T16:55:23.915 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rsnaps": 0, 2023-11-16T16:55:23.915 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_backtrace_fetch": 0, 2023-11-16T16:55:23.915 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_set_tag": 0, 2023-11-16T16:55:23.915 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_backtrace_repaired": 0, 2023-11-16T16:55:23.916 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_inotable_repaired": 0, 2023-11-16T16:55:23.916 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dir_inodes": 0, 2023-11-16T16:55:23.916 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dir_base_inodes": 0, 2023-11-16T16:55:23.916 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dirfrag_rstats": 0, 2023-11-16T16:55:23.916 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_file_inodes": 0, 2023-11-16T16:55:23.916 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_inode_file_caps": 0, 2023-11-16T16:55:23.916 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_revoke": 0, 2023-11-16T16:55:23.916 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_grant": 0, 2023-11-16T16:55:23.917 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_trunc": 0, 2023-11-16T16:55:23.917 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-16T16:55:23.917 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-16T16:55:23.917 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_caps": 0, 2023-11-16T16:55:23.917 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_caps_dirty": 0, 2023-11-16T16:55:23.917 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_cap_release": 0, 2023-11-16T16:55:23.917 INFO:teuthology.orchestra.run.smithi125.stdout: "process_request_cap_release": 0 2023-11-16T16:55:23.917 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.918 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_cache": { 2023-11-16T16:55:23.918 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_update": 0, 2023-11-16T16:55:23.918 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_update_receipt": 0, 2023-11-16T16:55:23.918 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_try_discover": 0, 2023-11-16T16:55:23.918 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_send_discover": 0, 2023-11-16T16:55:23.918 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_handle_discover": 0, 2023-11-16T16:55:23.918 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays": 0, 2023-11-16T16:55:23.918 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays_delayed": 0, 2023-11-16T16:55:23.919 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays_enqueuing": 0, 2023-11-16T16:55:23.919 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_created": 0, 2023-11-16T16:55:23.919 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_enqueued": 0, 2023-11-16T16:55:23.919 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_reintegrated": 0, 2023-11-16T16:55:23.919 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_migrated": 0, 2023-11-16T16:55:23.919 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_processing": 0, 2023-11-16T16:55:23.919 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_enqueued": 0, 2023-11-16T16:55:23.920 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_prioritized": 0, 2023-11-16T16:55:23.920 INFO:teuthology.orchestra.run.smithi125.stdout: "recovery_started": 0, 2023-11-16T16:55:23.920 INFO:teuthology.orchestra.run.smithi125.stdout: "recovery_completed": 0, 2023-11-16T16:55:23.920 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_enqueue_scrub": 0, 2023-11-16T16:55:23.920 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_exportdir": 0, 2023-11-16T16:55:23.920 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_flush": 0, 2023-11-16T16:55:23.921 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_fragmentdir": 0, 2023-11-16T16:55:23.921 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_fragstats": 0, 2023-11-16T16:55:23.921 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_inodestats": 0 2023-11-16T16:55:23.921 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.921 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_log": { 2023-11-16T16:55:23.921 INFO:teuthology.orchestra.run.smithi125.stdout: "evadd": 0, 2023-11-16T16:55:23.921 INFO:teuthology.orchestra.run.smithi125.stdout: "evex": 0, 2023-11-16T16:55:23.922 INFO:teuthology.orchestra.run.smithi125.stdout: "evtrm": 0, 2023-11-16T16:55:23.922 INFO:teuthology.orchestra.run.smithi125.stdout: "ev": 0, 2023-11-16T16:55:23.922 INFO:teuthology.orchestra.run.smithi125.stdout: "evexg": 0, 2023-11-16T16:55:23.922 INFO:teuthology.orchestra.run.smithi125.stdout: "evexd": 0, 2023-11-16T16:55:23.922 INFO:teuthology.orchestra.run.smithi125.stdout: "segadd": 0, 2023-11-16T16:55:23.922 INFO:teuthology.orchestra.run.smithi125.stdout: "segex": 0, 2023-11-16T16:55:23.922 INFO:teuthology.orchestra.run.smithi125.stdout: "segtrm": 0, 2023-11-16T16:55:23.923 INFO:teuthology.orchestra.run.smithi125.stdout: "seg": 104, 2023-11-16T16:55:23.923 INFO:teuthology.orchestra.run.smithi125.stdout: "segexg": 0, 2023-11-16T16:55:23.923 INFO:teuthology.orchestra.run.smithi125.stdout: "segexd": 0, 2023-11-16T16:55:23.923 INFO:teuthology.orchestra.run.smithi125.stdout: "expos": 4194304, 2023-11-16T16:55:23.923 INFO:teuthology.orchestra.run.smithi125.stdout: "wrpos": 0, 2023-11-16T16:55:23.923 INFO:teuthology.orchestra.run.smithi125.stdout: "rdpos": 186787055, 2023-11-16T16:55:23.923 INFO:teuthology.orchestra.run.smithi125.stdout: "jlat": { 2023-11-16T16:55:23.923 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.924 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.924 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.924 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.924 INFO:teuthology.orchestra.run.smithi125.stdout: "replayed": 76560 2023-11-16T16:55:23.924 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.924 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_mem": { 2023-11-16T16:55:23.924 INFO:teuthology.orchestra.run.smithi125.stdout: "ino": 13, 2023-11-16T16:55:23.925 INFO:teuthology.orchestra.run.smithi125.stdout: "ino+": 21585, 2023-11-16T16:55:23.925 INFO:teuthology.orchestra.run.smithi125.stdout: "ino-": 21572, 2023-11-16T16:55:23.925 INFO:teuthology.orchestra.run.smithi125.stdout: "dir": 11, 2023-11-16T16:55:23.925 INFO:teuthology.orchestra.run.smithi125.stdout: "dir+": 1317, 2023-11-16T16:55:23.925 INFO:teuthology.orchestra.run.smithi125.stdout: "dir-": 1306, 2023-11-16T16:55:23.925 INFO:teuthology.orchestra.run.smithi125.stdout: "dn": 20697, 2023-11-16T16:55:23.925 INFO:teuthology.orchestra.run.smithi125.stdout: "dn+": 49824, 2023-11-16T16:55:23.926 INFO:teuthology.orchestra.run.smithi125.stdout: "dn-": 29127, 2023-11-16T16:55:23.926 INFO:teuthology.orchestra.run.smithi125.stdout: "cap": 0, 2023-11-16T16:55:23.926 INFO:teuthology.orchestra.run.smithi125.stdout: "cap+": 0, 2023-11-16T16:55:23.926 INFO:teuthology.orchestra.run.smithi125.stdout: "cap-": 0, 2023-11-16T16:55:23.926 INFO:teuthology.orchestra.run.smithi125.stdout: "rss": 88968, 2023-11-16T16:55:23.926 INFO:teuthology.orchestra.run.smithi125.stdout: "heap": 207156 2023-11-16T16:55:23.927 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.927 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_server": { 2023-11-16T16:55:23.927 INFO:teuthology.orchestra.run.smithi125.stdout: "dispatch_client_request": 0, 2023-11-16T16:55:23.927 INFO:teuthology.orchestra.run.smithi125.stdout: "dispatch_server_request": 0, 2023-11-16T16:55:23.927 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_request": 0, 2023-11-16T16:55:23.927 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_session": 0, 2023-11-16T16:55:23.927 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_peer_request": 0, 2023-11-16T16:55:23.927 INFO:teuthology.orchestra.run.smithi125.stdout: "req_create_latency": { 2023-11-16T16:55:23.928 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.928 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.928 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.928 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.928 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getattr_latency": { 2023-11-16T16:55:23.928 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.928 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.928 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.929 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.929 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getfilelock_latency": { 2023-11-16T16:55:23.929 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.929 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.929 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.929 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.929 INFO:teuthology.orchestra.run.smithi125.stdout: "req_link_latency": { 2023-11-16T16:55:23.930 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.930 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.930 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.930 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.930 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookup_latency": { 2023-11-16T16:55:23.930 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.930 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.931 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.933 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.934 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookuphash_latency": { 2023-11-16T16:55:23.934 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.934 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.934 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.934 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.935 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupino_latency": { 2023-11-16T16:55:23.935 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.935 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.935 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.935 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.935 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupname_latency": { 2023-11-16T16:55:23.936 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.936 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.936 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.936 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.936 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupparent_latency": { 2023-11-16T16:55:23.936 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.936 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.936 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.936 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.937 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupsnap_latency": { 2023-11-16T16:55:23.937 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.937 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.937 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.937 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.937 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lssnap_latency": { 2023-11-16T16:55:23.937 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.938 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.938 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.938 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.938 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mkdir_latency": { 2023-11-16T16:55:23.938 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.938 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.938 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.938 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.939 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mknod_latency": { 2023-11-16T16:55:23.939 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.939 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.939 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.939 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.939 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mksnap_latency": { 2023-11-16T16:55:23.939 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.939 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.940 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.940 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.940 INFO:teuthology.orchestra.run.smithi125.stdout: "req_open_latency": { 2023-11-16T16:55:23.940 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.940 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.940 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.940 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.940 INFO:teuthology.orchestra.run.smithi125.stdout: "req_readdir_latency": { 2023-11-16T16:55:23.940 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.941 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.941 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.941 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.941 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rename_latency": { 2023-11-16T16:55:23.941 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.941 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.941 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.941 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.942 INFO:teuthology.orchestra.run.smithi125.stdout: "req_renamesnap_latency": { 2023-11-16T16:55:23.942 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.942 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.942 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.942 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.942 INFO:teuthology.orchestra.run.smithi125.stdout: "req_snapdiff_latency": { 2023-11-16T16:55:23.943 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.943 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.943 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.943 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.943 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmdir_latency": { 2023-11-16T16:55:23.943 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.943 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.943 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.944 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.944 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmsnap_latency": { 2023-11-16T16:55:23.944 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.944 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.944 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.944 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.944 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmxattr_latency": { 2023-11-16T16:55:23.945 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.945 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.945 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.945 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.945 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setattr_latency": { 2023-11-16T16:55:23.945 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.945 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.946 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.946 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.946 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setdirlayout_latency": { 2023-11-16T16:55:23.946 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.946 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.946 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.947 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.947 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setfilelock_latency": { 2023-11-16T16:55:23.947 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.947 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.947 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.947 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.947 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setlayout_latency": { 2023-11-16T16:55:23.947 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.948 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.948 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.948 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.948 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setxattr_latency": { 2023-11-16T16:55:23.948 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.948 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.948 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.948 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.949 INFO:teuthology.orchestra.run.smithi125.stdout: "req_symlink_latency": { 2023-11-16T16:55:23.949 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.949 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.949 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.949 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.949 INFO:teuthology.orchestra.run.smithi125.stdout: "req_unlink_latency": { 2023-11-16T16:55:23.949 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.949 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.950 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.950 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.950 INFO:teuthology.orchestra.run.smithi125.stdout: "cap_revoke_eviction": 0, 2023-11-16T16:55:23.950 INFO:teuthology.orchestra.run.smithi125.stdout: "cap_acquisition_throttle": 0, 2023-11-16T16:55:23.950 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getvxattr_latency": { 2023-11-16T16:55:23.950 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.950 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.950 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.951 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:23.951 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.951 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_sessions": { 2023-11-16T16:55:23.951 INFO:teuthology.orchestra.run.smithi125.stdout: "session_count": 1, 2023-11-16T16:55:23.951 INFO:teuthology.orchestra.run.smithi125.stdout: "session_add": 2, 2023-11-16T16:55:23.951 INFO:teuthology.orchestra.run.smithi125.stdout: "session_remove": 1, 2023-11-16T16:55:23.951 INFO:teuthology.orchestra.run.smithi125.stdout: "sessions_open": 1, 2023-11-16T16:55:23.951 INFO:teuthology.orchestra.run.smithi125.stdout: "sessions_stale": 0, 2023-11-16T16:55:23.952 INFO:teuthology.orchestra.run.smithi125.stdout: "total_load": 0, 2023-11-16T16:55:23.952 INFO:teuthology.orchestra.run.smithi125.stdout: "average_load": 0, 2023-11-16T16:55:23.952 INFO:teuthology.orchestra.run.smithi125.stdout: "avg_session_uptime": 2444 2023-11-16T16:55:23.952 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.952 INFO:teuthology.orchestra.run.smithi125.stdout: "mempool": { 2023-11-16T16:55:23.952 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:55:23.952 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_items": 0, 2023-11-16T16:55:23.952 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:55:23.952 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:55:23.953 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:55:23.953 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:55:23.953 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:55:23.953 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:55:23.953 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:55:23.953 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:55:23.953 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:55:23.953 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:55:23.953 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:55:23.954 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:55:23.954 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:55:23.954 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_items": 0, 2023-11-16T16:55:23.954 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:55:23.954 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_items": 0, 2023-11-16T16:55:23.954 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:55:23.954 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:55:23.954 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:55:23.955 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:55:23.955 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:55:23.955 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:55:23.955 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:55:23.955 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_items": 0, 2023-11-16T16:55:23.955 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:55:23.955 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:55:23.955 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:55:23.955 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_items": 0, 2023-11-16T16:55:23.956 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_bytes": 0, 2023-11-16T16:55:23.956 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_items": 0, 2023-11-16T16:55:23.956 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:55:23.956 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:55:23.956 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:55:23.956 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:55:23.956 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_bytes": 231431, 2023-11-16T16:55:23.956 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_items": 76, 2023-11-16T16:55:23.956 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:55:23.957 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_items": 0, 2023-11-16T16:55:23.957 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_bytes": 0, 2023-11-16T16:55:23.957 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_items": 0, 2023-11-16T16:55:23.957 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:55:23.957 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_items": 0, 2023-11-16T16:55:23.957 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:55:23.957 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_items": 0, 2023-11-16T16:55:23.957 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_bytes": 9280, 2023-11-16T16:55:23.957 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_items": 179, 2023-11-16T16:55:23.958 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:55:23.958 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:55:23.958 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_bytes": 0, 2023-11-16T16:55:23.958 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_items": 0, 2023-11-16T16:55:23.958 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_bytes": 14613448, 2023-11-16T16:55:23.958 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_items": 82916, 2023-11-16T16:55:23.958 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_bytes": 0, 2023-11-16T16:55:23.958 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_items": 0, 2023-11-16T16:55:23.959 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_bytes": 0, 2023-11-16T16:55:23.959 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_items": 0 2023-11-16T16:55:23.959 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.959 INFO:teuthology.orchestra.run.smithi125.stdout: "objecter": { 2023-11-16T16:55:23.959 INFO:teuthology.orchestra.run.smithi125.stdout: "op_active": 0, 2023-11-16T16:55:23.959 INFO:teuthology.orchestra.run.smithi125.stdout: "op_laggy": 0, 2023-11-16T16:55:23.959 INFO:teuthology.orchestra.run.smithi125.stdout: "op_send": 1717, 2023-11-16T16:55:23.959 INFO:teuthology.orchestra.run.smithi125.stdout: "op_send_bytes": 21, 2023-11-16T16:55:23.960 INFO:teuthology.orchestra.run.smithi125.stdout: "op_resend": 3, 2023-11-16T16:55:23.960 INFO:teuthology.orchestra.run.smithi125.stdout: "op_reply": 1717, 2023-11-16T16:55:23.960 INFO:teuthology.orchestra.run.smithi125.stdout: "op_latency": { 2023-11-16T16:55:23.960 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 1717, 2023-11-16T16:55:23.960 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 4.978916313, 2023-11-16T16:55:23.960 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.002899776 2023-11-16T16:55:23.960 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.960 INFO:teuthology.orchestra.run.smithi125.stdout: "op_inflight": 0, 2023-11-16T16:55:23.960 INFO:teuthology.orchestra.run.smithi125.stdout: "oplen_avg": { 2023-11-16T16:55:23.961 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 1717, 2023-11-16T16:55:23.961 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 1718 2023-11-16T16:55:23.961 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.961 INFO:teuthology.orchestra.run.smithi125.stdout: "op": 1717, 2023-11-16T16:55:23.961 INFO:teuthology.orchestra.run.smithi125.stdout: "op_r": 1717, 2023-11-16T16:55:23.961 INFO:teuthology.orchestra.run.smithi125.stdout: "op_w": 0, 2023-11-16T16:55:23.961 INFO:teuthology.orchestra.run.smithi125.stdout: "op_rmw": 0, 2023-11-16T16:55:23.961 INFO:teuthology.orchestra.run.smithi125.stdout: "op_pg": 0, 2023-11-16T16:55:23.961 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_stat": 1051, 2023-11-16T16:55:23.962 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_create": 0, 2023-11-16T16:55:23.962 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_read": 664, 2023-11-16T16:55:23.962 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_write": 0, 2023-11-16T16:55:23.962 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_writefull": 0, 2023-11-16T16:55:23.962 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_writesame": 0, 2023-11-16T16:55:23.962 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_append": 0, 2023-11-16T16:55:23.962 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_zero": 0, 2023-11-16T16:55:23.962 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_truncate": 0, 2023-11-16T16:55:23.962 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_delete": 0, 2023-11-16T16:55:23.963 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_mapext": 0, 2023-11-16T16:55:23.963 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_sparse_read": 0, 2023-11-16T16:55:23.963 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_clonerange": 0, 2023-11-16T16:55:23.963 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_getxattr": 1, 2023-11-16T16:55:23.963 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_setxattr": 0, 2023-11-16T16:55:23.963 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_cmpxattr": 0, 2023-11-16T16:55:23.963 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_rmxattr": 0, 2023-11-16T16:55:23.963 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_resetxattrs": 0, 2023-11-16T16:55:23.963 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_call": 0, 2023-11-16T16:55:23.964 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_watch": 0, 2023-11-16T16:55:23.964 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_notify": 0, 2023-11-16T16:55:23.964 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_src_cmpxattr": 0, 2023-11-16T16:55:23.964 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_pgls": 0, 2023-11-16T16:55:23.964 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_pgls_filter": 0, 2023-11-16T16:55:23.964 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_other": 0, 2023-11-16T16:55:23.964 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_active": 0, 2023-11-16T16:55:23.964 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_send": 0, 2023-11-16T16:55:23.964 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_resend": 0, 2023-11-16T16:55:23.965 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_ping": 0, 2023-11-16T16:55:23.965 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_active": 0, 2023-11-16T16:55:23.965 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_send": 0, 2023-11-16T16:55:23.965 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_resend": 0, 2023-11-16T16:55:23.965 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_active": 0, 2023-11-16T16:55:23.965 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_send": 0, 2023-11-16T16:55:23.965 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_resend": 0, 2023-11-16T16:55:23.965 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_active": 0, 2023-11-16T16:55:23.965 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_send": 0, 2023-11-16T16:55:23.966 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_resend": 0, 2023-11-16T16:55:23.966 INFO:teuthology.orchestra.run.smithi125.stdout: "command_active": 0, 2023-11-16T16:55:23.966 INFO:teuthology.orchestra.run.smithi125.stdout: "command_send": 0, 2023-11-16T16:55:23.966 INFO:teuthology.orchestra.run.smithi125.stdout: "command_resend": 0, 2023-11-16T16:55:23.966 INFO:teuthology.orchestra.run.smithi125.stdout: "map_epoch": 94, 2023-11-16T16:55:23.966 INFO:teuthology.orchestra.run.smithi125.stdout: "map_full": 0, 2023-11-16T16:55:23.966 INFO:teuthology.orchestra.run.smithi125.stdout: "map_inc": 4, 2023-11-16T16:55:23.966 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_sessions": 10, 2023-11-16T16:55:23.966 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_session_open": 10, 2023-11-16T16:55:23.967 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_session_close": 0, 2023-11-16T16:55:23.967 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_laggy": 0, 2023-11-16T16:55:23.967 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_wr": 0, 2023-11-16T16:55:23.967 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_rd": 2, 2023-11-16T16:55:23.967 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_del": 0 2023-11-16T16:55:23.967 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.967 INFO:teuthology.orchestra.run.smithi125.stdout: "oft": { 2023-11-16T16:55:23.967 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_objs": 0, 2023-11-16T16:55:23.967 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_kv_pairs": 0, 2023-11-16T16:55:23.968 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_updates": 0, 2023-11-16T16:55:23.968 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_removes": 0 2023-11-16T16:55:23.968 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.968 INFO:teuthology.orchestra.run.smithi125.stdout: "osd-slow-ops": { 2023-11-16T16:55:23.968 INFO:teuthology.orchestra.run.smithi125.stdout: "slow_ops_count": 0 2023-11-16T16:55:23.968 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.968 INFO:teuthology.orchestra.run.smithi125.stdout: "purge_queue": { 2023-11-16T16:55:23.968 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_ops": 0, 2023-11-16T16:55:23.968 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_ops_high_water": 0, 2023-11-16T16:55:23.969 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing": 0, 2023-11-16T16:55:23.969 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_high_water": 0, 2023-11-16T16:55:23.969 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executed": 0, 2023-11-16T16:55:23.969 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_item_in_journal": 0 2023-11-16T16:55:23.969 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.969 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:55:23.969 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:23.969 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2023-11-16T16:55:23.970 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:23.970 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 1889, 2023-11-16T16:55:23.970 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 186486934, 2023-11-16T16:55:23.970 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:23.970 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 1889, 2023-11-16T16:55:23.970 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2023-11-16T16:55:23.970 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2023-11-16T16:55:23.970 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 1889, 2023-11-16T16:55:23.971 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 186486934, 2023-11-16T16:55:23.971 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:23.971 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.971 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.971 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.971 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:23.971 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.971 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-objecter_bytes": { 2023-11-16T16:55:23.971 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:23.972 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2023-11-16T16:55:23.972 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:23.972 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2023-11-16T16:55:23.972 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2023-11-16T16:55:23.972 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:23.972 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:23.972 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 1717, 2023-11-16T16:55:23.972 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 185685964, 2023-11-16T16:55:23.972 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 1717, 2023-11-16T16:55:23.973 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 185685964, 2023-11-16T16:55:23.973 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:23.973 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.973 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.973 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.973 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:23.973 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.973 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-objecter_ops": { 2023-11-16T16:55:23.973 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:23.974 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 1024, 2023-11-16T16:55:23.974 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:23.974 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2023-11-16T16:55:23.974 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2023-11-16T16:55:23.974 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:23.974 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:23.974 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 1717, 2023-11-16T16:55:23.974 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 1717, 2023-11-16T16:55:23.975 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 1717, 2023-11-16T16:55:23.975 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 1717, 2023-11-16T16:55:23.975 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:23.975 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.975 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.975 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.975 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:23.975 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.976 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-write_buf_throttle": { 2023-11-16T16:55:23.976 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:23.976 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 3758096384, 2023-11-16T16:55:23.976 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:23.976 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2023-11-16T16:55:23.976 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2023-11-16T16:55:23.976 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:23.976 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:23.976 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2023-11-16T16:55:23.977 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2023-11-16T16:55:23.977 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 0, 2023-11-16T16:55:23.977 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 0, 2023-11-16T16:55:23.977 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:23.977 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.977 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.977 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.977 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:23.977 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:23.978 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-write_buf_throttle-0x56298449a460": { 2023-11-16T16:55:23.978 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:23.978 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 3758096384, 2023-11-16T16:55:23.978 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:23.978 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2023-11-16T16:55:23.978 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2023-11-16T16:55:23.978 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:23.978 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:23.979 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2023-11-16T16:55:23.979 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2023-11-16T16:55:23.979 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 0, 2023-11-16T16:55:23.979 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 0, 2023-11-16T16:55:23.979 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:23.979 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:23.979 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:23.979 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:23.980 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:23.980 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:23.980 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-11-16T16:55:23.980 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:23 smithi093 ceph-mon[97863]: Adjusting osd_memory_target on smithi093 to 221.9M 2023-11-16T16:55:23.981 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:23 smithi093 ceph-mon[97863]: Unable to set osd_memory_target on smithi093 to 232709120: error parsing value: Value '232709120' is below minimum 939524096 2023-11-16T16:55:23.981 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:23 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:55:23.981 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:23 smithi093 ceph-mon[97863]: Service mgr has no related daemon already placed. 2023-11-16T16:55:23.981 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:23 smithi093 ceph-mon[97863]: Service mon has no related daemon already placed. 2023-11-16T16:55:23.981 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:23 smithi093 ceph-mon[97863]: Service mds.all has no related daemon already placed. 2023-11-16T16:55:23.981 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:23 smithi125 ceph-mon[131076]: Adjusting osd_memory_target on smithi093 to 221.9M 2023-11-16T16:55:23.982 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:23 smithi125 ceph-mon[131076]: Unable to set osd_memory_target on smithi093 to 232709120: error parsing value: Value '232709120' is below minimum 939524096 2023-11-16T16:55:23.982 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:23 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:55:23.982 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:23 smithi125 ceph-mon[131076]: Service mgr has no related daemon already placed. 2023-11-16T16:55:23.982 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:23 smithi125 ceph-mon[131076]: Service mon has no related daemon already placed. 2023-11-16T16:55:23.982 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:23 smithi125 ceph-mon[131076]: Service mds.all has no related daemon already placed. 2023-11-16T16:55:23.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:23 smithi159 ceph-mon[131528]: Adjusting osd_memory_target on smithi093 to 221.9M 2023-11-16T16:55:23.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:23 smithi159 ceph-mon[131528]: Unable to set osd_memory_target on smithi093 to 232709120: error parsing value: Value '232709120' is below minimum 939524096 2023-11-16T16:55:23.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:23 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:55:23.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:23 smithi159 ceph-mon[131528]: Service mgr has no related daemon already placed. 2023-11-16T16:55:23.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:23 smithi159 ceph-mon[131528]: Service mon has no related daemon already placed. 2023-11-16T16:55:23.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:23 smithi159 ceph-mon[131528]: Service mds.all has no related daemon already placed. 2023-11-16T16:55:24.274 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-11-16T16:55:24.274 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-11-16T16:55:24.275 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-11-16T16:55:24.275 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-11-16T16:55:24.275 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-11-16T16:55:24.275 DEBUG:tasks.check_counter:Getting stats from h 2023-11-16T16:55:24.275 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-11-16T16:55:24.913 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:55:24.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:24 smithi159 ceph-mon[131528]: pgmap v550: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:55:25.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:24 smithi125 ceph-mon[131076]: pgmap v550: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:55:25.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:24 smithi093 ceph-mon[97863]: pgmap v550: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:55:26.100 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-11-16T16:55:26.100 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:55:26.100 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 85418, 2023-11-16T16:55:26.100 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 149162, 2023-11-16T16:55:26.100 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 34598601, 2023-11-16T16:55:26.101 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 176213147, 2023-11-16T16:55:26.101 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 16, 2023-11-16T16:55:26.101 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 8, 2023-11-16T16:55:26.101 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 6.126033765, 2023-11-16T16:55:26.101 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 2.734496353, 2023-11-16T16:55:26.101 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 3.713483672, 2023-11-16T16:55:26.101 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.633734177, 2023-11-16T16:55:26.101 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:26.102 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 149158, 2023-11-16T16:55:26.102 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 12.229021600, 2023-11-16T16:55:26.102 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000081987 2023-11-16T16:55:26.102 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.102 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:26.102 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 157080, 2023-11-16T16:55:26.102 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.025200768, 2023-11-16T16:55:26.102 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000160 2023-11-16T16:55:26.103 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.103 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 26645008, 2023-11-16T16:55:26.103 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 88913984 2023-11-16T16:55:26.103 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.103 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:55:26.103 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 24302, 2023-11-16T16:55:26.103 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 27609, 2023-11-16T16:55:26.103 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 47715314, 2023-11-16T16:55:26.104 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 58290438, 2023-11-16T16:55:26.104 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 7, 2023-11-16T16:55:26.104 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 5, 2023-11-16T16:55:26.104 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 1.744976849, 2023-11-16T16:55:26.104 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.635776616, 2023-11-16T16:55:26.104 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.635450185, 2023-11-16T16:55:26.104 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.506867957, 2023-11-16T16:55:26.104 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:26.104 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 27611, 2023-11-16T16:55:26.105 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 3.903764922, 2023-11-16T16:55:26.105 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000141384 2023-11-16T16:55:26.105 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.105 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:26.105 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 21481, 2023-11-16T16:55:26.105 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.003821759, 2023-11-16T16:55:26.105 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000177 2023-11-16T16:55:26.105 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.106 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 511536, 2023-11-16T16:55:26.106 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 92208 2023-11-16T16:55:26.106 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.106 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:55:26.106 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 29930, 2023-11-16T16:55:26.106 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 43856, 2023-11-16T16:55:26.106 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 21102937, 2023-11-16T16:55:26.106 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 55342071, 2023-11-16T16:55:26.107 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 8, 2023-11-16T16:55:26.107 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 6, 2023-11-16T16:55:26.107 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 2.048010429, 2023-11-16T16:55:26.107 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.720974258, 2023-11-16T16:55:26.107 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.832247814, 2023-11-16T16:55:26.107 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.500658879, 2023-11-16T16:55:26.107 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:26.107 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 43854, 2023-11-16T16:55:26.108 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 5.348434328, 2023-11-16T16:55:26.108 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000121960 2023-11-16T16:55:26.108 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.108 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:26.108 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 41057, 2023-11-16T16:55:26.108 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.007533826, 2023-11-16T16:55:26.108 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000183 2023-11-16T16:55:26.108 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.109 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-16T16:55:26.109 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 258112 2023-11-16T16:55:26.109 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.109 INFO:teuthology.orchestra.run.smithi125.stdout: "cct": { 2023-11-16T16:55:26.109 INFO:teuthology.orchestra.run.smithi125.stdout: "total_workers": 1, 2023-11-16T16:55:26.109 INFO:teuthology.orchestra.run.smithi125.stdout: "unhealthy_workers": 0 2023-11-16T16:55:26.109 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.109 INFO:teuthology.orchestra.run.smithi125.stdout: "finisher-MDSRank": { 2023-11-16T16:55:26.110 INFO:teuthology.orchestra.run.smithi125.stdout: "queue_len": 0, 2023-11-16T16:55:26.110 INFO:teuthology.orchestra.run.smithi125.stdout: "complete_latency": { 2023-11-16T16:55:26.110 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 3455, 2023-11-16T16:55:26.110 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 51.103562669, 2023-11-16T16:55:26.110 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.014791190 2023-11-16T16:55:26.110 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:26.110 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.110 INFO:teuthology.orchestra.run.smithi125.stdout: "finisher-PurgeQueue": { 2023-11-16T16:55:26.111 INFO:teuthology.orchestra.run.smithi125.stdout: "queue_len": 0, 2023-11-16T16:55:26.111 INFO:teuthology.orchestra.run.smithi125.stdout: "complete_latency": { 2023-11-16T16:55:26.111 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 35477, 2023-11-16T16:55:26.111 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 14.134381967, 2023-11-16T16:55:26.111 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000398409 2023-11-16T16:55:26.111 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:26.111 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.111 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": { 2023-11-16T16:55:26.111 INFO:teuthology.orchestra.run.smithi125.stdout: "request": 52541, 2023-11-16T16:55:26.112 INFO:teuthology.orchestra.run.smithi125.stdout: "reply": 47706, 2023-11-16T16:55:26.112 INFO:teuthology.orchestra.run.smithi125.stdout: "reply_latency": { 2023-11-16T16:55:26.112 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 47706, 2023-11-16T16:55:26.112 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 142.825606233, 2023-11-16T16:55:26.112 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.002993870 2023-11-16T16:55:26.112 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.112 INFO:teuthology.orchestra.run.smithi125.stdout: "slow_reply": 0, 2023-11-16T16:55:26.112 INFO:teuthology.orchestra.run.smithi125.stdout: "forward": 4835, 2023-11-16T16:55:26.113 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_fetch_complete": 659, 2023-11-16T16:55:26.113 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_fetch_keys": 0, 2023-11-16T16:55:26.113 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_commit": 465, 2023-11-16T16:55:26.113 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_split": 53, 2023-11-16T16:55:26.113 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_merge": 30, 2023-11-16T16:55:26.113 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes": 19, 2023-11-16T16:55:26.113 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_top": 2, 2023-11-16T16:55:26.113 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_bottom": 7, 2023-11-16T16:55:26.114 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_pin_tail": 10, 2023-11-16T16:55:26.114 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_pinned": 15, 2023-11-16T16:55:26.114 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_expired": 25981, 2023-11-16T16:55:26.114 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_with_caps": 1, 2023-11-16T16:55:26.114 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": 1, 2023-11-16T16:55:26.114 INFO:teuthology.orchestra.run.smithi125.stdout: "subtrees": 4, 2023-11-16T16:55:26.114 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse": 76992, 2023-11-16T16:55:26.114 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_hit": 69248, 2023-11-16T16:55:26.115 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_forward": 4773, 2023-11-16T16:55:26.115 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_discover": 421, 2023-11-16T16:55:26.115 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_dir_fetch": 656, 2023-11-16T16:55:26.115 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_remote_ino": 0, 2023-11-16T16:55:26.115 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_lock": 0, 2023-11-16T16:55:26.115 INFO:teuthology.orchestra.run.smithi125.stdout: "load_cent": 0, 2023-11-16T16:55:26.115 INFO:teuthology.orchestra.run.smithi125.stdout: "q": 0, 2023-11-16T16:55:26.115 INFO:teuthology.orchestra.run.smithi125.stdout: "exported": 685, 2023-11-16T16:55:26.115 INFO:teuthology.orchestra.run.smithi125.stdout: "exported_inodes": 18225, 2023-11-16T16:55:26.116 INFO:teuthology.orchestra.run.smithi125.stdout: "imported": 826, 2023-11-16T16:55:26.116 INFO:teuthology.orchestra.run.smithi125.stdout: "imported_inodes": 0, 2023-11-16T16:55:26.116 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_dir_fetch": 0, 2023-11-16T16:55:26.116 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_backtrace_fetch": 0, 2023-11-16T16:55:26.116 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_peer_discover": 0, 2023-11-16T16:55:26.116 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rfiles": 7, 2023-11-16T16:55:26.116 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rbytes": 246, 2023-11-16T16:55:26.116 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rsnaps": 0, 2023-11-16T16:55:26.117 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_backtrace_fetch": 0, 2023-11-16T16:55:26.117 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_set_tag": 0, 2023-11-16T16:55:26.117 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_backtrace_repaired": 0, 2023-11-16T16:55:26.117 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_inotable_repaired": 0, 2023-11-16T16:55:26.117 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dir_inodes": 0, 2023-11-16T16:55:26.117 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dir_base_inodes": 0, 2023-11-16T16:55:26.117 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dirfrag_rstats": 0, 2023-11-16T16:55:26.117 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_file_inodes": 0, 2023-11-16T16:55:26.118 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_inode_file_caps": 3, 2023-11-16T16:55:26.118 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_revoke": 703, 2023-11-16T16:55:26.118 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_grant": 27005, 2023-11-16T16:55:26.118 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_trunc": 0, 2023-11-16T16:55:26.118 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-16T16:55:26.118 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-16T16:55:26.118 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_caps": 13626, 2023-11-16T16:55:26.118 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_caps_dirty": 12919, 2023-11-16T16:55:26.119 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_cap_release": 865, 2023-11-16T16:55:26.119 INFO:teuthology.orchestra.run.smithi125.stdout: "process_request_cap_release": 41193 2023-11-16T16:55:26.119 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.119 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_cache": { 2023-11-16T16:55:26.119 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_update": 424, 2023-11-16T16:55:26.119 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_update_receipt": 3238, 2023-11-16T16:55:26.119 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_try_discover": 1621, 2023-11-16T16:55:26.119 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_send_discover": 2260, 2023-11-16T16:55:26.120 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_handle_discover": 424, 2023-11-16T16:55:26.120 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays": 0, 2023-11-16T16:55:26.120 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays_delayed": 0, 2023-11-16T16:55:26.120 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays_enqueuing": 0, 2023-11-16T16:55:26.120 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_created": 20687, 2023-11-16T16:55:26.120 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_enqueued": 20687, 2023-11-16T16:55:26.120 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_reintegrated": 0, 2023-11-16T16:55:26.120 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_migrated": 0, 2023-11-16T16:55:26.121 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_processing": 0, 2023-11-16T16:55:26.121 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_enqueued": 0, 2023-11-16T16:55:26.121 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_prioritized": 0, 2023-11-16T16:55:26.121 INFO:teuthology.orchestra.run.smithi125.stdout: "recovery_started": 0, 2023-11-16T16:55:26.121 INFO:teuthology.orchestra.run.smithi125.stdout: "recovery_completed": 0, 2023-11-16T16:55:26.121 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_enqueue_scrub": 0, 2023-11-16T16:55:26.121 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_exportdir": 717, 2023-11-16T16:55:26.121 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_flush": 0, 2023-11-16T16:55:26.121 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_fragmentdir": 83, 2023-11-16T16:55:26.122 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_fragstats": 0, 2023-11-16T16:55:26.122 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_inodestats": 0 2023-11-16T16:55:26.122 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.122 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_log": { 2023-11-16T16:55:26.122 INFO:teuthology.orchestra.run.smithi125.stdout: "evadd": 76560, 2023-11-16T16:55:26.122 INFO:teuthology.orchestra.run.smithi125.stdout: "evex": 0, 2023-11-16T16:55:26.122 INFO:teuthology.orchestra.run.smithi125.stdout: "evtrm": 0, 2023-11-16T16:55:26.122 INFO:teuthology.orchestra.run.smithi125.stdout: "ev": 76560, 2023-11-16T16:55:26.123 INFO:teuthology.orchestra.run.smithi125.stdout: "evexg": 0, 2023-11-16T16:55:26.123 INFO:teuthology.orchestra.run.smithi125.stdout: "evexd": 0, 2023-11-16T16:55:26.123 INFO:teuthology.orchestra.run.smithi125.stdout: "segadd": 104, 2023-11-16T16:55:26.123 INFO:teuthology.orchestra.run.smithi125.stdout: "segex": 0, 2023-11-16T16:55:26.123 INFO:teuthology.orchestra.run.smithi125.stdout: "segtrm": 0, 2023-11-16T16:55:26.123 INFO:teuthology.orchestra.run.smithi125.stdout: "seg": 104, 2023-11-16T16:55:26.123 INFO:teuthology.orchestra.run.smithi125.stdout: "segexg": 0, 2023-11-16T16:55:26.123 INFO:teuthology.orchestra.run.smithi125.stdout: "segexd": 0, 2023-11-16T16:55:26.124 INFO:teuthology.orchestra.run.smithi125.stdout: "expos": 4194304, 2023-11-16T16:55:26.124 INFO:teuthology.orchestra.run.smithi125.stdout: "wrpos": 186788193, 2023-11-16T16:55:26.124 INFO:teuthology.orchestra.run.smithi125.stdout: "rdpos": 0, 2023-11-16T16:55:26.124 INFO:teuthology.orchestra.run.smithi125.stdout: "jlat": { 2023-11-16T16:55:26.124 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 2806, 2023-11-16T16:55:26.124 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 200.489293341, 2023-11-16T16:55:26.124 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.071450211 2023-11-16T16:55:26.124 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.124 INFO:teuthology.orchestra.run.smithi125.stdout: "replayed": 0 2023-11-16T16:55:26.125 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.125 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_mem": { 2023-11-16T16:55:26.125 INFO:teuthology.orchestra.run.smithi125.stdout: "ino": 24, 2023-11-16T16:55:26.125 INFO:teuthology.orchestra.run.smithi125.stdout: "ino+": 25277, 2023-11-16T16:55:26.125 INFO:teuthology.orchestra.run.smithi125.stdout: "ino-": 25253, 2023-11-16T16:55:26.125 INFO:teuthology.orchestra.run.smithi125.stdout: "dir": 22, 2023-11-16T16:55:26.125 INFO:teuthology.orchestra.run.smithi125.stdout: "dir+": 2792, 2023-11-16T16:55:26.125 INFO:teuthology.orchestra.run.smithi125.stdout: "dir-": 2770, 2023-11-16T16:55:26.126 INFO:teuthology.orchestra.run.smithi125.stdout: "dn": 19, 2023-11-16T16:55:26.126 INFO:teuthology.orchestra.run.smithi125.stdout: "dn+": 50572, 2023-11-16T16:55:26.126 INFO:teuthology.orchestra.run.smithi125.stdout: "dn-": 50553, 2023-11-16T16:55:26.126 INFO:teuthology.orchestra.run.smithi125.stdout: "cap": 1, 2023-11-16T16:55:26.126 INFO:teuthology.orchestra.run.smithi125.stdout: "cap+": 42101, 2023-11-16T16:55:26.126 INFO:teuthology.orchestra.run.smithi125.stdout: "cap-": 42100, 2023-11-16T16:55:26.126 INFO:teuthology.orchestra.run.smithi125.stdout: "rss": 146720, 2023-11-16T16:55:26.126 INFO:teuthology.orchestra.run.smithi125.stdout: "heap": 198964 2023-11-16T16:55:26.127 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.127 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_server": { 2023-11-16T16:55:26.127 INFO:teuthology.orchestra.run.smithi125.stdout: "dispatch_client_request": 60462, 2023-11-16T16:55:26.127 INFO:teuthology.orchestra.run.smithi125.stdout: "dispatch_server_request": 11, 2023-11-16T16:55:26.127 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_request": 52541, 2023-11-16T16:55:26.127 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_session": 10, 2023-11-16T16:55:26.127 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_peer_request": 276, 2023-11-16T16:55:26.127 INFO:teuthology.orchestra.run.smithi125.stdout: "req_create_latency": { 2023-11-16T16:55:26.128 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 6309, 2023-11-16T16:55:26.128 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 10.668339635, 2023-11-16T16:55:26.128 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.001690971 2023-11-16T16:55:26.128 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.128 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getattr_latency": { 2023-11-16T16:55:26.128 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 2, 2023-11-16T16:55:26.128 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.371495747, 2023-11-16T16:55:26.128 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.185747873 2023-11-16T16:55:26.129 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.129 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getfilelock_latency": { 2023-11-16T16:55:26.129 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.129 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.129 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.129 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.129 INFO:teuthology.orchestra.run.smithi125.stdout: "req_link_latency": { 2023-11-16T16:55:26.129 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.130 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.130 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.130 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.130 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookup_latency": { 2023-11-16T16:55:26.130 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 19863, 2023-11-16T16:55:26.130 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 44.721916987, 2023-11-16T16:55:26.130 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.002251518 2023-11-16T16:55:26.131 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.131 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookuphash_latency": { 2023-11-16T16:55:26.131 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.131 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.131 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.131 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.131 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupino_latency": { 2023-11-16T16:55:26.131 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 1, 2023-11-16T16:55:26.131 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000126446, 2023-11-16T16:55:26.132 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000126446 2023-11-16T16:55:26.132 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.132 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupname_latency": { 2023-11-16T16:55:26.132 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.132 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.132 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.132 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.132 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupparent_latency": { 2023-11-16T16:55:26.133 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.133 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.133 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.133 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.133 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupsnap_latency": { 2023-11-16T16:55:26.133 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.133 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.133 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.134 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.134 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lssnap_latency": { 2023-11-16T16:55:26.134 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.134 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.134 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.134 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.134 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mkdir_latency": { 2023-11-16T16:55:26.134 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.135 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.135 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.135 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.135 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mknod_latency": { 2023-11-16T16:55:26.135 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.135 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.135 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.135 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.135 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mksnap_latency": { 2023-11-16T16:55:26.136 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.136 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.136 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.136 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.136 INFO:teuthology.orchestra.run.smithi125.stdout: "req_open_latency": { 2023-11-16T16:55:26.136 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.136 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.136 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.137 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.137 INFO:teuthology.orchestra.run.smithi125.stdout: "req_readdir_latency": { 2023-11-16T16:55:26.137 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 844, 2023-11-16T16:55:26.137 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 1.286141869, 2023-11-16T16:55:26.137 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.001523864 2023-11-16T16:55:26.137 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.137 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rename_latency": { 2023-11-16T16:55:26.137 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.137 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.138 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.138 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.138 INFO:teuthology.orchestra.run.smithi125.stdout: "req_renamesnap_latency": { 2023-11-16T16:55:26.138 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.138 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.138 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.138 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.139 INFO:teuthology.orchestra.run.smithi125.stdout: "req_snapdiff_latency": { 2023-11-16T16:55:26.139 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.139 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.139 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.139 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.139 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmdir_latency": { 2023-11-16T16:55:26.139 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 1, 2023-11-16T16:55:26.139 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.007267152, 2023-11-16T16:55:26.140 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.007267152 2023-11-16T16:55:26.140 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.140 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmsnap_latency": { 2023-11-16T16:55:26.140 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.140 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.140 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.140 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.141 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmxattr_latency": { 2023-11-16T16:55:26.141 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.141 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.141 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.141 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.141 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setattr_latency": { 2023-11-16T16:55:26.141 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.141 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.141 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.142 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.142 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setdirlayout_latency": { 2023-11-16T16:55:26.142 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.142 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.142 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.142 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.142 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setfilelock_latency": { 2023-11-16T16:55:26.142 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.143 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.143 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.143 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.143 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setlayout_latency": { 2023-11-16T16:55:26.143 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.143 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.143 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.143 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.144 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setxattr_latency": { 2023-11-16T16:55:26.144 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.144 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.144 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.144 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.144 INFO:teuthology.orchestra.run.smithi125.stdout: "req_symlink_latency": { 2023-11-16T16:55:26.144 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.144 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.145 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.145 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.145 INFO:teuthology.orchestra.run.smithi125.stdout: "req_unlink_latency": { 2023-11-16T16:55:26.145 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 20686, 2023-11-16T16:55:26.145 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 85.770318397, 2023-11-16T16:55:26.145 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.004146297 2023-11-16T16:55:26.145 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.145 INFO:teuthology.orchestra.run.smithi125.stdout: "cap_revoke_eviction": 0, 2023-11-16T16:55:26.145 INFO:teuthology.orchestra.run.smithi125.stdout: "cap_acquisition_throttle": 0, 2023-11-16T16:55:26.146 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getvxattr_latency": { 2023-11-16T16:55:26.146 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.146 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.146 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.146 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:26.146 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.146 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_sessions": { 2023-11-16T16:55:26.146 INFO:teuthology.orchestra.run.smithi125.stdout: "session_count": 1, 2023-11-16T16:55:26.147 INFO:teuthology.orchestra.run.smithi125.stdout: "session_add": 2, 2023-11-16T16:55:26.147 INFO:teuthology.orchestra.run.smithi125.stdout: "session_remove": 1, 2023-11-16T16:55:26.147 INFO:teuthology.orchestra.run.smithi125.stdout: "sessions_open": 1, 2023-11-16T16:55:26.147 INFO:teuthology.orchestra.run.smithi125.stdout: "sessions_stale": 0, 2023-11-16T16:55:26.147 INFO:teuthology.orchestra.run.smithi125.stdout: "total_load": 25356, 2023-11-16T16:55:26.147 INFO:teuthology.orchestra.run.smithi125.stdout: "average_load": 25356, 2023-11-16T16:55:26.147 INFO:teuthology.orchestra.run.smithi125.stdout: "avg_session_uptime": 394 2023-11-16T16:55:26.147 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.148 INFO:teuthology.orchestra.run.smithi125.stdout: "mempool": { 2023-11-16T16:55:26.148 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:55:26.148 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_items": 0, 2023-11-16T16:55:26.148 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:55:26.148 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:55:26.148 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:55:26.148 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:55:26.148 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:55:26.149 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:55:26.149 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:55:26.149 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:55:26.149 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:55:26.149 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:55:26.149 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:55:26.149 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:55:26.149 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:55:26.150 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_items": 0, 2023-11-16T16:55:26.150 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:55:26.150 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_items": 0, 2023-11-16T16:55:26.150 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:55:26.150 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:55:26.150 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:55:26.150 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:55:26.150 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:55:26.151 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:55:26.151 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:55:26.151 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_items": 0, 2023-11-16T16:55:26.151 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:55:26.151 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:55:26.151 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:55:26.151 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_items": 0, 2023-11-16T16:55:26.151 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_bytes": 0, 2023-11-16T16:55:26.152 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_items": 0, 2023-11-16T16:55:26.152 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:55:26.152 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:55:26.152 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:55:26.152 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:55:26.152 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_bytes": 294011, 2023-11-16T16:55:26.152 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_items": 147, 2023-11-16T16:55:26.152 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_bytes": 1760, 2023-11-16T16:55:26.153 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_items": 20, 2023-11-16T16:55:26.153 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_bytes": 0, 2023-11-16T16:55:26.153 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_items": 0, 2023-11-16T16:55:26.153 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:55:26.153 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_items": 0, 2023-11-16T16:55:26.153 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:55:26.153 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_items": 0, 2023-11-16T16:55:26.153 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_bytes": 9280, 2023-11-16T16:55:26.154 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_items": 179, 2023-11-16T16:55:26.154 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:55:26.154 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:55:26.154 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_bytes": 0, 2023-11-16T16:55:26.154 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_items": 0, 2023-11-16T16:55:26.154 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_bytes": 97207, 2023-11-16T16:55:26.154 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_items": 555, 2023-11-16T16:55:26.154 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_bytes": 0, 2023-11-16T16:55:26.155 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_items": 0, 2023-11-16T16:55:26.155 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_bytes": 0, 2023-11-16T16:55:26.155 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_items": 0 2023-11-16T16:55:26.155 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.155 INFO:teuthology.orchestra.run.smithi125.stdout: "objecter": { 2023-11-16T16:55:26.155 INFO:teuthology.orchestra.run.smithi125.stdout: "op_active": 0, 2023-11-16T16:55:26.155 INFO:teuthology.orchestra.run.smithi125.stdout: "op_laggy": 0, 2023-11-16T16:55:26.155 INFO:teuthology.orchestra.run.smithi125.stdout: "op_send": 47158, 2023-11-16T16:55:26.156 INFO:teuthology.orchestra.run.smithi125.stdout: "op_send_bytes": 189969617, 2023-11-16T16:55:26.156 INFO:teuthology.orchestra.run.smithi125.stdout: "op_resend": 33, 2023-11-16T16:55:26.156 INFO:teuthology.orchestra.run.smithi125.stdout: "op_reply": 47158, 2023-11-16T16:55:26.156 INFO:teuthology.orchestra.run.smithi125.stdout: "op_latency": { 2023-11-16T16:55:26.156 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 47158, 2023-11-16T16:55:26.156 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 475.375019206, 2023-11-16T16:55:26.156 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.010080474 2023-11-16T16:55:26.156 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.157 INFO:teuthology.orchestra.run.smithi125.stdout: "op_inflight": 0, 2023-11-16T16:55:26.157 INFO:teuthology.orchestra.run.smithi125.stdout: "oplen_avg": { 2023-11-16T16:55:26.157 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 47158, 2023-11-16T16:55:26.157 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 48325 2023-11-16T16:55:26.157 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.157 INFO:teuthology.orchestra.run.smithi125.stdout: "op": 47158, 2023-11-16T16:55:26.157 INFO:teuthology.orchestra.run.smithi125.stdout: "op_r": 1157, 2023-11-16T16:55:26.157 INFO:teuthology.orchestra.run.smithi125.stdout: "op_w": 46001, 2023-11-16T16:55:26.158 INFO:teuthology.orchestra.run.smithi125.stdout: "op_rmw": 0, 2023-11-16T16:55:26.158 INFO:teuthology.orchestra.run.smithi125.stdout: "op_pg": 0, 2023-11-16T16:55:26.158 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_stat": 0, 2023-11-16T16:55:26.158 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_create": 10, 2023-11-16T16:55:26.158 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_read": 498, 2023-11-16T16:55:26.158 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_write": 3304, 2023-11-16T16:55:26.158 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_writefull": 161, 2023-11-16T16:55:26.158 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_writesame": 0, 2023-11-16T16:55:26.159 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_append": 0, 2023-11-16T16:55:26.159 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_zero": 0, 2023-11-16T16:55:26.159 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_truncate": 0, 2023-11-16T16:55:26.159 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_delete": 42023, 2023-11-16T16:55:26.159 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_mapext": 0, 2023-11-16T16:55:26.159 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_sparse_read": 0, 2023-11-16T16:55:26.159 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_clonerange": 0, 2023-11-16T16:55:26.160 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_getxattr": 0, 2023-11-16T16:55:26.160 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_setxattr": 20, 2023-11-16T16:55:26.160 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_cmpxattr": 0, 2023-11-16T16:55:26.160 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_rmxattr": 0, 2023-11-16T16:55:26.160 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_resetxattrs": 0, 2023-11-16T16:55:26.160 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_call": 0, 2023-11-16T16:55:26.160 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_watch": 0, 2023-11-16T16:55:26.160 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_notify": 0, 2023-11-16T16:55:26.161 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_src_cmpxattr": 0, 2023-11-16T16:55:26.161 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_pgls": 0, 2023-11-16T16:55:26.161 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_pgls_filter": 0, 2023-11-16T16:55:26.161 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_other": 0, 2023-11-16T16:55:26.161 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_active": 0, 2023-11-16T16:55:26.161 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_send": 0, 2023-11-16T16:55:26.161 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_resend": 0, 2023-11-16T16:55:26.161 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_ping": 0, 2023-11-16T16:55:26.162 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_active": 0, 2023-11-16T16:55:26.162 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_send": 0, 2023-11-16T16:55:26.162 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_resend": 0, 2023-11-16T16:55:26.162 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_active": 0, 2023-11-16T16:55:26.162 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_send": 0, 2023-11-16T16:55:26.162 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_resend": 0, 2023-11-16T16:55:26.162 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_active": 0, 2023-11-16T16:55:26.162 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_send": 0, 2023-11-16T16:55:26.163 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_resend": 0, 2023-11-16T16:55:26.163 INFO:teuthology.orchestra.run.smithi125.stdout: "command_active": 0, 2023-11-16T16:55:26.163 INFO:teuthology.orchestra.run.smithi125.stdout: "command_send": 0, 2023-11-16T16:55:26.163 INFO:teuthology.orchestra.run.smithi125.stdout: "command_resend": 0, 2023-11-16T16:55:26.163 INFO:teuthology.orchestra.run.smithi125.stdout: "map_epoch": 94, 2023-11-16T16:55:26.163 INFO:teuthology.orchestra.run.smithi125.stdout: "map_full": 0, 2023-11-16T16:55:26.163 INFO:teuthology.orchestra.run.smithi125.stdout: "map_inc": 4, 2023-11-16T16:55:26.163 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_sessions": 12, 2023-11-16T16:55:26.163 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_session_open": 12, 2023-11-16T16:55:26.164 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_session_close": 0, 2023-11-16T16:55:26.164 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_laggy": 0, 2023-11-16T16:55:26.164 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_wr": 963, 2023-11-16T16:55:26.164 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_rd": 1318, 2023-11-16T16:55:26.164 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_del": 28 2023-11-16T16:55:26.164 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.164 INFO:teuthology.orchestra.run.smithi125.stdout: "oft": { 2023-11-16T16:55:26.164 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_objs": 1, 2023-11-16T16:55:26.165 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_kv_pairs": 0, 2023-11-16T16:55:26.165 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_updates": 0, 2023-11-16T16:55:26.165 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_removes": 0 2023-11-16T16:55:26.165 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.165 INFO:teuthology.orchestra.run.smithi125.stdout: "osd-slow-ops": { 2023-11-16T16:55:26.165 INFO:teuthology.orchestra.run.smithi125.stdout: "slow_ops_count": 53352 2023-11-16T16:55:26.165 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.165 INFO:teuthology.orchestra.run.smithi125.stdout: "purge_queue": { 2023-11-16T16:55:26.165 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_ops": 0, 2023-11-16T16:55:26.166 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_ops_high_water": 37, 2023-11-16T16:55:26.166 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing": 0, 2023-11-16T16:55:26.166 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_high_water": 8, 2023-11-16T16:55:26.166 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executed": 20687, 2023-11-16T16:55:26.166 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_item_in_journal": 0 2023-11-16T16:55:26.166 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.166 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:55:26.166 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:26.167 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2023-11-16T16:55:26.167 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:26.167 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 139646, 2023-11-16T16:55:26.167 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 87590584, 2023-11-16T16:55:26.167 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:26.167 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 139646, 2023-11-16T16:55:26.167 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2023-11-16T16:55:26.167 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2023-11-16T16:55:26.168 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 139646, 2023-11-16T16:55:26.168 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 87590584, 2023-11-16T16:55:26.168 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:26.168 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.168 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.168 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.168 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:26.168 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.168 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-objecter_bytes": { 2023-11-16T16:55:26.169 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:26.169 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2023-11-16T16:55:26.169 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:26.169 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2023-11-16T16:55:26.169 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2023-11-16T16:55:26.169 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:26.169 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:26.170 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 47158, 2023-11-16T16:55:26.170 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 192214004, 2023-11-16T16:55:26.170 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 47158, 2023-11-16T16:55:26.170 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 192214004, 2023-11-16T16:55:26.170 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:26.170 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.170 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.170 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.171 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:26.171 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.171 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-objecter_ops": { 2023-11-16T16:55:26.171 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:26.171 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 1024, 2023-11-16T16:55:26.171 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:26.171 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2023-11-16T16:55:26.171 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2023-11-16T16:55:26.172 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:26.172 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:26.172 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 47158, 2023-11-16T16:55:26.172 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 47158, 2023-11-16T16:55:26.172 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 47158, 2023-11-16T16:55:26.172 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 47158, 2023-11-16T16:55:26.172 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:26.172 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.172 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.173 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.173 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:26.173 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.173 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-write_buf_throttle": { 2023-11-16T16:55:26.173 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:26.173 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 3758096384, 2023-11-16T16:55:26.173 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:26.173 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 20687, 2023-11-16T16:55:26.174 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 2254931, 2023-11-16T16:55:26.174 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:26.174 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 20687, 2023-11-16T16:55:26.174 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2023-11-16T16:55:26.174 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2023-11-16T16:55:26.174 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 498, 2023-11-16T16:55:26.174 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 2254931, 2023-11-16T16:55:26.175 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:26.175 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.175 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.175 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.175 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:26.175 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:26.175 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-write_buf_throttle-0x55a78a88a0a0": { 2023-11-16T16:55:26.175 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:26.176 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 3758096384, 2023-11-16T16:55:26.176 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:26.176 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 76560, 2023-11-16T16:55:26.176 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 182593889, 2023-11-16T16:55:26.176 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:26.176 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 76560, 2023-11-16T16:55:26.176 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2023-11-16T16:55:26.176 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2023-11-16T16:55:26.176 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 2806, 2023-11-16T16:55:26.177 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 182593889, 2023-11-16T16:55:26.177 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:26.177 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:26.177 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:26.177 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:26.177 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:26.177 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:26.177 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-11-16T16:55:26.454 INFO:tasks.check_counter:Daemon mds.h mds.exported=685 2023-11-16T16:55:26.454 INFO:tasks.check_counter:Daemon mds.h mds.imported=826 2023-11-16T16:55:26.454 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=424 2023-11-16T16:55:26.454 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=3238 2023-11-16T16:55:26.454 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=53 2023-11-16T16:55:26.454 DEBUG:tasks.check_counter:Getting stats from k 2023-11-16T16:55:26.455 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-11-16T16:55:26.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:26 smithi125 ceph-mon[131076]: pgmap v551: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:55:26.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:26 smithi159 ceph-mon[131528]: pgmap v551: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:55:27.090 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:55:27.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:26 smithi093 ceph-mon[97863]: pgmap v551: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:55:28.288 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-11-16T16:55:28.289 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:55:28.289 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 0, 2023-11-16T16:55:28.289 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 2, 2023-11-16T16:55:28.289 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 0, 2023-11-16T16:55:28.289 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 0, 2023-11-16T16:55:28.289 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 4, 2023-11-16T16:55:28.290 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 0, 2023-11-16T16:55:28.290 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.001215299, 2023-11-16T16:55:28.290 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.000000000, 2023-11-16T16:55:28.290 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.000374284, 2023-11-16T16:55:28.290 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:55:28.290 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:28.290 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:28.290 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:28.291 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:28.291 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:28.291 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:28.291 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:28.291 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:28.291 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:28.291 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:28.291 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:55:28.292 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:55:28.292 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:28.292 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:55:28.292 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 5, 2023-11-16T16:55:28.292 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 118, 2023-11-16T16:55:28.292 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 3152, 2023-11-16T16:55:28.292 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 96640, 2023-11-16T16:55:28.292 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 3, 2023-11-16T16:55:28.293 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 2, 2023-11-16T16:55:28.293 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.006782414, 2023-11-16T16:55:28.293 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.004301738, 2023-11-16T16:55:28.293 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.001254475, 2023-11-16T16:55:28.293 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:55:28.293 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:28.293 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 117, 2023-11-16T16:55:28.293 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.010997547, 2023-11-16T16:55:28.294 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000093996 2023-11-16T16:55:28.294 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:28.294 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:28.294 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:28.294 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:28.294 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:28.294 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:28.294 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-16T16:55:28.295 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 96640 2023-11-16T16:55:28.295 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:28.295 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:55:28.295 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 158, 2023-11-16T16:55:28.295 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 155, 2023-11-16T16:55:28.295 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 454544, 2023-11-16T16:55:28.295 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 87440, 2023-11-16T16:55:28.295 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 2, 2023-11-16T16:55:28.296 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 1, 2023-11-16T16:55:28.296 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.023754269, 2023-11-16T16:55:28.296 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.008511213, 2023-11-16T16:55:28.296 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.011526651, 2023-11-16T16:55:28.296 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.004071571, 2023-11-16T16:55:28.296 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:28.296 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 154, 2023-11-16T16:55:28.296 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.007339301, 2023-11-16T16:55:28.297 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000047657 2023-11-16T16:55:28.297 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:28.297 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:28.297 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:28.297 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:28.297 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:28.297 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:28.297 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 454544, 2023-11-16T16:55:28.298 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 87440 2023-11-16T16:55:28.298 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:28.298 INFO:teuthology.orchestra.run.smithi125.stdout: "cct": { 2023-11-16T16:55:28.298 INFO:teuthology.orchestra.run.smithi125.stdout: "total_workers": 0, 2023-11-16T16:55:28.298 INFO:teuthology.orchestra.run.smithi125.stdout: "unhealthy_workers": 0 2023-11-16T16:55:28.298 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:28.298 INFO:teuthology.orchestra.run.smithi125.stdout: "mempool": { 2023-11-16T16:55:28.299 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:55:28.299 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_items": 0, 2023-11-16T16:55:28.299 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:55:28.299 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:55:28.299 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:55:28.299 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:55:28.299 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:55:28.299 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:55:28.300 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:55:28.300 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:55:28.300 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:55:28.300 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:55:28.300 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:55:28.300 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:55:28.300 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:55:28.300 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_items": 0, 2023-11-16T16:55:28.301 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:55:28.301 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_items": 0, 2023-11-16T16:55:28.301 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:55:28.301 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:55:28.301 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:55:28.301 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:55:28.301 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:55:28.301 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:55:28.301 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:55:28.302 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_items": 0, 2023-11-16T16:55:28.302 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:55:28.302 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:55:28.302 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:55:28.302 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_items": 0, 2023-11-16T16:55:28.302 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_bytes": 0, 2023-11-16T16:55:28.302 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_items": 0, 2023-11-16T16:55:28.303 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:55:28.303 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:55:28.303 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:55:28.303 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:55:28.303 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_bytes": 149007, 2023-11-16T16:55:28.303 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_items": 30, 2023-11-16T16:55:28.303 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:55:28.303 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_items": 0, 2023-11-16T16:55:28.304 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_bytes": 0, 2023-11-16T16:55:28.304 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_items": 0, 2023-11-16T16:55:28.304 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:55:28.304 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_items": 0, 2023-11-16T16:55:28.304 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:55:28.304 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_items": 0, 2023-11-16T16:55:28.304 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_bytes": 0, 2023-11-16T16:55:28.304 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_items": 0, 2023-11-16T16:55:28.304 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:55:28.305 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:55:28.305 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_bytes": 0, 2023-11-16T16:55:28.305 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_items": 0, 2023-11-16T16:55:28.305 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_bytes": 928, 2023-11-16T16:55:28.305 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_items": 2, 2023-11-16T16:55:28.305 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_bytes": 0, 2023-11-16T16:55:28.305 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_items": 0, 2023-11-16T16:55:28.306 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_bytes": 0, 2023-11-16T16:55:28.306 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_items": 0 2023-11-16T16:55:28.306 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-11-16T16:55:28.306 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:55:28.306 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2023-11-16T16:55:28.306 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2023-11-16T16:55:28.306 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2023-11-16T16:55:28.306 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 159, 2023-11-16T16:55:28.307 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 432227, 2023-11-16T16:55:28.307 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:28.307 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 159, 2023-11-16T16:55:28.307 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2023-11-16T16:55:28.307 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2023-11-16T16:55:28.307 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 159, 2023-11-16T16:55:28.307 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 432227, 2023-11-16T16:55:28.307 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2023-11-16T16:55:28.308 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2023-11-16T16:55:28.308 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2023-11-16T16:55:28.308 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2023-11-16T16:55:28.308 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:28.308 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-11-16T16:55:28.308 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-11-16T16:55:28.631 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-11-16T16:55:28.631 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-11-16T16:55:28.631 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-11-16T16:55:28.631 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-11-16T16:55:28.632 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-11-16T16:55:28.632 DEBUG:tasks.check_counter:Getting stats from c 2023-11-16T16:55:28.632 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-11-16T16:55:28.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:28 smithi125 ceph-mon[131076]: pgmap v552: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-11-16T16:55:28.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:28 smithi159 ceph-mon[131528]: pgmap v552: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-11-16T16:55:29.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:28 smithi093 ceph-mon[97863]: pgmap v552: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-11-16T16:55:29.296 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:55:30.587 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-11-16T16:55:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:55:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 223, 2023-11-16T16:55:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 225, 2023-11-16T16:55:30.587 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 7851122, 2023-11-16T16:55:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 69363, 2023-11-16T16:55:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2023-11-16T16:55:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2023-11-16T16:55:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.026943778, 2023-11-16T16:55:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.005538297, 2023-11-16T16:55:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.009082389, 2023-11-16T16:55:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.009247478, 2023-11-16T16:55:30.588 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 223, 2023-11-16T16:55:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.018289719, 2023-11-16T16:55:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000082016 2023-11-16T16:55:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:55:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:55:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:55:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 63, 2023-11-16T16:55:30.590 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 62, 2023-11-16T16:55:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 4363704, 2023-11-16T16:55:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 25347, 2023-11-16T16:55:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 4, 2023-11-16T16:55:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2023-11-16T16:55:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.009374216, 2023-11-16T16:55:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.002293598, 2023-11-16T16:55:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.006801432, 2023-11-16T16:55:30.591 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.001258536, 2023-11-16T16:55:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 61, 2023-11-16T16:55:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.007733514, 2023-11-16T16:55:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000126778 2023-11-16T16:55:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.592 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 161408, 2023-11-16T16:55:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 18128 2023-11-16T16:55:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:55:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 119, 2023-11-16T16:55:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 140, 2023-11-16T16:55:30.593 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 3239837, 2023-11-16T16:55:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 155743, 2023-11-16T16:55:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 5, 2023-11-16T16:55:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2023-11-16T16:55:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.015047736, 2023-11-16T16:55:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.003715507, 2023-11-16T16:55:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.006424007, 2023-11-16T16:55:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.003659443, 2023-11-16T16:55:30.594 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 139, 2023-11-16T16:55:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.011555552, 2023-11-16T16:55:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000083133 2023-11-16T16:55:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.595 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-16T16:55:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 120288 2023-11-16T16:55:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2023-11-16T16:55:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2023-11-16T16:55:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2023-11-16T16:55:30.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2023-11-16T16:55:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-11-16T16:55:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-11-16T16:55:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 601, 2023-11-16T16:55:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.103251089, 2023-11-16T16:55:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000171798 2023-11-16T16:55:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.597 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2023-11-16T16:55:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-11-16T16:55:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-11-16T16:55:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:30.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2023-11-16T16:55:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2023-11-16T16:55:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2023-11-16T16:55:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2023-11-16T16:55:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2023-11-16T16:55:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2023-11-16T16:55:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2023-11-16T16:55:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2023-11-16T16:55:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2023-11-16T16:55:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2023-11-16T16:55:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2023-11-16T16:55:30.600 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 1273, 2023-11-16T16:55:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 877, 2023-11-16T16:55:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 380, 2023-11-16T16:55:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 16, 2023-11-16T16:55:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 20, 2023-11-16T16:55:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 6844, 2023-11-16T16:55:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2023-11-16T16:55:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2023-11-16T16:55:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 2, 2023-11-16T16:55:30.601 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2023-11-16T16:55:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2023-11-16T16:55:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2023-11-16T16:55:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2023-11-16T16:55:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2023-11-16T16:55:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2023-11-16T16:55:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2023-11-16T16:55:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2023-11-16T16:55:30.602 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2023-11-16T16:55:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2023-11-16T16:55:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2023-11-16T16:55:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2023-11-16T16:55:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2023-11-16T16:55:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2023-11-16T16:55:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2023-11-16T16:55:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2023-11-16T16:55:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 4, 2023-11-16T16:55:30.603 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 246, 2023-11-16T16:55:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2023-11-16T16:55:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2023-11-16T16:55:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2023-11-16T16:55:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2023-11-16T16:55:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2023-11-16T16:55:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2023-11-16T16:55:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2023-11-16T16:55:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2023-11-16T16:55:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2023-11-16T16:55:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2023-11-16T16:55:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2023-11-16T16:55:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2023-11-16T16:55:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2023-11-16T16:55:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-16T16:55:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-16T16:55:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2023-11-16T16:55:30.605 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2023-11-16T16:55:30.606 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2023-11-16T16:55:30.606 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2023-11-16T16:55:30.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.606 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2023-11-16T16:55:30.606 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2023-11-16T16:55:30.606 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2023-11-16T16:55:30.606 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2023-11-16T16:55:30.606 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2023-11-16T16:55:30.607 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2023-11-16T16:55:30.607 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2023-11-16T16:55:30.607 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2023-11-16T16:55:30.607 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2023-11-16T16:55:30.607 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2023-11-16T16:55:30.607 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2023-11-16T16:55:30.607 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2023-11-16T16:55:30.607 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2023-11-16T16:55:30.608 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2023-11-16T16:55:30.608 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2023-11-16T16:55:30.608 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2023-11-16T16:55:30.608 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2023-11-16T16:55:30.608 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2023-11-16T16:55:30.608 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2023-11-16T16:55:30.608 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2023-11-16T16:55:30.608 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2023-11-16T16:55:30.609 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2023-11-16T16:55:30.609 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2023-11-16T16:55:30.609 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2023-11-16T16:55:30.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.609 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2023-11-16T16:55:30.609 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2023-11-16T16:55:30.609 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2023-11-16T16:55:30.609 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2023-11-16T16:55:30.610 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 0, 2023-11-16T16:55:30.610 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2023-11-16T16:55:30.610 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2023-11-16T16:55:30.610 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2023-11-16T16:55:30.610 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2023-11-16T16:55:30.610 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2023-11-16T16:55:30.610 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 2, 2023-11-16T16:55:30.610 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2023-11-16T16:55:30.611 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2023-11-16T16:55:30.611 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 196069747, 2023-11-16T16:55:30.611 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 0, 2023-11-16T16:55:30.611 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 196141862, 2023-11-16T16:55:30.611 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2023-11-16T16:55:30.611 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.611 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.611 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.612 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 7174 2023-11-16T16:55:30.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.612 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2023-11-16T16:55:30.612 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 21, 2023-11-16T16:55:30.612 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 3024, 2023-11-16T16:55:30.612 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 3003, 2023-11-16T16:55:30.612 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 83, 2023-11-16T16:55:30.612 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 439, 2023-11-16T16:55:30.612 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 356, 2023-11-16T16:55:30.613 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 1273, 2023-11-16T16:55:30.613 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 10196, 2023-11-16T16:55:30.613 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 8923, 2023-11-16T16:55:30.613 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2023-11-16T16:55:30.613 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2023-11-16T16:55:30.613 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2023-11-16T16:55:30.613 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 53212, 2023-11-16T16:55:30.613 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 207132 2023-11-16T16:55:30.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.614 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2023-11-16T16:55:30.614 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2023-11-16T16:55:30.614 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2023-11-16T16:55:30.614 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2023-11-16T16:55:30.614 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2023-11-16T16:55:30.614 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2023-11-16T16:55:30.614 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2023-11-16T16:55:30.614 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.615 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.615 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.615 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2023-11-16T16:55:30.615 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.615 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.615 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.616 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2023-11-16T16:55:30.616 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.616 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.616 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.616 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2023-11-16T16:55:30.616 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.616 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.616 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.617 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2023-11-16T16:55:30.617 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.617 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.617 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.617 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2023-11-16T16:55:30.617 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.618 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.618 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.618 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2023-11-16T16:55:30.618 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.618 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.618 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.619 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2023-11-16T16:55:30.619 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.619 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.619 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.619 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2023-11-16T16:55:30.619 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.620 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.620 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.620 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2023-11-16T16:55:30.620 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.620 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.620 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.621 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2023-11-16T16:55:30.621 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.621 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.621 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.621 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2023-11-16T16:55:30.621 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.621 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.622 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.622 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2023-11-16T16:55:30.622 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.622 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.622 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.622 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2023-11-16T16:55:30.623 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.623 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.623 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.623 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2023-11-16T16:55:30.623 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.623 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.623 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.624 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2023-11-16T16:55:30.624 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.624 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.624 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.624 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.624 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2023-11-16T16:55:30.624 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.624 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.625 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.625 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2023-11-16T16:55:30.625 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.625 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.625 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.625 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2023-11-16T16:55:30.626 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.626 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.626 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.626 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.626 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2023-11-16T16:55:30.626 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.626 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.626 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.627 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2023-11-16T16:55:30.627 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.627 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.627 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.627 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2023-11-16T16:55:30.627 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.628 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.628 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.628 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2023-11-16T16:55:30.628 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.628 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.628 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.629 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2023-11-16T16:55:30.629 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.629 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.629 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.629 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.629 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2023-11-16T16:55:30.629 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.629 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.630 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.630 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2023-11-16T16:55:30.630 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.630 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.630 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.631 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2023-11-16T16:55:30.631 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.631 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.631 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.631 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.631 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2023-11-16T16:55:30.631 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.631 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.631 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.632 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.632 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2023-11-16T16:55:30.632 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.632 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.632 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.632 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.632 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2023-11-16T16:55:30.632 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2023-11-16T16:55:30.632 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2023-11-16T16:55:30.633 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.633 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.633 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.633 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:30.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2023-11-16T16:55:30.633 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2023-11-16T16:55:30.633 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 1, 2023-11-16T16:55:30.634 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 0, 2023-11-16T16:55:30.634 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 0, 2023-11-16T16:55:30.634 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2023-11-16T16:55:30.634 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2023-11-16T16:55:30.634 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2023-11-16T16:55:30.634 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 2392 2023-11-16T16:55:30.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.634 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2023-11-16T16:55:30.635 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:55:30.635 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2023-11-16T16:55:30.635 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:55:30.635 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:55:30.635 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:55:30.635 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:55:30.635 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:55:30.635 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:55:30.636 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:55:30.636 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:55:30.636 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:55:30.636 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:55:30.636 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:55:30.636 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:55:30.636 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:55:30.636 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2023-11-16T16:55:30.636 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:55:30.637 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2023-11-16T16:55:30.637 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:55:30.637 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:55:30.637 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:55:30.637 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:55:30.637 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:55:30.637 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:55:30.637 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:55:30.638 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2023-11-16T16:55:30.638 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:55:30.638 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:55:30.638 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:55:30.638 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2023-11-16T16:55:30.638 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2023-11-16T16:55:30.638 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2023-11-16T16:55:30.638 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:55:30.639 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:55:30.639 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:55:30.639 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:55:30.639 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 196655, 2023-11-16T16:55:30.639 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 53, 2023-11-16T16:55:30.639 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:55:30.639 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2023-11-16T16:55:30.639 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2023-11-16T16:55:30.640 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2023-11-16T16:55:30.640 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:55:30.640 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2023-11-16T16:55:30.640 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:55:30.640 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2023-11-16T16:55:30.640 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9280, 2023-11-16T16:55:30.640 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 179, 2023-11-16T16:55:30.640 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:55:30.640 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:55:30.641 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2023-11-16T16:55:30.641 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2023-11-16T16:55:30.641 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 1053284, 2023-11-16T16:55:30.641 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 5235, 2023-11-16T16:55:30.641 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2023-11-16T16:55:30.641 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2023-11-16T16:55:30.641 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2023-11-16T16:55:30.641 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2023-11-16T16:55:30.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.642 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2023-11-16T16:55:30.642 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2023-11-16T16:55:30.642 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2023-11-16T16:55:30.642 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 360, 2023-11-16T16:55:30.642 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 26, 2023-11-16T16:55:30.642 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 4, 2023-11-16T16:55:30.642 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 360, 2023-11-16T16:55:30.642 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2023-11-16T16:55:30.643 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 360, 2023-11-16T16:55:30.643 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.303994123, 2023-11-16T16:55:30.643 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000844428 2023-11-16T16:55:30.643 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.643 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2023-11-16T16:55:30.643 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2023-11-16T16:55:30.643 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 360, 2023-11-16T16:55:30.643 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 361 2023-11-16T16:55:30.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.644 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 360, 2023-11-16T16:55:30.644 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 360, 2023-11-16T16:55:30.644 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2023-11-16T16:55:30.644 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2023-11-16T16:55:30.644 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2023-11-16T16:55:30.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 204, 2023-11-16T16:55:30.644 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2023-11-16T16:55:30.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 153, 2023-11-16T16:55:30.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2023-11-16T16:55:30.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2023-11-16T16:55:30.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2023-11-16T16:55:30.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2023-11-16T16:55:30.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2023-11-16T16:55:30.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2023-11-16T16:55:30.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2023-11-16T16:55:30.645 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2023-11-16T16:55:30.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2023-11-16T16:55:30.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2023-11-16T16:55:30.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 2, 2023-11-16T16:55:30.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2023-11-16T16:55:30.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2023-11-16T16:55:30.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2023-11-16T16:55:30.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2023-11-16T16:55:30.646 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2023-11-16T16:55:30.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2023-11-16T16:55:30.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2023-11-16T16:55:30.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2023-11-16T16:55:30.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2023-11-16T16:55:30.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2023-11-16T16:55:30.647 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2023-11-16T16:55:30.647 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2023-11-16T16:55:30.647 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2023-11-16T16:55:30.647 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2023-11-16T16:55:30.648 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2023-11-16T16:55:30.648 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2023-11-16T16:55:30.648 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2023-11-16T16:55:30.648 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2023-11-16T16:55:30.648 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2023-11-16T16:55:30.648 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2023-11-16T16:55:30.648 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2023-11-16T16:55:30.648 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2023-11-16T16:55:30.649 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2023-11-16T16:55:30.649 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2023-11-16T16:55:30.649 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2023-11-16T16:55:30.649 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2023-11-16T16:55:30.649 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2023-11-16T16:55:30.649 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 0, 2023-11-16T16:55:30.649 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2023-11-16T16:55:30.649 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 0, 2023-11-16T16:55:30.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 6, 2023-11-16T16:55:30.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 6, 2023-11-16T16:55:30.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2023-11-16T16:55:30.650 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2023-11-16T16:55:30.650 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2023-11-16T16:55:30.650 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2023-11-16T16:55:30.650 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2023-11-16T16:55:30.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.651 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2023-11-16T16:55:30.651 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2023-11-16T16:55:30.651 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2023-11-16T16:55:30.651 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2023-11-16T16:55:30.651 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2023-11-16T16:55:30.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.651 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2023-11-16T16:55:30.651 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2023-11-16T16:55:30.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.652 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2023-11-16T16:55:30.652 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2023-11-16T16:55:30.652 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2023-11-16T16:55:30.652 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2023-11-16T16:55:30.652 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2023-11-16T16:55:30.652 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2023-11-16T16:55:30.652 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2023-11-16T16:55:30.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.653 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:55:30.653 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:30.653 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-11-16T16:55:30.653 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:30.653 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 401, 2023-11-16T16:55:30.653 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 15413530, 2023-11-16T16:55:30.653 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:30.653 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 401, 2023-11-16T16:55:30.654 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-11-16T16:55:30.654 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-11-16T16:55:30.654 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 401, 2023-11-16T16:55:30.654 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 15413530, 2023-11-16T16:55:30.654 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:30.654 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.654 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.654 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.655 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:30.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.655 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2023-11-16T16:55:30.655 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:30.655 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-11-16T16:55:30.655 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:30.655 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-11-16T16:55:30.655 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-11-16T16:55:30.656 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:30.656 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:30.656 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 360, 2023-11-16T16:55:30.656 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 15187709, 2023-11-16T16:55:30.656 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 360, 2023-11-16T16:55:30.656 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 15187709, 2023-11-16T16:55:30.656 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:30.656 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.656 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.657 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.657 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:30.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.657 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2023-11-16T16:55:30.657 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:30.657 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2023-11-16T16:55:30.657 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:30.657 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-11-16T16:55:30.658 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-11-16T16:55:30.658 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:30.658 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:30.658 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 360, 2023-11-16T16:55:30.658 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 360, 2023-11-16T16:55:30.658 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 360, 2023-11-16T16:55:30.658 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 360, 2023-11-16T16:55:30.658 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:30.659 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.659 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.659 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.659 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:30.659 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.659 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2023-11-16T16:55:30.659 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:30.660 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-11-16T16:55:30.660 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:30.660 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-11-16T16:55:30.660 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-11-16T16:55:30.660 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:30.660 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:30.660 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-11-16T16:55:30.660 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-11-16T16:55:30.661 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2023-11-16T16:55:30.661 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2023-11-16T16:55:30.661 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:30.661 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.661 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.661 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.661 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:30.661 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:30.662 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x562137c68460": { 2023-11-16T16:55:30.662 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:30.662 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-11-16T16:55:30.662 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:30.662 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-11-16T16:55:30.662 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-11-16T16:55:30.662 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:30.662 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:30.663 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-11-16T16:55:30.663 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-11-16T16:55:30.663 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2023-11-16T16:55:30.663 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2023-11-16T16:55:30.663 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:30.663 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:30.663 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:30.663 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:30.663 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:30.664 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:30.664 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-11-16T16:55:30.950 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:30 smithi159 ceph-mon[131528]: pgmap v553: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:55:30.951 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-11-16T16:55:30.952 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-11-16T16:55:30.952 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-11-16T16:55:30.952 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-11-16T16:55:30.952 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2023-11-16T16:55:30.952 DEBUG:tasks.check_counter:Getting stats from f 2023-11-16T16:55:30.952 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-11-16T16:55:31.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:30 smithi125 ceph-mon[131076]: pgmap v553: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:55:31.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:30 smithi093 ceph-mon[97863]: pgmap v553: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:55:31.594 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:55:32.672 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:32 smithi159 ceph-mon[131528]: pgmap v554: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:55:32.884 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-11-16T16:55:32.884 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:55:32.885 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 830, 2023-11-16T16:55:32.885 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 819, 2023-11-16T16:55:32.885 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 46851468, 2023-11-16T16:55:32.885 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 292119, 2023-11-16T16:55:32.885 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 7, 2023-11-16T16:55:32.885 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2023-11-16T16:55:32.886 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.116886990, 2023-11-16T16:55:32.886 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.027740604, 2023-11-16T16:55:32.886 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.052759706, 2023-11-16T16:55:32.886 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.029983270, 2023-11-16T16:55:32.886 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:32.886 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 818, 2023-11-16T16:55:32.887 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.069788973, 2023-11-16T16:55:32.887 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000085316 2023-11-16T16:55:32.887 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.887 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:32.887 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.887 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.887 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.888 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.888 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 501952, 2023-11-16T16:55:32.888 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 87792 2023-11-16T16:55:32.888 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.888 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:55:32.889 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 310, 2023-11-16T16:55:32.889 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 424, 2023-11-16T16:55:32.889 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 61955089, 2023-11-16T16:55:32.889 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 356177, 2023-11-16T16:55:32.889 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 7, 2023-11-16T16:55:32.890 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2023-11-16T16:55:32.890 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.063484209, 2023-11-16T16:55:32.890 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.010738361, 2023-11-16T16:55:32.890 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.041630856, 2023-11-16T16:55:32.890 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.007696638, 2023-11-16T16:55:32.890 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:32.891 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 422, 2023-11-16T16:55:32.891 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.022153039, 2023-11-16T16:55:32.891 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000052495 2023-11-16T16:55:32.891 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.891 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:32.891 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.892 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.892 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.892 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.892 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-16T16:55:32.892 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 260016 2023-11-16T16:55:32.892 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.893 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:55:32.893 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 806, 2023-11-16T16:55:32.893 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 806, 2023-11-16T16:55:32.893 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 88315384, 2023-11-16T16:55:32.893 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 249877, 2023-11-16T16:55:32.893 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2023-11-16T16:55:32.894 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2023-11-16T16:55:32.894 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.101910096, 2023-11-16T16:55:32.894 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.020191793, 2023-11-16T16:55:32.894 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.054668663, 2023-11-16T16:55:32.894 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.021892211, 2023-11-16T16:55:32.894 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:32.895 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 805, 2023-11-16T16:55:32.895 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.036857144, 2023-11-16T16:55:32.895 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000045785 2023-11-16T16:55:32.895 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.895 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:32.895 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.896 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.896 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.896 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.896 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-16T16:55:32.896 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 448 2023-11-16T16:55:32.896 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.897 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2023-11-16T16:55:32.897 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2023-11-16T16:55:32.897 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2023-11-16T16:55:32.897 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.897 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2023-11-16T16:55:32.897 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-11-16T16:55:32.897 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-11-16T16:55:32.898 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2885, 2023-11-16T16:55:32.898 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.084012786, 2023-11-16T16:55:32.898 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000029120 2023-11-16T16:55:32.898 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:32.898 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.899 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2023-11-16T16:55:32.899 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-11-16T16:55:32.899 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-11-16T16:55:32.899 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.899 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.899 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.899 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:32.900 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.900 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2023-11-16T16:55:32.900 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2023-11-16T16:55:32.900 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2023-11-16T16:55:32.900 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2023-11-16T16:55:32.900 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.901 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.901 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.901 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.901 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2023-11-16T16:55:32.901 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2023-11-16T16:55:32.901 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2023-11-16T16:55:32.902 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2023-11-16T16:55:32.902 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2023-11-16T16:55:32.902 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2023-11-16T16:55:32.902 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2023-11-16T16:55:32.902 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 25724, 2023-11-16T16:55:32.902 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2023-11-16T16:55:32.903 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 0, 2023-11-16T16:55:32.903 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 25724, 2023-11-16T16:55:32.903 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 25724, 2023-11-16T16:55:32.903 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 155, 2023-11-16T16:55:32.903 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2023-11-16T16:55:32.903 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2023-11-16T16:55:32.904 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 1, 2023-11-16T16:55:32.904 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2023-11-16T16:55:32.904 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2023-11-16T16:55:32.904 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2023-11-16T16:55:32.904 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2023-11-16T16:55:32.904 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2023-11-16T16:55:32.905 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2023-11-16T16:55:32.905 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2023-11-16T16:55:32.905 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2023-11-16T16:55:32.905 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2023-11-16T16:55:32.905 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2023-11-16T16:55:32.905 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2023-11-16T16:55:32.905 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2023-11-16T16:55:32.906 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2023-11-16T16:55:32.906 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2023-11-16T16:55:32.906 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2023-11-16T16:55:32.906 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2023-11-16T16:55:32.906 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 0, 2023-11-16T16:55:32.906 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 0, 2023-11-16T16:55:32.907 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2023-11-16T16:55:32.907 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2023-11-16T16:55:32.907 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2023-11-16T16:55:32.907 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2023-11-16T16:55:32.907 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2023-11-16T16:55:32.907 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2023-11-16T16:55:32.907 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2023-11-16T16:55:32.908 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2023-11-16T16:55:32.908 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2023-11-16T16:55:32.908 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2023-11-16T16:55:32.908 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2023-11-16T16:55:32.908 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2023-11-16T16:55:32.908 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2023-11-16T16:55:32.909 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-16T16:55:32.909 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-16T16:55:32.909 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2023-11-16T16:55:32.909 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2023-11-16T16:55:32.909 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2023-11-16T16:55:32.909 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2023-11-16T16:55:32.910 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.910 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2023-11-16T16:55:32.910 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2023-11-16T16:55:32.910 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2023-11-16T16:55:32.910 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2023-11-16T16:55:32.910 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2023-11-16T16:55:32.911 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2023-11-16T16:55:32.911 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2023-11-16T16:55:32.911 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2023-11-16T16:55:32.911 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2023-11-16T16:55:32.911 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2023-11-16T16:55:32.911 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2023-11-16T16:55:32.912 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2023-11-16T16:55:32.912 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2023-11-16T16:55:32.912 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2023-11-16T16:55:32.912 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2023-11-16T16:55:32.912 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2023-11-16T16:55:32.912 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2023-11-16T16:55:32.912 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2023-11-16T16:55:32.913 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2023-11-16T16:55:32.913 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2023-11-16T16:55:32.913 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2023-11-16T16:55:32.913 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2023-11-16T16:55:32.913 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2023-11-16T16:55:32.913 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2023-11-16T16:55:32.914 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.914 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2023-11-16T16:55:32.914 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2023-11-16T16:55:32.914 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2023-11-16T16:55:32.914 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2023-11-16T16:55:32.914 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 0, 2023-11-16T16:55:32.915 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2023-11-16T16:55:32.915 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2023-11-16T16:55:32.915 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2023-11-16T16:55:32.915 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2023-11-16T16:55:32.915 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2023-11-16T16:55:32.915 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 94, 2023-11-16T16:55:32.915 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2023-11-16T16:55:32.916 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2023-11-16T16:55:32.916 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2023-11-16T16:55:32.916 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 0, 2023-11-16T16:55:32.916 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 200281037, 2023-11-16T16:55:32.916 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2023-11-16T16:55:32.916 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.917 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.917 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.917 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.917 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 76592 2023-11-16T16:55:32.917 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.917 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2023-11-16T16:55:32.917 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 14, 2023-11-16T16:55:32.918 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 28603, 2023-11-16T16:55:32.918 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 28589, 2023-11-16T16:55:32.918 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 12, 2023-11-16T16:55:32.918 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 1837, 2023-11-16T16:55:32.918 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 1825, 2023-11-16T16:55:32.918 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 25724, 2023-11-16T16:55:32.919 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 49053, 2023-11-16T16:55:32.919 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 23329, 2023-11-16T16:55:32.919 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2023-11-16T16:55:32.919 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2023-11-16T16:55:32.919 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2023-11-16T16:55:32.919 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 116760, 2023-11-16T16:55:32.919 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 207156 2023-11-16T16:55:32.920 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.920 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2023-11-16T16:55:32.920 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2023-11-16T16:55:32.920 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2023-11-16T16:55:32.920 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2023-11-16T16:55:32.920 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2023-11-16T16:55:32.921 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2023-11-16T16:55:32.921 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2023-11-16T16:55:32.921 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.921 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.921 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.921 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.922 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2023-11-16T16:55:32.922 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.922 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.922 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.922 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.922 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2023-11-16T16:55:32.922 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.923 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.923 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.923 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.923 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2023-11-16T16:55:32.923 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.923 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.924 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.924 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.924 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2023-11-16T16:55:32.924 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.924 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.924 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.924 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.925 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2023-11-16T16:55:32.925 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.925 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.925 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.925 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.925 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2023-11-16T16:55:32.926 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.926 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.926 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.926 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.926 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2023-11-16T16:55:32.926 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.926 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.927 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.927 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.927 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2023-11-16T16:55:32.927 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.927 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.927 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.927 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.928 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2023-11-16T16:55:32.928 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.928 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.928 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.928 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.928 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2023-11-16T16:55:32.929 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.929 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.929 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.929 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.929 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2023-11-16T16:55:32.929 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.929 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.930 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.930 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.930 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2023-11-16T16:55:32.930 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.930 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.930 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.931 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.931 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2023-11-16T16:55:32.931 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.931 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.931 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.931 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.932 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2023-11-16T16:55:32.932 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.932 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.932 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.932 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.932 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2023-11-16T16:55:32.932 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.933 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.933 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.933 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.933 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2023-11-16T16:55:32.933 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.933 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.933 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.934 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.934 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2023-11-16T16:55:32.934 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.934 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.934 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.935 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.935 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2023-11-16T16:55:32.935 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.935 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.935 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.935 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.935 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2023-11-16T16:55:32.936 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.936 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.936 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.936 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.936 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2023-11-16T16:55:32.936 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.936 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.937 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.937 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.937 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2023-11-16T16:55:32.937 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.937 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.937 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.937 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.938 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2023-11-16T16:55:32.938 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.938 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.938 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.938 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.938 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2023-11-16T16:55:32.939 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.939 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.939 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.939 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.939 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2023-11-16T16:55:32.939 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.939 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.940 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.940 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.940 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2023-11-16T16:55:32.940 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.940 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.940 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.940 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.941 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2023-11-16T16:55:32.941 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.941 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.941 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.941 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.941 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2023-11-16T16:55:32.942 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.942 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.942 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.942 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.942 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2023-11-16T16:55:32.942 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.942 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.943 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.943 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.943 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2023-11-16T16:55:32.943 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2023-11-16T16:55:32.943 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2023-11-16T16:55:32.943 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.943 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.944 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.944 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:32.944 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.944 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2023-11-16T16:55:32.944 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2023-11-16T16:55:32.944 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 1, 2023-11-16T16:55:32.944 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 0, 2023-11-16T16:55:32.945 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2023-11-16T16:55:32.945 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2023-11-16T16:55:32.945 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2023-11-16T16:55:32.945 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2023-11-16T16:55:32.945 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 2392 2023-11-16T16:55:32.945 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.945 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2023-11-16T16:55:32.946 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:55:32.946 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2023-11-16T16:55:32.946 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:55:32.946 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:55:32.946 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:55:32.946 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:55:32.946 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:55:32.947 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:55:32.947 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:55:32.947 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:55:32.947 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:55:32.947 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:55:32.947 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:55:32.948 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:55:32.948 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:55:32.948 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2023-11-16T16:55:32.948 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:55:32.948 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2023-11-16T16:55:32.948 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:55:32.948 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:55:32.949 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:55:32.949 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:55:32.949 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:55:32.949 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:55:32.949 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:55:32.949 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2023-11-16T16:55:32.949 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:55:32.950 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:55:32.950 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:55:32.950 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2023-11-16T16:55:32.950 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2023-11-16T16:55:32.950 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2023-11-16T16:55:32.950 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:55:32.951 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:55:32.951 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:55:32.951 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:55:32.951 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 239545, 2023-11-16T16:55:32.951 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 81, 2023-11-16T16:55:32.951 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:55:32.951 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2023-11-16T16:55:32.952 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2023-11-16T16:55:32.952 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2023-11-16T16:55:32.952 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:55:32.952 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2023-11-16T16:55:32.952 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:55:32.952 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2023-11-16T16:55:32.952 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9280, 2023-11-16T16:55:32.953 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 179, 2023-11-16T16:55:32.953 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:55:32.953 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:55:32.953 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2023-11-16T16:55:32.953 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2023-11-16T16:55:32.953 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 18122001, 2023-11-16T16:55:32.953 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 98875, 2023-11-16T16:55:32.954 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2023-11-16T16:55:32.954 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2023-11-16T16:55:32.954 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2023-11-16T16:55:32.954 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2023-11-16T16:55:32.954 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.954 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2023-11-16T16:55:32.954 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2023-11-16T16:55:32.955 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2023-11-16T16:55:32.955 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 1768, 2023-11-16T16:55:32.955 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 21, 2023-11-16T16:55:32.955 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 3, 2023-11-16T16:55:32.955 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 1768, 2023-11-16T16:55:32.955 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2023-11-16T16:55:32.955 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1768, 2023-11-16T16:55:32.955 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4.399913161, 2023-11-16T16:55:32.956 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002488638 2023-11-16T16:55:32.956 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.956 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2023-11-16T16:55:32.956 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2023-11-16T16:55:32.956 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1768, 2023-11-16T16:55:32.956 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1769 2023-11-16T16:55:32.956 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.957 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 1768, 2023-11-16T16:55:32.957 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 1768, 2023-11-16T16:55:32.957 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2023-11-16T16:55:32.957 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2023-11-16T16:55:32.957 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2023-11-16T16:55:32.957 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 1075, 2023-11-16T16:55:32.957 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2023-11-16T16:55:32.958 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 691, 2023-11-16T16:55:32.958 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2023-11-16T16:55:32.958 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2023-11-16T16:55:32.958 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2023-11-16T16:55:32.958 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2023-11-16T16:55:32.958 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2023-11-16T16:55:32.958 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2023-11-16T16:55:32.959 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2023-11-16T16:55:32.959 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2023-11-16T16:55:32.959 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2023-11-16T16:55:32.959 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2023-11-16T16:55:32.959 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 1, 2023-11-16T16:55:32.959 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2023-11-16T16:55:32.959 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2023-11-16T16:55:32.960 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2023-11-16T16:55:32.960 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2023-11-16T16:55:32.960 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2023-11-16T16:55:32.960 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2023-11-16T16:55:32.960 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2023-11-16T16:55:32.960 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2023-11-16T16:55:32.960 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2023-11-16T16:55:32.961 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2023-11-16T16:55:32.961 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2023-11-16T16:55:32.961 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2023-11-16T16:55:32.961 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2023-11-16T16:55:32.961 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2023-11-16T16:55:32.961 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2023-11-16T16:55:32.961 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2023-11-16T16:55:32.962 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2023-11-16T16:55:32.962 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2023-11-16T16:55:32.962 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2023-11-16T16:55:32.962 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2023-11-16T16:55:32.962 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2023-11-16T16:55:32.962 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2023-11-16T16:55:32.962 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2023-11-16T16:55:32.962 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2023-11-16T16:55:32.963 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2023-11-16T16:55:32.963 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2023-11-16T16:55:32.963 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2023-11-16T16:55:32.963 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 94, 2023-11-16T16:55:32.963 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2023-11-16T16:55:32.963 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 4, 2023-11-16T16:55:32.963 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 11, 2023-11-16T16:55:32.964 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 11, 2023-11-16T16:55:32.964 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2023-11-16T16:55:32.964 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2023-11-16T16:55:32.964 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2023-11-16T16:55:32.964 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2023-11-16T16:55:32.964 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2023-11-16T16:55:32.964 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.964 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2023-11-16T16:55:32.965 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2023-11-16T16:55:32.965 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2023-11-16T16:55:32.965 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2023-11-16T16:55:32.965 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2023-11-16T16:55:32.965 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.965 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2023-11-16T16:55:32.965 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2023-11-16T16:55:32.966 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.966 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2023-11-16T16:55:32.966 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2023-11-16T16:55:32.966 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2023-11-16T16:55:32.966 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2023-11-16T16:55:32.966 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2023-11-16T16:55:32.966 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2023-11-16T16:55:32.967 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2023-11-16T16:55:32.967 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.967 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:55:32.967 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:32.967 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-11-16T16:55:32.967 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:32.967 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 1942, 2023-11-16T16:55:32.968 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 196935323, 2023-11-16T16:55:32.968 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:32.968 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 1942, 2023-11-16T16:55:32.968 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-11-16T16:55:32.968 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-11-16T16:55:32.968 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1942, 2023-11-16T16:55:32.968 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 196935323, 2023-11-16T16:55:32.968 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:32.969 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.969 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.969 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.969 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:32.969 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.969 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2023-11-16T16:55:32.969 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:32.970 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-11-16T16:55:32.970 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:32.970 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-11-16T16:55:32.970 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-11-16T16:55:32.970 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:32.970 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:32.970 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 1768, 2023-11-16T16:55:32.971 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 196124581, 2023-11-16T16:55:32.971 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1768, 2023-11-16T16:55:32.971 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 196124581, 2023-11-16T16:55:32.971 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:32.971 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.971 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.971 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.972 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:32.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.972 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2023-11-16T16:55:32.972 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:32.972 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2023-11-16T16:55:32.972 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:32.972 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-11-16T16:55:32.973 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-11-16T16:55:32.973 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:32.973 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:32.973 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 1768, 2023-11-16T16:55:32.973 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 1768, 2023-11-16T16:55:32.973 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1768, 2023-11-16T16:55:32.973 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1768, 2023-11-16T16:55:32.973 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:32.974 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.974 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.974 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.974 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:32.974 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.974 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2023-11-16T16:55:32.974 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:32.975 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-11-16T16:55:32.975 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:32.975 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-11-16T16:55:32.975 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-11-16T16:55:32.975 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:32.975 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:32.975 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-11-16T16:55:32.975 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-11-16T16:55:32.976 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2023-11-16T16:55:32.976 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2023-11-16T16:55:32.976 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:32.976 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.976 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.976 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.976 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:32.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:32.977 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55f91528c460": { 2023-11-16T16:55:32.977 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:32.977 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-11-16T16:55:32.977 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:32.977 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-11-16T16:55:32.977 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-11-16T16:55:32.978 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:32.978 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:32.978 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-11-16T16:55:32.978 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-11-16T16:55:32.978 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2023-11-16T16:55:32.978 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2023-11-16T16:55:32.978 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:32.979 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:32.979 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:32.979 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:32.979 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:32.979 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:32.979 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-11-16T16:55:33.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:32 smithi125 ceph-mon[131076]: pgmap v554: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:55:33.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:32 smithi093 ceph-mon[97863]: pgmap v554: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:55:33.445 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-11-16T16:55:33.445 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-11-16T16:55:33.445 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-11-16T16:55:33.446 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-11-16T16:55:33.446 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-11-16T16:55:33.446 DEBUG:tasks.check_counter:Getting stats from i 2023-11-16T16:55:33.446 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-11-16T16:55:34.088 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:55:34.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:34 smithi159 ceph-mon[131528]: pgmap v555: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:55:35.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:34 smithi125 ceph-mon[131076]: pgmap v555: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:55:35.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:34 smithi093 ceph-mon[97863]: pgmap v555: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:55:35.393 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-11-16T16:55:35.393 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:55:35.393 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 24524, 2023-11-16T16:55:35.393 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 19840, 2023-11-16T16:55:35.393 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 49493113, 2023-11-16T16:55:35.394 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 76677094, 2023-11-16T16:55:35.394 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 8, 2023-11-16T16:55:35.394 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2023-11-16T16:55:35.394 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 1.452796299, 2023-11-16T16:55:35.394 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.508571931, 2023-11-16T16:55:35.394 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.624032269, 2023-11-16T16:55:35.395 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.415657465, 2023-11-16T16:55:35.395 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:35.395 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 19839, 2023-11-16T16:55:35.395 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.795102974, 2023-11-16T16:55:35.395 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000090483 2023-11-16T16:55:35.395 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.395 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:35.396 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 22760, 2023-11-16T16:55:35.396 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002983235, 2023-11-16T16:55:35.396 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000131 2023-11-16T16:55:35.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.396 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 511600, 2023-11-16T16:55:35.396 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 91552 2023-11-16T16:55:35.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.397 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:55:35.397 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 40277, 2023-11-16T16:55:35.397 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 26572, 2023-11-16T16:55:35.397 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 16207773, 2023-11-16T16:55:35.397 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 63428739, 2023-11-16T16:55:35.397 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 8, 2023-11-16T16:55:35.397 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2023-11-16T16:55:35.398 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 1.806520102, 2023-11-16T16:55:35.398 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.661327572, 2023-11-16T16:55:35.398 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 1.025419888, 2023-11-16T16:55:35.398 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.351890198, 2023-11-16T16:55:35.398 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:35.398 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 26573, 2023-11-16T16:55:35.399 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2.518791120, 2023-11-16T16:55:35.399 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000094787 2023-11-16T16:55:35.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.399 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:35.399 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 44048, 2023-11-16T16:55:35.399 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.005561957, 2023-11-16T16:55:35.399 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000126 2023-11-16T16:55:35.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.400 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-16T16:55:35.400 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 261840 2023-11-16T16:55:35.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.400 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:55:35.400 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 101869, 2023-11-16T16:55:35.400 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 189478, 2023-11-16T16:55:35.401 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 42489135, 2023-11-16T16:55:35.401 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 194878559, 2023-11-16T16:55:35.401 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 13, 2023-11-16T16:55:35.401 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 9, 2023-11-16T16:55:35.401 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 7.114757752, 2023-11-16T16:55:35.401 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 3.256746278, 2023-11-16T16:55:35.401 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 4.438629839, 2023-11-16T16:55:35.401 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.550070512, 2023-11-16T16:55:35.402 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:35.402 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 189474, 2023-11-16T16:55:35.402 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 13.973571844, 2023-11-16T16:55:35.402 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000073749 2023-11-16T16:55:35.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.402 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:35.403 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 206347, 2023-11-16T16:55:35.403 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.032589876, 2023-11-16T16:55:35.403 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000157 2023-11-16T16:55:35.403 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.403 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 33810768, 2023-11-16T16:55:35.403 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 107057920 2023-11-16T16:55:35.403 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.403 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2023-11-16T16:55:35.404 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2023-11-16T16:55:35.404 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2023-11-16T16:55:35.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.404 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2023-11-16T16:55:35.404 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-11-16T16:55:35.404 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-11-16T16:55:35.404 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2790, 2023-11-16T16:55:35.405 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 63.515868691, 2023-11-16T16:55:35.405 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.022765544 2023-11-16T16:55:35.405 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:35.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.405 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2023-11-16T16:55:35.405 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-11-16T16:55:35.405 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-11-16T16:55:35.406 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 28502, 2023-11-16T16:55:35.406 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 6.803256236, 2023-11-16T16:55:35.406 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000238693 2023-11-16T16:55:35.406 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:35.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.406 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2023-11-16T16:55:35.406 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 59321, 2023-11-16T16:55:35.407 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 52678, 2023-11-16T16:55:35.407 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2023-11-16T16:55:35.407 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 52678, 2023-11-16T16:55:35.407 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 166.333203465, 2023-11-16T16:55:35.407 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003157545 2023-11-16T16:55:35.407 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.407 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2023-11-16T16:55:35.408 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 6643, 2023-11-16T16:55:35.408 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 787, 2023-11-16T16:55:35.408 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2023-11-16T16:55:35.408 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 1560, 2023-11-16T16:55:35.408 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 185, 2023-11-16T16:55:35.408 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 69, 2023-11-16T16:55:35.408 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 60, 2023-11-16T16:55:35.409 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 14, 2023-11-16T16:55:35.409 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 35, 2023-11-16T16:55:35.409 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 11, 2023-11-16T16:55:35.409 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 16, 2023-11-16T16:55:35.409 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 16385, 2023-11-16T16:55:35.409 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 1, 2023-11-16T16:55:35.409 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 1, 2023-11-16T16:55:35.410 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 4, 2023-11-16T16:55:35.410 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 76584, 2023-11-16T16:55:35.410 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 69791, 2023-11-16T16:55:35.410 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 6138, 2023-11-16T16:55:35.410 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 5, 2023-11-16T16:55:35.410 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 2, 2023-11-16T16:55:35.411 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2023-11-16T16:55:35.411 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2023-11-16T16:55:35.411 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2023-11-16T16:55:35.411 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2023-11-16T16:55:35.411 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 663, 2023-11-16T16:55:35.411 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 13138, 2023-11-16T16:55:35.411 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 200, 2023-11-16T16:55:35.412 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2023-11-16T16:55:35.412 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2023-11-16T16:55:35.412 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2023-11-16T16:55:35.412 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2023-11-16T16:55:35.412 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 7, 2023-11-16T16:55:35.412 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 246, 2023-11-16T16:55:35.412 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2023-11-16T16:55:35.413 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2023-11-16T16:55:35.413 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2023-11-16T16:55:35.413 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2023-11-16T16:55:35.413 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2023-11-16T16:55:35.413 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2023-11-16T16:55:35.413 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2023-11-16T16:55:35.413 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2023-11-16T16:55:35.414 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2023-11-16T16:55:35.414 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 9, 2023-11-16T16:55:35.414 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 589, 2023-11-16T16:55:35.414 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 35605, 2023-11-16T16:55:35.414 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2023-11-16T16:55:35.414 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-16T16:55:35.415 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-16T16:55:35.415 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 24403, 2023-11-16T16:55:35.415 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 23804, 2023-11-16T16:55:35.415 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 327, 2023-11-16T16:55:35.415 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 32767 2023-11-16T16:55:35.415 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.415 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2023-11-16T16:55:35.415 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 1019, 2023-11-16T16:55:35.416 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 2038, 2023-11-16T16:55:35.416 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 1021, 2023-11-16T16:55:35.416 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 1447, 2023-11-16T16:55:35.416 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 1546, 2023-11-16T16:55:35.416 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 1, 2023-11-16T16:55:35.416 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2023-11-16T16:55:35.416 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2023-11-16T16:55:35.417 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 15867, 2023-11-16T16:55:35.417 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 15866, 2023-11-16T16:55:35.417 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2023-11-16T16:55:35.417 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2023-11-16T16:55:35.417 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2023-11-16T16:55:35.417 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2023-11-16T16:55:35.417 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2023-11-16T16:55:35.418 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2023-11-16T16:55:35.418 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2023-11-16T16:55:35.418 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2023-11-16T16:55:35.418 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 664, 2023-11-16T16:55:35.418 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2023-11-16T16:55:35.418 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 256, 2023-11-16T16:55:35.419 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2023-11-16T16:55:35.419 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2023-11-16T16:55:35.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.419 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2023-11-16T16:55:35.419 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 76592, 2023-11-16T16:55:35.419 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2023-11-16T16:55:35.419 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2023-11-16T16:55:35.420 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 76592, 2023-11-16T16:55:35.420 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2023-11-16T16:55:35.420 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2023-11-16T16:55:35.420 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 94, 2023-11-16T16:55:35.420 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2023-11-16T16:55:35.420 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2023-11-16T16:55:35.420 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 94, 2023-11-16T16:55:35.421 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2023-11-16T16:55:35.421 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2023-11-16T16:55:35.421 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2023-11-16T16:55:35.421 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 200282175, 2023-11-16T16:55:35.421 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2023-11-16T16:55:35.421 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2023-11-16T16:55:35.421 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1731, 2023-11-16T16:55:35.422 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 138.171323984, 2023-11-16T16:55:35.422 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.079821677 2023-11-16T16:55:35.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.422 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2023-11-16T16:55:35.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.422 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2023-11-16T16:55:35.422 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 41, 2023-11-16T16:55:35.423 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 30427, 2023-11-16T16:55:35.423 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 30386, 2023-11-16T16:55:35.423 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 39, 2023-11-16T16:55:35.423 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 2947, 2023-11-16T16:55:35.423 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 2908, 2023-11-16T16:55:35.423 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 60, 2023-11-16T16:55:35.423 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 52284, 2023-11-16T16:55:35.423 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 52224, 2023-11-16T16:55:35.424 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 1, 2023-11-16T16:55:35.424 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 52015, 2023-11-16T16:55:35.424 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 52014, 2023-11-16T16:55:35.424 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 177028, 2023-11-16T16:55:35.424 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 198964 2023-11-16T16:55:35.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.424 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2023-11-16T16:55:35.425 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 60568, 2023-11-16T16:55:35.425 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 103, 2023-11-16T16:55:35.425 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 59321, 2023-11-16T16:55:35.425 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 647, 2023-11-16T16:55:35.425 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 219, 2023-11-16T16:55:35.425 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2023-11-16T16:55:35.425 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 19732, 2023-11-16T16:55:35.425 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 89.594651923, 2023-11-16T16:55:35.426 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004540576 2023-11-16T16:55:35.426 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.426 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2023-11-16T16:55:35.426 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 27, 2023-11-16T16:55:35.426 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.074042823, 2023-11-16T16:55:35.426 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002742326 2023-11-16T16:55:35.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.427 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2023-11-16T16:55:35.427 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.427 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.427 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.427 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2023-11-16T16:55:35.427 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.428 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.428 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.428 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2023-11-16T16:55:35.428 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 16019, 2023-11-16T16:55:35.428 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 20.981205149, 2023-11-16T16:55:35.428 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001309769 2023-11-16T16:55:35.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.429 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2023-11-16T16:55:35.429 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.429 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.429 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.429 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2023-11-16T16:55:35.430 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2023-11-16T16:55:35.430 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000130618, 2023-11-16T16:55:35.430 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000130618 2023-11-16T16:55:35.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.430 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2023-11-16T16:55:35.430 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.430 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.431 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.431 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2023-11-16T16:55:35.431 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.431 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.431 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.432 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2023-11-16T16:55:35.432 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.432 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.432 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.432 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2023-11-16T16:55:35.432 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.433 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.433 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.433 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.433 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2023-11-16T16:55:35.433 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.433 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.433 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.434 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.434 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2023-11-16T16:55:35.434 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.434 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.434 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.434 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.434 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2023-11-16T16:55:35.435 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.435 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.435 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.435 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.435 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2023-11-16T16:55:35.435 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.435 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.436 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.436 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.436 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2023-11-16T16:55:35.436 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1032, 2023-11-16T16:55:35.436 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4.294235644, 2023-11-16T16:55:35.436 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004161081 2023-11-16T16:55:35.436 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.436 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2023-11-16T16:55:35.437 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.437 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.437 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.437 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2023-11-16T16:55:35.437 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.437 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.438 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.438 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.438 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2023-11-16T16:55:35.438 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.438 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.438 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.438 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.439 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2023-11-16T16:55:35.439 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2023-11-16T16:55:35.439 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.039075159, 2023-11-16T16:55:35.439 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.019537579 2023-11-16T16:55:35.439 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.439 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2023-11-16T16:55:35.439 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.439 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.440 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.440 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2023-11-16T16:55:35.440 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.440 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.440 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.441 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2023-11-16T16:55:35.441 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.441 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.441 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.441 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.441 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2023-11-16T16:55:35.441 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.442 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.442 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.442 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.442 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2023-11-16T16:55:35.442 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.442 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.442 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.443 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2023-11-16T16:55:35.443 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.443 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.443 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.443 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2023-11-16T16:55:35.443 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.444 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.444 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.444 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2023-11-16T16:55:35.444 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.444 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.444 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.445 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2023-11-16T16:55:35.445 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 15865, 2023-11-16T16:55:35.445 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 51.349862149, 2023-11-16T16:55:35.445 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003236675 2023-11-16T16:55:35.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.445 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2023-11-16T16:55:35.445 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2023-11-16T16:55:35.446 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2023-11-16T16:55:35.446 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.446 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.446 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.446 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:35.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.446 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2023-11-16T16:55:35.446 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2023-11-16T16:55:35.447 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 1, 2023-11-16T16:55:35.447 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 0, 2023-11-16T16:55:35.447 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2023-11-16T16:55:35.447 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2023-11-16T16:55:35.447 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 20369, 2023-11-16T16:55:35.447 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 20369, 2023-11-16T16:55:35.447 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 401 2023-11-16T16:55:35.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.448 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2023-11-16T16:55:35.448 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:55:35.448 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2023-11-16T16:55:35.448 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:55:35.448 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:55:35.448 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:55:35.448 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:55:35.448 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:55:35.449 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:55:35.449 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:55:35.449 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:55:35.449 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:55:35.449 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:55:35.449 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:55:35.449 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:55:35.450 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:55:35.450 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2023-11-16T16:55:35.450 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:55:35.450 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2023-11-16T16:55:35.450 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:55:35.450 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:55:35.450 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:55:35.451 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:55:35.451 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:55:35.451 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:55:35.451 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:55:35.451 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2023-11-16T16:55:35.451 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:55:35.451 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:55:35.451 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:55:35.452 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2023-11-16T16:55:35.452 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2023-11-16T16:55:35.452 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2023-11-16T16:55:35.452 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:55:35.452 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:55:35.452 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:55:35.452 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:55:35.452 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 290701, 2023-11-16T16:55:35.453 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 122, 2023-11-16T16:55:35.453 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 792, 2023-11-16T16:55:35.453 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 9, 2023-11-16T16:55:35.453 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2023-11-16T16:55:35.453 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2023-11-16T16:55:35.453 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:55:35.453 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2023-11-16T16:55:35.454 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:55:35.454 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2023-11-16T16:55:35.454 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9280, 2023-11-16T16:55:35.454 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 179, 2023-11-16T16:55:35.454 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:55:35.454 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:55:35.454 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2023-11-16T16:55:35.454 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2023-11-16T16:55:35.455 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 183408, 2023-11-16T16:55:35.455 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 948, 2023-11-16T16:55:35.455 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2023-11-16T16:55:35.455 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2023-11-16T16:55:35.455 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2023-11-16T16:55:35.455 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2023-11-16T16:55:35.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.456 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2023-11-16T16:55:35.456 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2023-11-16T16:55:35.456 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2023-11-16T16:55:35.456 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 37832, 2023-11-16T16:55:35.456 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 213461980, 2023-11-16T16:55:35.456 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 28, 2023-11-16T16:55:35.456 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 37832, 2023-11-16T16:55:35.456 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2023-11-16T16:55:35.457 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 37832, 2023-11-16T16:55:35.457 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 377.313630805, 2023-11-16T16:55:35.457 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.009973398 2023-11-16T16:55:35.457 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.457 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2023-11-16T16:55:35.457 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2023-11-16T16:55:35.457 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 37832, 2023-11-16T16:55:35.458 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 40191 2023-11-16T16:55:35.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.458 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 37832, 2023-11-16T16:55:35.458 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 1048, 2023-11-16T16:55:35.458 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 36784, 2023-11-16T16:55:35.458 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2023-11-16T16:55:35.458 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2023-11-16T16:55:35.459 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2023-11-16T16:55:35.459 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2023-11-16T16:55:35.459 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 261, 2023-11-16T16:55:35.459 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 1992, 2023-11-16T16:55:35.459 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 118, 2023-11-16T16:55:35.459 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2023-11-16T16:55:35.459 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2023-11-16T16:55:35.459 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2023-11-16T16:55:35.460 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2023-11-16T16:55:35.460 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 33066, 2023-11-16T16:55:35.460 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2023-11-16T16:55:35.460 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2023-11-16T16:55:35.460 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2023-11-16T16:55:35.460 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 0, 2023-11-16T16:55:35.460 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2023-11-16T16:55:35.461 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2023-11-16T16:55:35.461 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2023-11-16T16:55:35.461 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2023-11-16T16:55:35.461 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2023-11-16T16:55:35.461 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2023-11-16T16:55:35.461 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2023-11-16T16:55:35.461 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2023-11-16T16:55:35.461 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2023-11-16T16:55:35.462 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2023-11-16T16:55:35.462 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2023-11-16T16:55:35.462 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2023-11-16T16:55:35.462 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2023-11-16T16:55:35.462 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2023-11-16T16:55:35.462 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2023-11-16T16:55:35.462 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2023-11-16T16:55:35.463 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2023-11-16T16:55:35.463 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2023-11-16T16:55:35.463 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2023-11-16T16:55:35.463 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2023-11-16T16:55:35.463 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2023-11-16T16:55:35.463 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2023-11-16T16:55:35.463 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2023-11-16T16:55:35.463 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2023-11-16T16:55:35.464 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2023-11-16T16:55:35.464 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2023-11-16T16:55:35.464 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2023-11-16T16:55:35.464 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 94, 2023-11-16T16:55:35.464 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2023-11-16T16:55:35.464 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 4, 2023-11-16T16:55:35.464 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2023-11-16T16:55:35.465 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2023-11-16T16:55:35.465 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2023-11-16T16:55:35.465 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2023-11-16T16:55:35.465 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 3117, 2023-11-16T16:55:35.465 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 1574, 2023-11-16T16:55:35.465 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 33 2023-11-16T16:55:35.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.465 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2023-11-16T16:55:35.466 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2023-11-16T16:55:35.466 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2023-11-16T16:55:35.466 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2023-11-16T16:55:35.466 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2023-11-16T16:55:35.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.466 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2023-11-16T16:55:35.466 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 60345 2023-11-16T16:55:35.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.467 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2023-11-16T16:55:35.467 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2023-11-16T16:55:35.467 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 16, 2023-11-16T16:55:35.467 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2023-11-16T16:55:35.467 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 8, 2023-11-16T16:55:35.467 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 15866, 2023-11-16T16:55:35.467 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2023-11-16T16:55:35.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.468 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:55:35.468 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:35.468 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-11-16T16:55:35.468 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:35.468 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 166666, 2023-11-16T16:55:35.468 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 89090195, 2023-11-16T16:55:35.468 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:35.469 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 166666, 2023-11-16T16:55:35.469 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-11-16T16:55:35.469 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-11-16T16:55:35.469 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 166666, 2023-11-16T16:55:35.469 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 89090195, 2023-11-16T16:55:35.469 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:35.469 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.470 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.470 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.470 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:35.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.470 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2023-11-16T16:55:35.470 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:35.470 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-11-16T16:55:35.471 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:35.471 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-11-16T16:55:35.471 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-11-16T16:55:35.471 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:35.471 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:35.471 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 37832, 2023-11-16T16:55:35.471 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 215178846, 2023-11-16T16:55:35.471 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 37832, 2023-11-16T16:55:35.472 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 215178846, 2023-11-16T16:55:35.472 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:35.472 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.472 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.472 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.472 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:35.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.472 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2023-11-16T16:55:35.473 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:35.473 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2023-11-16T16:55:35.473 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:35.473 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-11-16T16:55:35.473 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-11-16T16:55:35.473 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:35.473 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-11-16T16:55:35.473 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 37832, 2023-11-16T16:55:35.474 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 37832, 2023-11-16T16:55:35.474 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 37832, 2023-11-16T16:55:35.474 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 37832, 2023-11-16T16:55:35.474 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:35.474 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.474 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.474 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.475 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:35.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.475 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2023-11-16T16:55:35.475 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:35.475 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-11-16T16:55:35.475 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:35.475 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 15866, 2023-11-16T16:55:35.475 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1729458, 2023-11-16T16:55:35.476 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:35.476 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 15866, 2023-11-16T16:55:35.476 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-11-16T16:55:35.476 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-11-16T16:55:35.476 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 261, 2023-11-16T16:55:35.476 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1729458, 2023-11-16T16:55:35.476 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:35.476 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.477 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.477 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.477 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:35.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:35.477 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x563b13fa60a0": { 2023-11-16T16:55:35.477 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:35.477 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-11-16T16:55:35.477 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:35.478 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 76592, 2023-11-16T16:55:35.478 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 196087871, 2023-11-16T16:55:35.478 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:35.478 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 76592, 2023-11-16T16:55:35.478 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-11-16T16:55:35.478 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-11-16T16:55:35.478 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1731, 2023-11-16T16:55:35.479 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 196087871, 2023-11-16T16:55:35.479 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:35.479 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:35.479 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:35.479 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:35.479 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:35.479 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:35.480 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-11-16T16:55:35.732 INFO:tasks.check_counter:Daemon mds.i mds.exported=663 2023-11-16T16:55:35.732 INFO:tasks.check_counter:Daemon mds.i mds.imported=200 2023-11-16T16:55:35.732 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=1019 2023-11-16T16:55:35.732 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=2038 2023-11-16T16:55:35.732 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=185 2023-11-16T16:55:35.732 DEBUG:tasks.check_counter:Getting stats from l 2023-11-16T16:55:35.732 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-11-16T16:55:36.371 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:55:36.943 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:36 smithi159 ceph-mon[131528]: pgmap v556: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:55:37.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:36 smithi125 ceph-mon[131076]: pgmap v556: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:55:37.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:36 smithi093 ceph-mon[97863]: pgmap v556: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:55:37.752 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-11-16T16:55:37.752 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:55:37.753 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 0, 2023-11-16T16:55:37.753 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2, 2023-11-16T16:55:37.753 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 0, 2023-11-16T16:55:37.753 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 0, 2023-11-16T16:55:37.753 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 4, 2023-11-16T16:55:37.753 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 0, 2023-11-16T16:55:37.753 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.001039402, 2023-11-16T16:55:37.753 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.000000000, 2023-11-16T16:55:37.754 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.000121218, 2023-11-16T16:55:37.754 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:55:37.754 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:37.754 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:37.754 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:37.754 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:37.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:37.754 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:37.755 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:37.755 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:37.755 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:37.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:37.755 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:55:37.755 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:55:37.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:37.755 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:55:37.755 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1, 2023-11-16T16:55:37.756 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 119, 2023-11-16T16:55:37.756 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 160, 2023-11-16T16:55:37.756 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 96720, 2023-11-16T16:55:37.756 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 3, 2023-11-16T16:55:37.756 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2023-11-16T16:55:37.756 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.006440984, 2023-11-16T16:55:37.756 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.004612235, 2023-11-16T16:55:37.756 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.000531450, 2023-11-16T16:55:37.756 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:55:37.757 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:37.757 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 117, 2023-11-16T16:55:37.757 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.009598080, 2023-11-16T16:55:37.757 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000082034 2023-11-16T16:55:37.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:37.757 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:37.757 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:37.757 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:37.757 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:37.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:37.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-16T16:55:37.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 96720 2023-11-16T16:55:37.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:37.758 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:55:37.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 165, 2023-11-16T16:55:37.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 160, 2023-11-16T16:55:37.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 458352, 2023-11-16T16:55:37.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 89664, 2023-11-16T16:55:37.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 2, 2023-11-16T16:55:37.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 1, 2023-11-16T16:55:37.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.025823018, 2023-11-16T16:55:37.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.009297093, 2023-11-16T16:55:37.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.012415520, 2023-11-16T16:55:37.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.004435508, 2023-11-16T16:55:37.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:55:37.760 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 160, 2023-11-16T16:55:37.760 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.008763921, 2023-11-16T16:55:37.760 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000054774 2023-11-16T16:55:37.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:37.760 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:55:37.760 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:37.760 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:37.760 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:37.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:37.761 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 458352, 2023-11-16T16:55:37.761 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 89664 2023-11-16T16:55:37.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:37.761 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2023-11-16T16:55:37.761 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 0, 2023-11-16T16:55:37.761 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2023-11-16T16:55:37.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:37.761 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2023-11-16T16:55:37.761 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:55:37.762 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2023-11-16T16:55:37.762 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:55:37.762 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:55:37.762 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:55:37.762 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:55:37.762 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:55:37.762 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:55:37.762 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:55:37.763 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:55:37.763 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:55:37.763 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:55:37.763 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:55:37.763 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:55:37.763 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:55:37.763 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2023-11-16T16:55:37.763 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:55:37.764 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2023-11-16T16:55:37.764 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:55:37.764 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:55:37.764 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:55:37.764 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:55:37.764 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:55:37.764 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:55:37.764 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:55:37.764 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2023-11-16T16:55:37.765 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:55:37.765 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:55:37.765 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:55:37.765 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2023-11-16T16:55:37.765 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2023-11-16T16:55:37.765 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2023-11-16T16:55:37.765 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:55:37.765 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:55:37.765 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:55:37.766 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:55:37.766 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 148923, 2023-11-16T16:55:37.766 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 29, 2023-11-16T16:55:37.766 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:55:37.766 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2023-11-16T16:55:37.766 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2023-11-16T16:55:37.766 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2023-11-16T16:55:37.767 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:55:37.767 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2023-11-16T16:55:37.767 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:55:37.767 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2023-11-16T16:55:37.767 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 0, 2023-11-16T16:55:37.767 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 0, 2023-11-16T16:55:37.767 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:55:37.767 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:55:37.768 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2023-11-16T16:55:37.768 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2023-11-16T16:55:37.768 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 928, 2023-11-16T16:55:37.768 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 2, 2023-11-16T16:55:37.768 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2023-11-16T16:55:37.768 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2023-11-16T16:55:37.768 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2023-11-16T16:55:37.768 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2023-11-16T16:55:37.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-11-16T16:55:37.769 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:55:37.769 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-11-16T16:55:37.769 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-11-16T16:55:37.769 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-11-16T16:55:37.769 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 162, 2023-11-16T16:55:37.769 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 432617, 2023-11-16T16:55:37.769 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-11-16T16:55:37.769 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 162, 2023-11-16T16:55:37.770 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-11-16T16:55:37.770 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-11-16T16:55:37.770 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 162, 2023-11-16T16:55:37.770 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 432617, 2023-11-16T16:55:37.770 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-11-16T16:55:37.770 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-11-16T16:55:37.770 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-11-16T16:55:37.770 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-11-16T16:55:37.771 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:37.771 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-11-16T16:55:37.771 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-11-16T16:55:38.126 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-11-16T16:55:38.127 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-11-16T16:55:38.127 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-11-16T16:55:38.127 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-11-16T16:55:38.127 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-11-16T16:55:38.127 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-16T16:55:38.137 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-16T16:55:38.138 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:55:38.138 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-16T16:55:38.208 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-16T16:55:38.208 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-16T16:55:38.258 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:55:38.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.476+0000 7fbd66dff700 1 -- 172.21.15.93:0/2905671788 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd60101e30 msgr2=0x7fbd601086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:38.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.476+0000 7fbd66dff700 1 --2- 172.21.15.93:0/2905671788 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd60101e30 0x7fbd601086f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800b0a0 tx=0x7fbd5800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.476+0000 7fbd66dff700 1 -- 172.21.15.93:0/2905671788 shutdown_connections 2023-11-16T16:55:38.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.476+0000 7fbd66dff700 1 --2- 172.21.15.93:0/2905671788 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd60101e30 0x7fbd601086f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.476+0000 7fbd66dff700 1 --2- 172.21.15.93:0/2905671788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd601002b0 0x7fbd601006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.476+0000 7fbd66dff700 1 --2- 172.21.15.93:0/2905671788 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd601014a0 0x7fbd601018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.476+0000 7fbd66dff700 1 -- 172.21.15.93:0/2905671788 >> 172.21.15.93:0/2905671788 conn(0x7fbd600fb890 msgr2=0x7fbd600fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:38.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.476+0000 7fbd66dff700 1 -- 172.21.15.93:0/2905671788 shutdown_connections 2023-11-16T16:55:38.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.476+0000 7fbd66dff700 1 -- 172.21.15.93:0/2905671788 wait complete. 2023-11-16T16:55:38.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.477+0000 7fbd66dff700 1 Processor -- start 2023-11-16T16:55:38.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.478+0000 7fbd66dff700 1 -- start start 2023-11-16T16:55:38.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.478+0000 7fbd66dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd601002b0 0x7fbd60198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:38.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.478+0000 7fbd66dff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd601014a0 0x7fbd601987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:38.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.478+0000 7fbd66dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd60101e30 0x7fbd6019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:38.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.478+0000 7fbd66dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6019fe10 con 0x7fbd601002b0 2023-11-16T16:55:38.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.478+0000 7fbd66dff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6019ff50 con 0x7fbd601014a0 2023-11-16T16:55:38.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.478+0000 7fbd66dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd601a00e0 con 0x7fbd60101e30 2023-11-16T16:55:38.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.478+0000 7fbd64b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd601002b0 0x7fbd60198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:38.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.478+0000 7fbd64b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd601002b0 0x7fbd60198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59850/0 (socket says 172.21.15.93:59850) 2023-11-16T16:55:38.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.478+0000 7fbd64b9b700 1 -- 172.21.15.93:0/1781234611 learned_addr learned my addr 172.21.15.93:0/1781234611 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:38.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.478+0000 7fbd5ffff700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd601014a0 0x7fbd601987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:38.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.478+0000 7fbd6539c700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd60101e30 0x7fbd6019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:38.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.479+0000 7fbd64b9b700 1 -- 172.21.15.93:0/1781234611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd60101e30 msgr2=0x7fbd6019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:38.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.479+0000 7fbd64b9b700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd60101e30 0x7fbd6019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.479+0000 7fbd64b9b700 1 -- 172.21.15.93:0/1781234611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd601014a0 msgr2=0x7fbd601987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:38.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.479+0000 7fbd64b9b700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd601014a0 0x7fbd601987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.479+0000 7fbd64b9b700 1 -- 172.21.15.93:0/1781234611 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd58009d00 con 0x7fbd601002b0 2023-11-16T16:55:38.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.479+0000 7fbd64b9b700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd601002b0 0x7fbd60198290 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400e970 tx=0x7fbd5400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:38.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.479+0000 7fbd5dffb700 1 -- 172.21.15.93:0/1781234611 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54009790 con 0x7fbd601002b0 2023-11-16T16:55:38.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.479+0000 7fbd5dffb700 1 -- 172.21.15.93:0/1781234611 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd5400c7e0 con 0x7fbd601002b0 2023-11-16T16:55:38.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.479+0000 7fbd66dff700 1 -- 172.21.15.93:0/1781234611 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6010bc80 con 0x7fbd601002b0 2023-11-16T16:55:38.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.479+0000 7fbd5dffb700 1 -- 172.21.15.93:0/1781234611 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54003c70 con 0x7fbd601002b0 2023-11-16T16:55:38.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.479+0000 7fbd66dff700 1 -- 172.21.15.93:0/1781234611 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6010baf0 con 0x7fbd601002b0 2023-11-16T16:55:38.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.480+0000 7fbd66dff700 1 -- 172.21.15.93:0/1781234611 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd601a07e0 con 0x7fbd601002b0 2023-11-16T16:55:38.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.482+0000 7fbd5dffb700 1 -- 172.21.15.93:0/1781234611 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbd5400cc50 con 0x7fbd601002b0 2023-11-16T16:55:38.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.483+0000 7fbd5dffb700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbd500a1150 0x7fbd500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:38.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.484+0000 7fbd5dffb700 1 -- 172.21.15.93:0/1781234611 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fbd54014070 con 0x7fbd601002b0 2023-11-16T16:55:38.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.484+0000 7fbd5ffff700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbd500a1150 0x7fbd500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:38.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.484+0000 7fbd5ffff700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbd500a1150 0x7fbd500a3600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009c60 tx=0x7fbd4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:38.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.487+0000 7fbd5dffb700 1 -- 172.21.15.93:0/1781234611 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbd540d2020 con 0x7fbd601002b0 2023-11-16T16:55:38.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.693+0000 7fbd66dff700 1 -- 172.21.15.93:0/1781234611 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd601a0e80 con 0x7fbd601002b0 2023-11-16T16:55:38.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.694+0000 7fbd5dffb700 1 -- 172.21.15.93:0/1781234611 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24795 (secure 0 0 0) 0x7fbd540961f0 con 0x7fbd601002b0 2023-11-16T16:55:38.697 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:55:38.697 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":94,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:53:48.219838+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":1,"to":2}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:55:38.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.697+0000 7fbd66dff700 1 -- 172.21.15.93:0/1781234611 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbd500a1150 msgr2=0x7fbd500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:38.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.697+0000 7fbd66dff700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbd500a1150 0x7fbd500a3600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009c60 tx=0x7fbd4c009260 comp rx=0 tx=0).stop 2023-11-16T16:55:38.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.697+0000 7fbd66dff700 1 -- 172.21.15.93:0/1781234611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd601002b0 msgr2=0x7fbd60198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:38.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.697+0000 7fbd66dff700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd601002b0 0x7fbd60198290 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400e970 tx=0x7fbd5400ec80 comp rx=0 tx=0).stop 2023-11-16T16:55:38.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.697+0000 7fbd66dff700 1 -- 172.21.15.93:0/1781234611 shutdown_connections 2023-11-16T16:55:38.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.697+0000 7fbd66dff700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbd500a1150 0x7fbd500a3600 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.697+0000 7fbd66dff700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd601002b0 0x7fbd60198290 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.697+0000 7fbd66dff700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd60101e30 0x7fbd6019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.697+0000 7fbd66dff700 1 --2- 172.21.15.93:0/1781234611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd601014a0 0x7fbd601987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.697+0000 7fbd66dff700 1 -- 172.21.15.93:0/1781234611 >> 172.21.15.93:0/1781234611 conn(0x7fbd600fb890 msgr2=0x7fbd600fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:38.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.697+0000 7fbd66dff700 1 -- 172.21.15.93:0/1781234611 shutdown_connections 2023-11-16T16:55:38.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.697+0000 7fbd66dff700 1 -- 172.21.15.93:0/1781234611 wait complete. 2023-11-16T16:55:38.719 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:42:13.029135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:46:35.301321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:46:36.552545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-16T16:46:39.628380+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-16T16:55:38.720 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:55:38.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.919+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2213320414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e41014b0 msgr2=0x7f39e4101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:38.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.919+0000 7f39ec1ff700 1 --2- 172.21.15.93:0/2213320414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e41014b0 0x7f39e4101900 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009990 tx=0x7f39d4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.920+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2213320414 shutdown_connections 2023-11-16T16:55:38.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.920+0000 7f39ec1ff700 1 --2- 172.21.15.93:0/2213320414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e4101e40 0x7f39e4108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.920+0000 7f39ec1ff700 1 --2- 172.21.15.93:0/2213320414 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e41002c0 0x7f39e41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.920+0000 7f39ec1ff700 1 --2- 172.21.15.93:0/2213320414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e41014b0 0x7f39e4101900 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.920+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2213320414 >> 172.21.15.93:0/2213320414 conn(0x7f39e40fb8a0 msgr2=0x7f39e40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:38.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.920+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2213320414 shutdown_connections 2023-11-16T16:55:38.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.920+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2213320414 wait complete. 2023-11-16T16:55:38.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39ec1ff700 1 Processor -- start 2023-11-16T16:55:38.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39ec1ff700 1 -- start start 2023-11-16T16:55:38.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39ec1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e41002c0 0x7f39e4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:38.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39ec1ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e41014b0 0x7f39e41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:38.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39ec1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4101e40 0x7f39e419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:38.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39ec1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e419fde0 con 0x7f39e4101e40 2023-11-16T16:55:38.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39ec1ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e419ff20 con 0x7f39e41014b0 2023-11-16T16:55:38.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39ec1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e41a00b0 con 0x7f39e41002c0 2023-11-16T16:55:38.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39ea79c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4101e40 0x7f39e419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:38.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39ea79c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4101e40 0x7f39e419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59868/0 (socket says 172.21.15.93:59868) 2023-11-16T16:55:38.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39ea79c700 1 -- 172.21.15.93:0/2067682775 learned_addr learned my addr 172.21.15.93:0/2067682775 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:38.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39e979a700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e41014b0 0x7f39e41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:38.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.921+0000 7f39e9f9b700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e41002c0 0x7f39e4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:38.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.922+0000 7f39ea79c700 1 -- 172.21.15.93:0/2067682775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e41002c0 msgr2=0x7f39e4198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:38.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.922+0000 7f39ea79c700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e41002c0 0x7f39e4198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.922+0000 7f39ea79c700 1 -- 172.21.15.93:0/2067682775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e41014b0 msgr2=0x7f39e41987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:38.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.922+0000 7f39ea79c700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e41014b0 0x7f39e41987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:38.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.922+0000 7f39ea79c700 1 -- 172.21.15.93:0/2067682775 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d40095f0 con 0x7f39e4101e40 2023-11-16T16:55:38.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.922+0000 7f39e9f9b700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e41002c0 0x7f39e4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:55:38.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.922+0000 7f39ea79c700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4101e40 0x7f39e419f810 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00ef30 tx=0x7f39dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:38.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.922+0000 7f39daffd700 1 -- 172.21.15.93:0/2067682775 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc018070 con 0x7f39e4101e40 2023-11-16T16:55:38.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.922+0000 7f39daffd700 1 -- 172.21.15.93:0/2067682775 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39dc00f040 con 0x7f39e4101e40 2023-11-16T16:55:38.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.922+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2067682775 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e410bc70 con 0x7f39e4101e40 2023-11-16T16:55:38.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.922+0000 7f39daffd700 1 -- 172.21.15.93:0/2067682775 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00cdb0 con 0x7f39e4101e40 2023-11-16T16:55:38.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.922+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2067682775 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e410bae0 con 0x7f39e4101e40 2023-11-16T16:55:38.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.923+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2067682775 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e41a0740 con 0x7f39e4101e40 2023-11-16T16:55:38.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.925+0000 7f39daffd700 1 -- 172.21.15.93:0/2067682775 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f39dc003a00 con 0x7f39e4101e40 2023-11-16T16:55:38.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.929+0000 7f39daffd700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f39d00a1150 0x7f39d00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:38.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.929+0000 7f39daffd700 1 -- 172.21.15.93:0/2067682775 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f39dc0cc940 con 0x7f39e4101e40 2023-11-16T16:55:38.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.929+0000 7f39e9f9b700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f39d00a1150 0x7f39d00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:38.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.929+0000 7f39e9f9b700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f39d00a1150 0x7f39d00a3600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f39e4101310 tx=0x7f39e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:38.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:38.932+0000 7f39daffd700 1 -- 172.21.15.93:0/2067682775 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f39dc095cf0 con 0x7f39e4101e40 2023-11-16T16:55:38.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:38 smithi159 ceph-mon[131528]: pgmap v557: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-11-16T16:55:39.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:38 smithi125 ceph-mon[131076]: pgmap v557: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-11-16T16:55:39.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.130+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2067682775 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39e41a0cf0 con 0x7f39e4101e40 2023-11-16T16:55:39.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.130+0000 7f39daffd700 1 -- 172.21.15.93:0/2067682775 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39dc099310 con 0x7f39e4101e40 2023-11-16T16:55:39.133 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-11-16T16:55:39.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.133+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2067682775 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f39d00a1150 msgr2=0x7f39d00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:39.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.133+0000 7f39ec1ff700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f39d00a1150 0x7f39d00a3600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f39e4101310 tx=0x7f39e0009260 comp rx=0 tx=0).stop 2023-11-16T16:55:39.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.133+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2067682775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4101e40 msgr2=0x7f39e419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:39.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.133+0000 7f39ec1ff700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4101e40 0x7f39e419f810 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00ef30 tx=0x7f39dc00c3a0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.133+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2067682775 shutdown_connections 2023-11-16T16:55:39.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.133+0000 7f39ec1ff700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f39d00a1150 0x7f39d00a3600 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.133+0000 7f39ec1ff700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39e4101e40 0x7f39e419f810 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.133+0000 7f39ec1ff700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39e41002c0 0x7f39e4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.133+0000 7f39ec1ff700 1 --2- 172.21.15.93:0/2067682775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39e41014b0 0x7f39e41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.133+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2067682775 >> 172.21.15.93:0/2067682775 conn(0x7f39e40fb8a0 msgr2=0x7f39e4106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:39.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.133+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2067682775 shutdown_connections 2023-11-16T16:55:39.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.133+0000 7f39ec1ff700 1 -- 172.21.15.93:0/2067682775 wait complete. 2023-11-16T16:55:39.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:38 smithi093 ceph-mon[97863]: pgmap v557: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-11-16T16:55:39.154 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:55:39.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.353+0000 7f90625ff700 1 -- 172.21.15.93:0/3758207353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f905c103700 msgr2=0x7f905c103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:39.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.353+0000 7f90625ff700 1 --2- 172.21.15.93:0/3758207353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f905c103700 0x7f905c103b50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9044009990 tx=0x7f9044009ca0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.354+0000 7f90625ff700 1 -- 172.21.15.93:0/3758207353 shutdown_connections 2023-11-16T16:55:39.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.354+0000 7f90625ff700 1 --2- 172.21.15.93:0/3758207353 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f905c104090 0x7f905c10a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.354+0000 7f90625ff700 1 --2- 172.21.15.93:0/3758207353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f905c102510 0x7f905c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.354+0000 7f90625ff700 1 --2- 172.21.15.93:0/3758207353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f905c103700 0x7f905c103b50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.354+0000 7f90625ff700 1 -- 172.21.15.93:0/3758207353 >> 172.21.15.93:0/3758207353 conn(0x7f905c0fdb30 msgr2=0x7f905c0fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:39.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.354+0000 7f90625ff700 1 -- 172.21.15.93:0/3758207353 shutdown_connections 2023-11-16T16:55:39.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.354+0000 7f90625ff700 1 -- 172.21.15.93:0/3758207353 wait complete. 2023-11-16T16:55:39.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f90625ff700 1 Processor -- start 2023-11-16T16:55:39.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f90625ff700 1 -- start start 2023-11-16T16:55:39.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f90625ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f905c102510 0x7f905c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:39.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f90625ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f905c103700 0x7f905c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:39.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f90625ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f905c104090 0x7f905c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:39.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f90625ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c19fe10 con 0x7f905c102510 2023-11-16T16:55:39.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f90625ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c19ff50 con 0x7f905c103700 2023-11-16T16:55:39.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f90625ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c1a00e0 con 0x7f905c104090 2023-11-16T16:55:39.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f905bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f905c102510 0x7f905c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:39.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f905bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f905c102510 0x7f905c198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59888/0 (socket says 172.21.15.93:59888) 2023-11-16T16:55:39.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f905bfff700 1 -- 172.21.15.93:0/3901734449 learned_addr learned my addr 172.21.15.93:0/3901734449 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:39.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f9053fff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f905c103700 0x7f905c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:39.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f9060b9c700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f905c104090 0x7f905c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:39.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f905bfff700 1 -- 172.21.15.93:0/3901734449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f905c104090 msgr2=0x7f905c19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:39.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f905bfff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f905c104090 0x7f905c19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f905bfff700 1 -- 172.21.15.93:0/3901734449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f905c103700 msgr2=0x7f905c1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:39.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f905bfff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f905c103700 0x7f905c1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f905bfff700 1 -- 172.21.15.93:0/3901734449 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440095f0 con 0x7f905c102510 2023-11-16T16:55:39.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f9053fff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f905c103700 0x7f905c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:55:39.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f905bfff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f905c102510 0x7f905c198290 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f904c00e970 tx=0x7f904c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:39.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f9059ffb700 1 -- 172.21.15.93:0/3901734449 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c009790 con 0x7f905c102510 2023-11-16T16:55:39.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f90625ff700 1 -- 172.21.15.93:0/3901734449 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905c10ded0 con 0x7f905c102510 2023-11-16T16:55:39.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f9059ffb700 1 -- 172.21.15.93:0/3901734449 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f904c00c7e0 con 0x7f905c102510 2023-11-16T16:55:39.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f9059ffb700 1 -- 172.21.15.93:0/3901734449 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c003c70 con 0x7f905c102510 2023-11-16T16:55:39.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.355+0000 7f90625ff700 1 -- 172.21.15.93:0/3901734449 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c10dd40 con 0x7f905c102510 2023-11-16T16:55:39.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.356+0000 7f90625ff700 1 -- 172.21.15.93:0/3901734449 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905c109ab0 con 0x7f905c102510 2023-11-16T16:55:39.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.359+0000 7f9059ffb700 1 -- 172.21.15.93:0/3901734449 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f904c00cc50 con 0x7f905c102510 2023-11-16T16:55:39.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.360+0000 7f9059ffb700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f90480a1150 0x7f90480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:39.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.360+0000 7f9059ffb700 1 -- 172.21.15.93:0/3901734449 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f904c014070 con 0x7f905c102510 2023-11-16T16:55:39.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.360+0000 7f9053fff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f90480a1150 0x7f90480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:39.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.362+0000 7f9053fff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f90480a1150 0x7f90480a3600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f904400b2e0 tx=0x7f9044005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:39.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.365+0000 7f9059ffb700 1 -- 172.21.15.93:0/3901734449 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f904c096210 con 0x7f905c102510 2023-11-16T16:55:39.563 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.562+0000 7f90625ff700 1 -- 172.21.15.93:0/3901734449 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f905c061970 con 0x7f905c102510 2023-11-16T16:55:39.564 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.563+0000 7f9059ffb700 1 -- 172.21.15.93:0/3901734449 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f904c003dd0 con 0x7f905c102510 2023-11-16T16:55:39.564 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:55:39.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.565+0000 7f90625ff700 1 -- 172.21.15.93:0/3901734449 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f90480a1150 msgr2=0x7f90480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:39.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.565+0000 7f90625ff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f90480a1150 0x7f90480a3600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f904400b2e0 tx=0x7f9044005fb0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.565+0000 7f90625ff700 1 -- 172.21.15.93:0/3901734449 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f905c102510 msgr2=0x7f905c198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:39.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.565+0000 7f90625ff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f905c102510 0x7f905c198290 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f904c00e970 tx=0x7f904c00ec80 comp rx=0 tx=0).stop 2023-11-16T16:55:39.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.565+0000 7f90625ff700 1 -- 172.21.15.93:0/3901734449 shutdown_connections 2023-11-16T16:55:39.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.565+0000 7f90625ff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f90480a1150 0x7f90480a3600 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.565+0000 7f90625ff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f905c102510 0x7f905c198290 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.565+0000 7f90625ff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f905c104090 0x7f905c19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.565+0000 7f90625ff700 1 --2- 172.21.15.93:0/3901734449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f905c103700 0x7f905c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.565+0000 7f90625ff700 1 -- 172.21.15.93:0/3901734449 >> 172.21.15.93:0/3901734449 conn(0x7f905c0fdb30 msgr2=0x7f905c106bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:39.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.565+0000 7f90625ff700 1 -- 172.21.15.93:0/3901734449 shutdown_connections 2023-11-16T16:55:39.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.565+0000 7f90625ff700 1 -- 172.21.15.93:0/3901734449 wait complete. 2023-11-16T16:55:39.585 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:55:39.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.785+0000 7f35331ff700 1 -- 172.21.15.93:0/4057240323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c100d40 msgr2=0x7f352c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:39.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.785+0000 7f35331ff700 1 --2- 172.21.15.93:0/4057240323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c100d40 0x7f352c103120 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3520009990 tx=0x7f3520009ca0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.786+0000 7f35331ff700 1 -- 172.21.15.93:0/4057240323 shutdown_connections 2023-11-16T16:55:39.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.786+0000 7f35331ff700 1 --2- 172.21.15.93:0/4057240323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c100d40 0x7f352c103120 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.786+0000 7f35331ff700 1 --2- 172.21.15.93:0/4057240323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352c105f80 0x7f352c1083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.786+0000 7f35331ff700 1 --2- 172.21.15.93:0/4057240323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f352c103660 0x7f352c105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.786+0000 7f35331ff700 1 -- 172.21.15.93:0/4057240323 >> 172.21.15.93:0/4057240323 conn(0x7f352c0fa7c0 msgr2=0x7f352c0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:39.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.786+0000 7f35331ff700 1 -- 172.21.15.93:0/4057240323 shutdown_connections 2023-11-16T16:55:39.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.786+0000 7f35331ff700 1 -- 172.21.15.93:0/4057240323 wait complete. 2023-11-16T16:55:39.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.787+0000 7f35331ff700 1 Processor -- start 2023-11-16T16:55:39.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.787+0000 7f35331ff700 1 -- start start 2023-11-16T16:55:39.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.787+0000 7f35331ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f352c100d40 0x7f352c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:39.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.787+0000 7f35331ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352c103660 0x7f352c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:39.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.787+0000 7f35331ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c105f80 0x7f352c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:39.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.787+0000 7f35331ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c19fdb0 con 0x7f352c105f80 2023-11-16T16:55:39.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.787+0000 7f35331ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c19fef0 con 0x7f352c103660 2023-11-16T16:55:39.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.787+0000 7f35331ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c1a0080 con 0x7f352c100d40 2023-11-16T16:55:39.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.787+0000 7f353179c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c105f80 0x7f352c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:39.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.787+0000 7f353179c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c105f80 0x7f352c19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59898/0 (socket says 172.21.15.93:59898) 2023-11-16T16:55:39.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.787+0000 7f353179c700 1 -- 172.21.15.93:0/2992326322 learned_addr learned my addr 172.21.15.93:0/2992326322 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:39.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f352bfff700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352c103660 0x7f352c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:39.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f3530f9b700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f352c100d40 0x7f352c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:39.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f353179c700 1 -- 172.21.15.93:0/2992326322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f352c100d40 msgr2=0x7f352c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:39.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f353179c700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f352c100d40 0x7f352c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f353179c700 1 -- 172.21.15.93:0/2992326322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352c103660 msgr2=0x7f352c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:39.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f353179c700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352c103660 0x7f352c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:39.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f353179c700 1 -- 172.21.15.93:0/2992326322 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35200095f0 con 0x7f352c105f80 2023-11-16T16:55:39.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f352bfff700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352c103660 0x7f352c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:55:39.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f353179c700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c105f80 0x7f352c19f7e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3524007ee0 tx=0x7f352400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:39.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f3529ffb700 1 -- 172.21.15.93:0/2992326322 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524017070 con 0x7f352c105f80 2023-11-16T16:55:39.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f3529ffb700 1 -- 172.21.15.93:0/2992326322 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f352400f040 con 0x7f352c105f80 2023-11-16T16:55:39.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f35331ff700 1 -- 172.21.15.93:0/2992326322 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c10bca0 con 0x7f352c105f80 2023-11-16T16:55:39.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.788+0000 7f3529ffb700 1 -- 172.21.15.93:0/2992326322 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524005580 con 0x7f352c105f80 2023-11-16T16:55:39.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.789+0000 7f35331ff700 1 -- 172.21.15.93:0/2992326322 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c10bb10 con 0x7f352c105f80 2023-11-16T16:55:39.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.790+0000 7f35331ff700 1 -- 172.21.15.93:0/2992326322 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c107af0 con 0x7f352c105f80 2023-11-16T16:55:39.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.791+0000 7f3529ffb700 1 -- 172.21.15.93:0/2992326322 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f352400db20 con 0x7f352c105f80 2023-11-16T16:55:39.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.793+0000 7f3529ffb700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f351c0a1150 0x7f351c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:39.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.793+0000 7f3529ffb700 1 -- 172.21.15.93:0/2992326322 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f35240cc450 con 0x7f352c105f80 2023-11-16T16:55:39.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.793+0000 7f3530f9b700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f351c0a1150 0x7f351c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:39.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.794+0000 7f3530f9b700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f351c0a1150 0x7f351c0a3600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f352000b5c0 tx=0x7f3520005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:39.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.797+0000 7f3529ffb700 1 -- 172.21.15.93:0/2992326322 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3524095770 con 0x7f352c105f80 2023-11-16T16:55:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:39 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1781234611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:55:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:39 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2067682775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:55:39.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:39 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/3901734449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:55:39.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:39 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1781234611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:55:39.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:39 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2067682775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:55:39.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:39 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/3901734449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:55:40.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.999+0000 7f35331ff700 1 -- 172.21.15.93:0/2992326322 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f352c061c40 con 0x7f352c105f80 2023-11-16T16:55:40.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:39.999+0000 7f3529ffb700 1 -- 172.21.15.93:0/2992326322 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3524098d90 con 0x7f352c105f80 2023-11-16T16:55:40.001 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-11-16T16:55:40.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.002+0000 7f35331ff700 1 -- 172.21.15.93:0/2992326322 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f351c0a1150 msgr2=0x7f351c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:40.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.002+0000 7f35331ff700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f351c0a1150 0x7f351c0a3600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f352000b5c0 tx=0x7f3520005fb0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.002+0000 7f35331ff700 1 -- 172.21.15.93:0/2992326322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c105f80 msgr2=0x7f352c19f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:40.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.002+0000 7f35331ff700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c105f80 0x7f352c19f7e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3524007ee0 tx=0x7f352400d3b0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.002+0000 7f35331ff700 1 -- 172.21.15.93:0/2992326322 shutdown_connections 2023-11-16T16:55:40.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.002+0000 7f35331ff700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f351c0a1150 0x7f351c0a3600 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.002+0000 7f35331ff700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c105f80 0x7f352c19f7e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.002+0000 7f35331ff700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f352c100d40 0x7f352c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.002+0000 7f35331ff700 1 --2- 172.21.15.93:0/2992326322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352c103660 0x7f352c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.002+0000 7f35331ff700 1 -- 172.21.15.93:0/2992326322 >> 172.21.15.93:0/2992326322 conn(0x7f352c0fa7c0 msgr2=0x7f352c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:40.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.002+0000 7f35331ff700 1 -- 172.21.15.93:0/2992326322 shutdown_connections 2023-11-16T16:55:40.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.002+0000 7f35331ff700 1 -- 172.21.15.93:0/2992326322 wait complete. 2023-11-16T16:55:40.022 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-16T16:55:40.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:39 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1781234611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:55:40.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:39 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2067682775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:55:40.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:39 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/3901734449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:55:40.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.220+0000 7f43d25ff700 1 -- 172.21.15.93:0/3233579541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43cc1024e0 msgr2=0x7f43cc1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:40.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.220+0000 7f43d25ff700 1 --2- 172.21.15.93:0/3233579541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43cc1024e0 0x7f43cc1028f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009990 tx=0x7f43bc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.220+0000 7f43d25ff700 1 -- 172.21.15.93:0/3233579541 shutdown_connections 2023-11-16T16:55:40.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.220+0000 7f43d25ff700 1 --2- 172.21.15.93:0/3233579541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43cc1024e0 0x7f43cc1028f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.220+0000 7f43d25ff700 1 --2- 172.21.15.93:0/3233579541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43cc104060 0x7f43cc10a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.220+0000 7f43d25ff700 1 --2- 172.21.15.93:0/3233579541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43cc1036d0 0x7f43cc103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.220+0000 7f43d25ff700 1 -- 172.21.15.93:0/3233579541 >> 172.21.15.93:0/3233579541 conn(0x7f43cc0fdb20 msgr2=0x7f43cc0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:40.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.220+0000 7f43d25ff700 1 -- 172.21.15.93:0/3233579541 shutdown_connections 2023-11-16T16:55:40.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.220+0000 7f43d25ff700 1 -- 172.21.15.93:0/3233579541 wait complete. 2023-11-16T16:55:40.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.221+0000 7f43d25ff700 1 Processor -- start 2023-11-16T16:55:40.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.222+0000 7f43d25ff700 1 -- start start 2023-11-16T16:55:40.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.222+0000 7f43d25ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43cc1024e0 0x7f43cc1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:40.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.222+0000 7f43d25ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43cc1036d0 0x7f43cc198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:40.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.222+0000 7f43d25ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43cc104060 0x7f43cc19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:40.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.222+0000 7f43d25ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc19fe50 con 0x7f43cc1036d0 2023-11-16T16:55:40.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.222+0000 7f43d25ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc19ff90 con 0x7f43cc104060 2023-11-16T16:55:40.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.222+0000 7f43d25ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc1a0120 con 0x7f43cc1024e0 2023-11-16T16:55:40.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.222+0000 7f43cb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43cc1036d0 0x7f43cc198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:40.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.222+0000 7f43cbfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43cc1024e0 0x7f43cc1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:40.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.222+0000 7f43cb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43cc1036d0 0x7f43cc198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59924/0 (socket says 172.21.15.93:59924) 2023-11-16T16:55:40.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.222+0000 7f43cb7fe700 1 -- 172.21.15.93:0/1467607094 learned_addr learned my addr 172.21.15.93:0/1467607094 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:40.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.223+0000 7f43d0b9c700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43cc104060 0x7f43cc19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:40.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.223+0000 7f43cb7fe700 1 -- 172.21.15.93:0/1467607094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43cc1024e0 msgr2=0x7f43cc1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:40.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.223+0000 7f43cb7fe700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43cc1024e0 0x7f43cc1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.223+0000 7f43cb7fe700 1 -- 172.21.15.93:0/1467607094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43cc104060 msgr2=0x7f43cc19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:40.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.223+0000 7f43cb7fe700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43cc104060 0x7f43cc19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.223+0000 7f43cb7fe700 1 -- 172.21.15.93:0/1467607094 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc0095f0 con 0x7f43cc1036d0 2023-11-16T16:55:40.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.223+0000 7f43cb7fe700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43cc1036d0 0x7f43cc198810 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f43c000d950 tx=0x7f43c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:40.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.223+0000 7f43c97fa700 1 -- 172.21.15.93:0/1467607094 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c000f890 con 0x7f43cc1036d0 2023-11-16T16:55:40.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.223+0000 7f43d25ff700 1 -- 172.21.15.93:0/1467607094 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc10de90 con 0x7f43cc1036d0 2023-11-16T16:55:40.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.223+0000 7f43c97fa700 1 -- 172.21.15.93:0/1467607094 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43c0010460 con 0x7f43cc1036d0 2023-11-16T16:55:40.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.223+0000 7f43c97fa700 1 -- 172.21.15.93:0/1467607094 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c000e610 con 0x7f43cc1036d0 2023-11-16T16:55:40.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.223+0000 7f43d25ff700 1 -- 172.21.15.93:0/1467607094 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc10dd00 con 0x7f43cc1036d0 2023-11-16T16:55:40.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.224+0000 7f43d25ff700 1 -- 172.21.15.93:0/1467607094 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc109aa0 con 0x7f43cc1036d0 2023-11-16T16:55:40.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.226+0000 7f43c97fa700 1 -- 172.21.15.93:0/1467607094 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f43c00108d0 con 0x7f43cc1036d0 2023-11-16T16:55:40.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.228+0000 7f43c97fa700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f43b40a1150 0x7f43b40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:40.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.228+0000 7f43c97fa700 1 -- 172.21.15.93:0/1467607094 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f43c00ccb00 con 0x7f43cc1036d0 2023-11-16T16:55:40.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.228+0000 7f43cbfff700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f43b40a1150 0x7f43b40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:40.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.228+0000 7f43cbfff700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f43b40a1150 0x7f43b40a3600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00b5c0 tx=0x7f43bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:40.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.230+0000 7f43c97fa700 1 -- 172.21.15.93:0/1467607094 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f43c0095e20 con 0x7f43cc1036d0 2023-11-16T16:55:40.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.430+0000 7f43d25ff700 1 -- 172.21.15.93:0/1467607094 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f43cc061700 con 0x7f43cc1036d0 2023-11-16T16:55:40.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.431+0000 7f43c97fa700 1 -- 172.21.15.93:0/1467607094 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f43c0020020 con 0x7f43cc1036d0 2023-11-16T16:55:40.432 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-11-16T16:55:40.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.433+0000 7f43d25ff700 1 -- 172.21.15.93:0/1467607094 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f43b40a1150 msgr2=0x7f43b40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:40.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.433+0000 7f43d25ff700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f43b40a1150 0x7f43b40a3600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00b5c0 tx=0x7f43bc005fb0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.433+0000 7f43d25ff700 1 -- 172.21.15.93:0/1467607094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43cc1036d0 msgr2=0x7f43cc198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:40.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.433+0000 7f43d25ff700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43cc1036d0 0x7f43cc198810 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f43c000d950 tx=0x7f43c000dc60 comp rx=0 tx=0).stop 2023-11-16T16:55:40.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.433+0000 7f43d25ff700 1 -- 172.21.15.93:0/1467607094 shutdown_connections 2023-11-16T16:55:40.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.433+0000 7f43d25ff700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f43b40a1150 0x7f43b40a3600 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.433+0000 7f43d25ff700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43cc104060 0x7f43cc19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.433+0000 7f43d25ff700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43cc1024e0 0x7f43cc1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.433+0000 7f43d25ff700 1 --2- 172.21.15.93:0/1467607094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43cc1036d0 0x7f43cc198810 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.433+0000 7f43d25ff700 1 -- 172.21.15.93:0/1467607094 >> 172.21.15.93:0/1467607094 conn(0x7f43cc0fdb20 msgr2=0x7f43cc106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:40.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.433+0000 7f43d25ff700 1 -- 172.21.15.93:0/1467607094 shutdown_connections 2023-11-16T16:55:40.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.433+0000 7f43d25ff700 1 -- 172.21.15.93:0/1467607094 wait complete. 2023-11-16T16:55:40.453 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-16T16:55:40.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.652+0000 7fa787dff700 1 -- 172.21.15.93:0/3699810572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780101430 msgr2=0x7fa780101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:40.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.652+0000 7fa787dff700 1 --2- 172.21.15.93:0/3699810572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780101430 0x7fa780101880 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa770009990 tx=0x7fa770009ca0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.652+0000 7fa787dff700 1 -- 172.21.15.93:0/3699810572 shutdown_connections 2023-11-16T16:55:40.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.652+0000 7fa787dff700 1 --2- 172.21.15.93:0/3699810572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa780101dc0 0x7fa780108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.652+0000 7fa787dff700 1 --2- 172.21.15.93:0/3699810572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa780100240 0x7fa780100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.652+0000 7fa787dff700 1 --2- 172.21.15.93:0/3699810572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780101430 0x7fa780101880 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.652+0000 7fa787dff700 1 -- 172.21.15.93:0/3699810572 >> 172.21.15.93:0/3699810572 conn(0x7fa7800fb820 msgr2=0x7fa7800fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:40.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.652+0000 7fa787dff700 1 -- 172.21.15.93:0/3699810572 shutdown_connections 2023-11-16T16:55:40.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.652+0000 7fa787dff700 1 -- 172.21.15.93:0/3699810572 wait complete. 2023-11-16T16:55:40.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.654+0000 7fa787dff700 1 Processor -- start 2023-11-16T16:55:40.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.654+0000 7fa787dff700 1 -- start start 2023-11-16T16:55:40.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.654+0000 7fa787dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa780100240 0x7fa780195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:40.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.654+0000 7fa787dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780101430 0x7fa7801964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:40.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.654+0000 7fa787dff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa780101dc0 0x7fa78019d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:40.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.654+0000 7fa787dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78019db30 con 0x7fa780101430 2023-11-16T16:55:40.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.654+0000 7fa787dff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78019dc70 con 0x7fa780101dc0 2023-11-16T16:55:40.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.654+0000 7fa787dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78019de00 con 0x7fa780100240 2023-11-16T16:55:40.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.655+0000 7fa78539a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780101430 0x7fa7801964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:40.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.655+0000 7fa78539a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780101430 0x7fa7801964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59950/0 (socket says 172.21.15.93:59950) 2023-11-16T16:55:40.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.655+0000 7fa78539a700 1 -- 172.21.15.93:0/280546643 learned_addr learned my addr 172.21.15.93:0/280546643 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:40.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.655+0000 7fa785b9b700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa780100240 0x7fa780195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:40.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.655+0000 7fa78639c700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa780101dc0 0x7fa78019d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:40.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.655+0000 7fa78539a700 1 -- 172.21.15.93:0/280546643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa780100240 msgr2=0x7fa780195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:40.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.655+0000 7fa78539a700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa780100240 0x7fa780195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.655+0000 7fa78539a700 1 -- 172.21.15.93:0/280546643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa780101dc0 msgr2=0x7fa78019d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:40.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.655+0000 7fa78539a700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa780101dc0 0x7fa78019d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.655+0000 7fa78539a700 1 -- 172.21.15.93:0/280546643 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700095f0 con 0x7fa780101430 2023-11-16T16:55:40.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.655+0000 7fa78539a700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780101430 0x7fa7801964f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa77000b620 tx=0x7fa770005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:40.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.656+0000 7fa776ffd700 1 -- 172.21.15.93:0/280546643 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77001d070 con 0x7fa780101430 2023-11-16T16:55:40.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.656+0000 7fa776ffd700 1 -- 172.21.15.93:0/280546643 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa770022470 con 0x7fa780101430 2023-11-16T16:55:40.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.656+0000 7fa787dff700 1 -- 172.21.15.93:0/280546643 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78010bc00 con 0x7fa780101430 2023-11-16T16:55:40.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.656+0000 7fa776ffd700 1 -- 172.21.15.93:0/280546643 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa770017650 con 0x7fa780101430 2023-11-16T16:55:40.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.656+0000 7fa787dff700 1 -- 172.21.15.93:0/280546643 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78010ba70 con 0x7fa780101430 2023-11-16T16:55:40.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.657+0000 7fa787dff700 1 -- 172.21.15.93:0/280546643 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78019e3a0 con 0x7fa780101430 2023-11-16T16:55:40.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.659+0000 7fa776ffd700 1 -- 172.21.15.93:0/280546643 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa7700225e0 con 0x7fa780101430 2023-11-16T16:55:40.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.660+0000 7fa776ffd700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa76c0a1150 0x7fa76c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:40.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.660+0000 7fa776ffd700 1 -- 172.21.15.93:0/280546643 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fa7700ce990 con 0x7fa780101430 2023-11-16T16:55:40.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.660+0000 7fa785b9b700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa76c0a1150 0x7fa76c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:40.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.661+0000 7fa785b9b700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa76c0a1150 0x7fa76c0a3600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa780101290 tx=0x7fa77c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:40.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.662+0000 7fa776ffd700 1 -- 172.21.15.93:0/280546643 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa770097d40 con 0x7fa780101430 2023-11-16T16:55:40.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.879+0000 7fa787dff700 1 -- 172.21.15.93:0/280546643 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa780107170 con 0x7fa780101430 2023-11-16T16:55:40.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.879+0000 7fa776ffd700 1 -- 172.21.15.93:0/280546643 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) v1 ==== 81+0+588 (secure 0 0 0) 0x7fa77009b360 con 0x7fa780101430 2023-11-16T16:55:40.881 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/1465890024 2023-11-17T16:39:50.432397+0000 2023-11-16T16:55:40.881 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/3409666551 2023-11-17T16:39:50.432397+0000 2023-11-16T16:55:40.881 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/2198359920 2023-11-17T16:39:50.432397+0000 2023-11-16T16:55:40.881 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/772471105 2023-11-17T16:39:24.569132+0000 2023-11-16T16:55:40.882 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/772471105 2023-11-17T16:39:24.569132+0000 2023-11-16T16:55:40.882 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/2198359920 2023-11-17T16:39:50.432397+0000 2023-11-16T16:55:40.882 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/438537209 2023-11-17T16:39:24.569132+0000 2023-11-16T16:55:40.882 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2518663109 2023-11-17T16:39:24.569132+0000 2023-11-16T16:55:40.882 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/696516505 2023-11-17T16:39:24.569132+0000 2023-11-16T16:55:40.882 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/3700969994 2023-11-17T16:39:50.432397+0000 2023-11-16T16:55:40.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.882+0000 7fa787dff700 1 -- 172.21.15.93:0/280546643 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa76c0a1150 msgr2=0x7fa76c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:40.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.882+0000 7fa787dff700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa76c0a1150 0x7fa76c0a3600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa780101290 tx=0x7fa77c009260 comp rx=0 tx=0).stop 2023-11-16T16:55:40.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.882+0000 7fa787dff700 1 -- 172.21.15.93:0/280546643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780101430 msgr2=0x7fa7801964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:40.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.882+0000 7fa787dff700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780101430 0x7fa7801964f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa77000b620 tx=0x7fa770005740 comp rx=0 tx=0).stop 2023-11-16T16:55:40.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.882+0000 7fa787dff700 1 -- 172.21.15.93:0/280546643 shutdown_connections 2023-11-16T16:55:40.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.882+0000 7fa787dff700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa76c0a1150 0x7fa76c0a3600 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.882+0000 7fa787dff700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa780101dc0 0x7fa78019d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.882+0000 7fa787dff700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa780100240 0x7fa780195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.882+0000 7fa787dff700 1 --2- 172.21.15.93:0/280546643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780101430 0x7fa7801964f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:40.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.882+0000 7fa787dff700 1 -- 172.21.15.93:0/280546643 >> 172.21.15.93:0/280546643 conn(0x7fa7800fb820 msgr2=0x7fa7800fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:40.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.882+0000 7fa787dff700 1 -- 172.21.15.93:0/280546643 shutdown_connections 2023-11-16T16:55:40.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:40.882+0000 7fa787dff700 1 -- 172.21.15.93:0/280546643 wait complete. 2023-11-16T16:55:40.885 INFO:teuthology.orchestra.run.smithi093.stderr:listed 10 entries 2023-11-16T16:55:40.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:40 smithi093 ceph-mon[97863]: pgmap v558: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:55:40.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:40 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2992326322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:55:40.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:40 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1467607094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:55:40.903 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-16T16:55:40.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:55:40.903 DEBUG:teuthology.orchestra.run.smithi093:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-16T16:55:40.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:40 smithi159 ceph-mon[131528]: pgmap v558: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:55:40.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:40 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2992326322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:55:40.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:40 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1467607094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:55:41.031 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-16T16:55:41.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:55:41.032 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-16T16:55:41.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:40 smithi125 ceph-mon[131076]: pgmap v558: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:55:41.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:40 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2992326322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:55:41.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:40 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1467607094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-16T16:55:41.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:55:41.123 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-11-16T16:55:41.262 INFO:teuthology.orchestra.run.smithi093.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-16T16:55:41.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:55:41.262 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-16T16:55:41.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:55:41.333 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link delete ceph-brx 2023-11-16T16:55:41.434 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-11-16T16:55:41.443 INFO:tasks.cephadm:Teardown begin 2023-11-16T16:55:41.444 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-16T16:55:41.444 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:55:41.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.654+0000 7f15d35ff700 1 -- 172.21.15.93:0/328083176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc1014a0 msgr2=0x7f15cc1018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:41.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.654+0000 7f15d35ff700 1 --2- 172.21.15.93:0/328083176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc1014a0 0x7f15cc1018f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009990 tx=0x7f15bc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:55:41.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.654+0000 7f15d35ff700 1 -- 172.21.15.93:0/328083176 shutdown_connections 2023-11-16T16:55:41.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.654+0000 7f15d35ff700 1 --2- 172.21.15.93:0/328083176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15cc101e30 0x7f15cc1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:41.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.654+0000 7f15d35ff700 1 --2- 172.21.15.93:0/328083176 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15cc1002b0 0x7f15cc1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:41.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.654+0000 7f15d35ff700 1 --2- 172.21.15.93:0/328083176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc1014a0 0x7f15cc1018f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:41.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.654+0000 7f15d35ff700 1 -- 172.21.15.93:0/328083176 >> 172.21.15.93:0/328083176 conn(0x7f15cc0fb890 msgr2=0x7f15cc0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:41.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.655+0000 7f15d35ff700 1 -- 172.21.15.93:0/328083176 shutdown_connections 2023-11-16T16:55:41.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.655+0000 7f15d35ff700 1 -- 172.21.15.93:0/328083176 wait complete. 2023-11-16T16:55:41.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.655+0000 7f15d35ff700 1 Processor -- start 2023-11-16T16:55:41.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.656+0000 7f15d35ff700 1 -- start start 2023-11-16T16:55:41.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.656+0000 7f15d35ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15cc1002b0 0x7f15cc0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:41.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.656+0000 7f15d35ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15cc1014a0 0x7f15cc072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:41.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.656+0000 7f15d35ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc101e30 0x7f15cc073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:41.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.656+0000 7f15d35ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc073bc0 con 0x7f15cc1014a0 2023-11-16T16:55:41.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.656+0000 7f15d35ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc073d30 con 0x7f15cc1002b0 2023-11-16T16:55:41.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.656+0000 7f15d35ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc19ffe0 con 0x7f15cc101e30 2023-11-16T16:55:41.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.656+0000 7f15d139b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15cc1002b0 0x7f15cc0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:41.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.656+0000 7f15d0b9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15cc1014a0 0x7f15cc072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:41.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.656+0000 7f15d139b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15cc1002b0 0x7f15cc0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:38080/0 (socket says 172.21.15.93:38080) 2023-11-16T16:55:41.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.656+0000 7f15d139b700 1 -- 172.21.15.93:0/1233020614 learned_addr learned my addr 172.21.15.93:0/1233020614 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:41.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.656+0000 7f15d1b9c700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc101e30 0x7f15cc073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:41.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.657+0000 7f15d139b700 1 -- 172.21.15.93:0/1233020614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc101e30 msgr2=0x7f15cc073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:41.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.657+0000 7f15d139b700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc101e30 0x7f15cc073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:41.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.657+0000 7f15d139b700 1 -- 172.21.15.93:0/1233020614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15cc1014a0 msgr2=0x7f15cc072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:41.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.657+0000 7f15d139b700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15cc1014a0 0x7f15cc072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:41.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.657+0000 7f15d139b700 1 -- 172.21.15.93:0/1233020614 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc0095f0 con 0x7f15cc1002b0 2023-11-16T16:55:41.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.657+0000 7f15d139b700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15cc1002b0 0x7f15cc0748a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f15c800e970 tx=0x7f15c800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:41.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.657+0000 7f15c27fc700 1 -- 172.21.15.93:0/1233020614 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8009790 con 0x7f15cc1002b0 2023-11-16T16:55:41.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.657+0000 7f15d35ff700 1 -- 172.21.15.93:0/1233020614 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc10bc10 con 0x7f15cc1002b0 2023-11-16T16:55:41.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.657+0000 7f15c27fc700 1 -- 172.21.15.93:0/1233020614 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15c800c7e0 con 0x7f15cc1002b0 2023-11-16T16:55:41.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.657+0000 7f15c27fc700 1 -- 172.21.15.93:0/1233020614 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8003c70 con 0x7f15cc1002b0 2023-11-16T16:55:41.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.657+0000 7f15d35ff700 1 -- 172.21.15.93:0/1233020614 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc10ba80 con 0x7f15cc1002b0 2023-11-16T16:55:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.658+0000 7f15d35ff700 1 -- 172.21.15.93:0/1233020614 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15cc1078a0 con 0x7f15cc1002b0 2023-11-16T16:55:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.660+0000 7f15c27fc700 1 -- 172.21.15.93:0/1233020614 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f15c800cc50 con 0x7f15cc1002b0 2023-11-16T16:55:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.661+0000 7f15c27fc700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15b80a1150 0x7f15b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.662+0000 7f15c27fc700 1 -- 172.21.15.93:0/1233020614 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f15c8014070 con 0x7f15cc1002b0 2023-11-16T16:55:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.662+0000 7f15d0b9a700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15b80a1150 0x7f15b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.662+0000 7f15d0b9a700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15b80a1150 0x7f15b80a3600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00b2e0 tx=0x7f15bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:41.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.665+0000 7f15c27fc700 1 -- 172.21.15.93:0/1233020614 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f15c8096210 con 0x7f15cc1002b0 2023-11-16T16:55:41.900 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:41 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/280546643' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-16T16:55:41.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.908+0000 7f15d35ff700 1 -- 172.21.15.93:0/1233020614 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15cc02cfe0 con 0x7f15cc1002b0 2023-11-16T16:55:41.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.909+0000 7f15c27fc700 1 -- 172.21.15.93:0/1233020614 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10820 (secure 0 0 0) 0x7f15c8003dd0 con 0x7f15cc1002b0 2023-11-16T16:55:41.911 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:55:41.911 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3614592201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3614592201},{"type":"v1","addr":"172.21.15.93:6839","nonce":3614592201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4068789362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4068789362},{"type":"v1","addr":"172.21.15.93:6841","nonce":4068789362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:46:37.124338+0000","modified":"2023-11-16T16:53:48.627343+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24541,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6839/149300707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":149300707},{"type":"v1","addr":"172.21.15.159:6839","nonce":149300707}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/3590027328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":3590027328},{"type":"v1","addr":"172.21.15.125:6839","nonce":3590027328}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25717":{"gid":25717,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/157086341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":157086341},{"type":"v1","addr":"172.21.15.159:6837","nonce":157086341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:55:41.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.911+0000 7f15d35ff700 1 -- 172.21.15.93:0/1233020614 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15b80a1150 msgr2=0x7f15b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:41.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.911+0000 7f15d35ff700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15b80a1150 0x7f15b80a3600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00b2e0 tx=0x7f15bc005fb0 comp rx=0 tx=0).stop 2023-11-16T16:55:41.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.911+0000 7f15d35ff700 1 -- 172.21.15.93:0/1233020614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15cc1002b0 msgr2=0x7f15cc0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:41.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.911+0000 7f15d35ff700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15cc1002b0 0x7f15cc0748a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f15c800e970 tx=0x7f15c800ec80 comp rx=0 tx=0).stop 2023-11-16T16:55:41.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.911+0000 7f15d35ff700 1 -- 172.21.15.93:0/1233020614 shutdown_connections 2023-11-16T16:55:41.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.911+0000 7f15d35ff700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f15b80a1150 0x7f15b80a3600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:41.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.911+0000 7f15d35ff700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc101e30 0x7f15cc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:41.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.911+0000 7f15d35ff700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15cc1002b0 0x7f15cc0748a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:41.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.911+0000 7f15d35ff700 1 --2- 172.21.15.93:0/1233020614 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15cc1014a0 0x7f15cc072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:41.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.912+0000 7f15d35ff700 1 -- 172.21.15.93:0/1233020614 >> 172.21.15.93:0/1233020614 conn(0x7f15cc0fb890 msgr2=0x7f15cc1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:41.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.912+0000 7f15d35ff700 1 -- 172.21.15.93:0/1233020614 shutdown_connections 2023-11-16T16:55:41.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:41.912+0000 7f15d35ff700 1 -- 172.21.15.93:0/1233020614 wait complete. 2023-11-16T16:55:41.915 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 31 2023-11-16T16:55:41.932 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:55:41.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:41 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/280546643' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-16T16:55:42.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:41 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/280546643' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-16T16:55:42.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.129+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2436558643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff164101e10 msgr2=0x7ff1641086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:42.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.129+0000 7ff16b5ff700 1 --2- 172.21.15.93:0/2436558643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff164101e10 0x7ff1641086d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009f90 tx=0x7ff15c00b330 comp rx=0 tx=0).stop 2023-11-16T16:55:42.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.129+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2436558643 shutdown_connections 2023-11-16T16:55:42.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.129+0000 7ff16b5ff700 1 --2- 172.21.15.93:0/2436558643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff164101e10 0x7ff1641086d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.129+0000 7ff16b5ff700 1 --2- 172.21.15.93:0/2436558643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff164100290 0x7ff1641006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.129+0000 7ff16b5ff700 1 --2- 172.21.15.93:0/2436558643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164101480 0x7ff1641018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.129+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2436558643 >> 172.21.15.93:0/2436558643 conn(0x7ff1640fb870 msgr2=0x7ff1640fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:42.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.129+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2436558643 shutdown_connections 2023-11-16T16:55:42.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.130+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2436558643 wait complete. 2023-11-16T16:55:42.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.130+0000 7ff16b5ff700 1 Processor -- start 2023-11-16T16:55:42.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.131+0000 7ff16b5ff700 1 -- start start 2023-11-16T16:55:42.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.131+0000 7ff16b5ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164100290 0x7ff164198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:42.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.131+0000 7ff16b5ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff164101480 0x7ff1641985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:42.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.131+0000 7ff16b5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff164101e10 0x7ff16419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:42.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.131+0000 7ff16b5ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16419fc00 con 0x7ff164100290 2023-11-16T16:55:42.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.131+0000 7ff16b5ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16419fd40 con 0x7ff164101480 2023-11-16T16:55:42.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.131+0000 7ff16b5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16419fed0 con 0x7ff164101e10 2023-11-16T16:55:42.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.131+0000 7ff16939b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164100290 0x7ff164198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:42.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.131+0000 7ff16939b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164100290 0x7ff164198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59984/0 (socket says 172.21.15.93:59984) 2023-11-16T16:55:42.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.131+0000 7ff16939b700 1 -- 172.21.15.93:0/2779881531 learned_addr learned my addr 172.21.15.93:0/2779881531 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:42.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.131+0000 7ff168b9a700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff164101480 0x7ff1641985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.131+0000 7ff169b9c700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff164101e10 0x7ff16419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.132+0000 7ff16939b700 1 -- 172.21.15.93:0/2779881531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff164101e10 msgr2=0x7ff16419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.132+0000 7ff16939b700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff164101e10 0x7ff16419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.132+0000 7ff16939b700 1 -- 172.21.15.93:0/2779881531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff164101480 msgr2=0x7ff1641985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.132+0000 7ff16939b700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff164101480 0x7ff1641985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.132+0000 7ff16939b700 1 -- 172.21.15.93:0/2779881531 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1540095f0 con 0x7ff164100290 2023-11-16T16:55:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.132+0000 7ff16939b700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164100290 0x7ff164198080 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff16000e970 tx=0x7ff16000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.132+0000 7ff15a7fc700 1 -- 172.21.15.93:0/2779881531 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160009790 con 0x7ff164100290 2023-11-16T16:55:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.132+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2779881531 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15c009c40 con 0x7ff164100290 2023-11-16T16:55:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.132+0000 7ff15a7fc700 1 -- 172.21.15.93:0/2779881531 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff16000c7e0 con 0x7ff164100290 2023-11-16T16:55:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.132+0000 7ff15a7fc700 1 -- 172.21.15.93:0/2779881531 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160003c70 con 0x7ff164100290 2023-11-16T16:55:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.132+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2779881531 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16410bb90 con 0x7ff164100290 2023-11-16T16:55:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.133+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2779881531 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff164067a60 con 0x7ff164100290 2023-11-16T16:55:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.135+0000 7ff15a7fc700 1 -- 172.21.15.93:0/2779881531 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff16000cc50 con 0x7ff164100290 2023-11-16T16:55:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.137+0000 7ff15a7fc700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff1500a1150 0x7ff1500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.137+0000 7ff15a7fc700 1 -- 172.21.15.93:0/2779881531 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7ff160014070 con 0x7ff164100290 2023-11-16T16:55:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.137+0000 7ff168b9a700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff1500a1150 0x7ff1500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:42.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.138+0000 7ff168b9a700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff1500a1150 0x7ff1500a3600 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff154005fd0 tx=0x7ff154009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:42.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.141+0000 7ff15a7fc700 1 -- 172.21.15.93:0/2779881531 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff1600961f0 con 0x7ff164100290 2023-11-16T16:55:42.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.381+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2779881531 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1641a0a40 con 0x7ff164100290 2023-11-16T16:55:42.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.381+0000 7ff15a7fc700 1 -- 172.21.15.93:0/2779881531 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10820 (secure 0 0 0) 0x7ff160003dd0 con 0x7ff164100290 2023-11-16T16:55:42.383 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:55:42.383 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/278429922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":278429922},{"type":"v1","addr":"172.21.15.93:6835","nonce":278429922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3647861869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3647861869},{"type":"v1","addr":"172.21.15.93:6837","nonce":3647861869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3614592201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3614592201},{"type":"v1","addr":"172.21.15.93:6839","nonce":3614592201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4068789362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4068789362},{"type":"v1","addr":"172.21.15.93:6841","nonce":4068789362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2711358292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2711358292},{"type":"v1","addr":"172.21.15.159:6833","nonce":2711358292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1299400539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1299400539},{"type":"v1","addr":"172.21.15.125:6833","nonce":1299400539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:46:37.124338+0000","modified":"2023-11-16T16:53:48.627343+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24541,"mds_2":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/2117404468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":2117404468},{"type":"v1","addr":"172.21.15.125:6835","nonce":2117404468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/3072074959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3072074959},{"type":"v1","addr":"172.21.15.159:6835","nonce":3072074959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.125:6837/1957598038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1957598038},{"type":"v1","addr":"172.21.15.125:6837","nonce":1957598038}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6839/149300707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":149300707},{"type":"v1","addr":"172.21.15.159:6839","nonce":149300707}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/3590027328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":3590027328},{"type":"v1","addr":"172.21.15.125:6839","nonce":3590027328}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25717":{"gid":25717,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6837/157086341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":157086341},{"type":"v1","addr":"172.21.15.159:6837","nonce":157086341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:55:42.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.384+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2779881531 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff1500a1150 msgr2=0x7ff1500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:42.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.384+0000 7ff16b5ff700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff1500a1150 0x7ff1500a3600 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff154005fd0 tx=0x7ff154009310 comp rx=0 tx=0).stop 2023-11-16T16:55:42.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.384+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2779881531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164100290 msgr2=0x7ff164198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:42.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.384+0000 7ff16b5ff700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164100290 0x7ff164198080 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff16000e970 tx=0x7ff16000ec80 comp rx=0 tx=0).stop 2023-11-16T16:55:42.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.384+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2779881531 shutdown_connections 2023-11-16T16:55:42.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.384+0000 7ff16b5ff700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7ff1500a1150 0x7ff1500a3600 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.384+0000 7ff16b5ff700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164100290 0x7ff164198080 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.384+0000 7ff16b5ff700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff164101e10 0x7ff16419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.384+0000 7ff16b5ff700 1 --2- 172.21.15.93:0/2779881531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff164101480 0x7ff1641985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.384+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2779881531 >> 172.21.15.93:0/2779881531 conn(0x7ff1640fb870 msgr2=0x7ff1640fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:42.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.384+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2779881531 shutdown_connections 2023-11-16T16:55:42.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.384+0000 7ff16b5ff700 1 -- 172.21.15.93:0/2779881531 wait complete. 2023-11-16T16:55:42.387 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 31 2023-11-16T16:55:42.404 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:55:42.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.600+0000 7f1fac7ff700 1 -- 172.21.15.93:0/1114547432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1fa4101e10 msgr2=0x7f1fa41086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:42.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.600+0000 7f1fac7ff700 1 --2- 172.21.15.93:0/1114547432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1fa4101e10 0x7f1fa41086d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00b0a0 tx=0x7f1f9c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.600+0000 7f1fac7ff700 1 -- 172.21.15.93:0/1114547432 shutdown_connections 2023-11-16T16:55:42.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.600+0000 7f1fac7ff700 1 --2- 172.21.15.93:0/1114547432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1fa4101e10 0x7f1fa41086d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.600+0000 7f1fac7ff700 1 --2- 172.21.15.93:0/1114547432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa4100290 0x7f1fa41006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.600+0000 7f1fac7ff700 1 --2- 172.21.15.93:0/1114547432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa4101480 0x7f1fa41018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.600+0000 7f1fac7ff700 1 -- 172.21.15.93:0/1114547432 >> 172.21.15.93:0/1114547432 conn(0x7f1fa40fb870 msgr2=0x7f1fa40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:42.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.601+0000 7f1fac7ff700 1 -- 172.21.15.93:0/1114547432 shutdown_connections 2023-11-16T16:55:42.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.601+0000 7f1fac7ff700 1 -- 172.21.15.93:0/1114547432 wait complete. 2023-11-16T16:55:42.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1fac7ff700 1 Processor -- start 2023-11-16T16:55:42.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1fac7ff700 1 -- start start 2023-11-16T16:55:42.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1fac7ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa4100290 0x7f1fa4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:42.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1fac7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1fa4101480 0x7f1fa41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:42.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1fac7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa4101e10 0x7f1fa419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:42.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1fac7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa419fe10 con 0x7f1fa4101480 2023-11-16T16:55:42.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1fac7ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa419ff50 con 0x7f1fa4100290 2023-11-16T16:55:42.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1fac7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa41a00e0 con 0x7f1fa4101e10 2023-11-16T16:55:42.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1fa9d9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1fa4101480 0x7f1fa41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:42.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1fa9d9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1fa4101480 0x7f1fa41987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59998/0 (socket says 172.21.15.93:59998) 2023-11-16T16:55:42.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1fa9d9a700 1 -- 172.21.15.93:0/827840868 learned_addr learned my addr 172.21.15.93:0/827840868 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:42.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1faad9c700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa4101e10 0x7f1fa419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:42.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.602+0000 7f1faa59b700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa4100290 0x7f1fa4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:42.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.603+0000 7f1fa9d9a700 1 -- 172.21.15.93:0/827840868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa4101e10 msgr2=0x7f1fa419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:42.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.603+0000 7f1fa9d9a700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa4101e10 0x7f1fa419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.603+0000 7f1fa9d9a700 1 -- 172.21.15.93:0/827840868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa4100290 msgr2=0x7f1fa4198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:42.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.603+0000 7f1fa9d9a700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa4100290 0x7f1fa4198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.603+0000 7f1fa9d9a700 1 -- 172.21.15.93:0/827840868 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c009d00 con 0x7f1fa4101480 2023-11-16T16:55:42.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.603+0000 7f1faad9c700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa4101e10 0x7f1fa419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:55:42.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.603+0000 7f1fa9d9a700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1fa4101480 0x7f1fa41987d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400e9a0 tx=0x7f1f9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:42.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.603+0000 7f1f9b7fe700 1 -- 172.21.15.93:0/827840868 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94009790 con 0x7f1fa4101480 2023-11-16T16:55:42.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.603+0000 7f1fac7ff700 1 -- 172.21.15.93:0/827840868 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa410bc70 con 0x7f1fa4101480 2023-11-16T16:55:42.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.603+0000 7f1f9b7fe700 1 -- 172.21.15.93:0/827840868 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f94010430 con 0x7f1fa4101480 2023-11-16T16:55:42.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.603+0000 7f1f9b7fe700 1 -- 172.21.15.93:0/827840868 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94018680 con 0x7f1fa4101480 2023-11-16T16:55:42.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.603+0000 7f1fac7ff700 1 -- 172.21.15.93:0/827840868 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa410bae0 con 0x7f1fa4101480 2023-11-16T16:55:42.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.604+0000 7f1fac7ff700 1 -- 172.21.15.93:0/827840868 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa41a07d0 con 0x7f1fa4101480 2023-11-16T16:55:42.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.606+0000 7f1f9b7fe700 1 -- 172.21.15.93:0/827840868 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1f940105a0 con 0x7f1fa4101480 2023-11-16T16:55:42.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.608+0000 7f1f9b7fe700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1f900a1150 0x7f1f900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:42.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.608+0000 7f1f9b7fe700 1 -- 172.21.15.93:0/827840868 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7f1f94014070 con 0x7f1fa4101480 2023-11-16T16:55:42.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.608+0000 7f1faa59b700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1f900a1150 0x7f1f900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:42.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.609+0000 7f1faa59b700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1f900a1150 0x7f1f900a3600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1fa41012e0 tx=0x7f1fa0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:42.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.612+0000 7f1f9b7fe700 1 -- 172.21.15.93:0/827840868 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1f94051730 con 0x7f1fa4101480 2023-11-16T16:55:42.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.811+0000 7f1fac7ff700 1 -- 172.21.15.93:0/827840868 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fa41a0e70 con 0x7f1fa4101480 2023-11-16T16:55:42.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.812+0000 7f1f9b7fe700 1 -- 172.21.15.93:0/827840868 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24795 (secure 0 0 0) 0x7f1f94096220 con 0x7f1fa4101480 2023-11-16T16:55:42.814 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-16T16:55:42.814 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":94,"fsid":"6a59c1a8-849e-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:38:54.903278+0000","modified":"2023-11-16T16:53:48.219838+0000","last_up_change":"2023-11-16T16:45:46.257230+0000","last_in_change":"2023-11-16T16:45:27.930006+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:42:13.029135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:46:35.301321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:46:36.552545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-16T16:46:39.628380+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"329eef05-603a-4f45-be62-d6317cc25e60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6803","nonce":1213182748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6805","nonce":1213182748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6809","nonce":1213182748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1213182748},{"type":"v1","addr":"172.21.15.93:6807","nonce":1213182748}]},"public_addr":"172.21.15.93:6803/1213182748","cluster_addr":"172.21.15.93:6805/1213182748","heartbeat_back_addr":"172.21.15.93:6809/1213182748","heartbeat_front_addr":"172.21.15.93:6807/1213182748","state":["exists","up"]},{"osd":1,"uuid":"b72319a2-0c12-4b08-b78f-85d3af107bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6801","nonce":2084419115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6803","nonce":2084419115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6807","nonce":2084419115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":2084419115},{"type":"v1","addr":"172.21.15.125:6805","nonce":2084419115}]},"public_addr":"172.21.15.125:6801/2084419115","cluster_addr":"172.21.15.125:6803/2084419115","heartbeat_back_addr":"172.21.15.125:6807/2084419115","heartbeat_front_addr":"172.21.15.125:6805/2084419115","state":["exists","up"]},{"osd":2,"uuid":"0d17be2a-b82b-4a45-9015-c451f7884bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6801","nonce":189817992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6803","nonce":189817992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6807","nonce":189817992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":189817992},{"type":"v1","addr":"172.21.15.159:6805","nonce":189817992}]},"public_addr":"172.21.15.159:6801/189817992","cluster_addr":"172.21.15.159:6803/189817992","heartbeat_back_addr":"172.21.15.159:6807/189817992","heartbeat_front_addr":"172.21.15.159:6805/189817992","state":["exists","up"]},{"osd":3,"uuid":"71e6b247-710a-4bbb-9f6e-6333b3ea25bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6811","nonce":2350765600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6813","nonce":2350765600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6817","nonce":2350765600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2350765600},{"type":"v1","addr":"172.21.15.93:6815","nonce":2350765600}]},"public_addr":"172.21.15.93:6811/2350765600","cluster_addr":"172.21.15.93:6813/2350765600","heartbeat_back_addr":"172.21.15.93:6817/2350765600","heartbeat_front_addr":"172.21.15.93:6815/2350765600","state":["exists","up"]},{"osd":4,"uuid":"b965a7b2-6838-4b3a-b469-e2165cc42d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6809","nonce":3405048088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6811","nonce":3405048088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6815","nonce":3405048088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3405048088},{"type":"v1","addr":"172.21.15.125:6813","nonce":3405048088}]},"public_addr":"172.21.15.125:6809/3405048088","cluster_addr":"172.21.15.125:6811/3405048088","heartbeat_back_addr":"172.21.15.125:6815/3405048088","heartbeat_front_addr":"172.21.15.125:6813/3405048088","state":["exists","up"]},{"osd":5,"uuid":"c4277ae8-0ebf-4650-a0aa-3bd3f7f96968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6809","nonce":1380209909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6811","nonce":1380209909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6815","nonce":1380209909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1380209909},{"type":"v1","addr":"172.21.15.159:6813","nonce":1380209909}]},"public_addr":"172.21.15.159:6809/1380209909","cluster_addr":"172.21.15.159:6811/1380209909","heartbeat_back_addr":"172.21.15.159:6815/1380209909","heartbeat_front_addr":"172.21.15.159:6813/1380209909","state":["exists","up"]},{"osd":6,"uuid":"5ee6be21-c42d-441f-ac51-099370cf27bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6819","nonce":1437083195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6821","nonce":1437083195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6825","nonce":1437083195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":1437083195},{"type":"v1","addr":"172.21.15.93:6823","nonce":1437083195}]},"public_addr":"172.21.15.93:6819/1437083195","cluster_addr":"172.21.15.93:6821/1437083195","heartbeat_back_addr":"172.21.15.93:6825/1437083195","heartbeat_front_addr":"172.21.15.93:6823/1437083195","state":["exists","up"]},{"osd":7,"uuid":"ca36594b-cd4d-4d71-a448-b677bdc862aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6817","nonce":3103492922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6819","nonce":3103492922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6823","nonce":3103492922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":3103492922},{"type":"v1","addr":"172.21.15.125:6821","nonce":3103492922}]},"public_addr":"172.21.15.125:6817/3103492922","cluster_addr":"172.21.15.125:6819/3103492922","heartbeat_back_addr":"172.21.15.125:6823/3103492922","heartbeat_front_addr":"172.21.15.125:6821/3103492922","state":["exists","up"]},{"osd":8,"uuid":"e4ec0a32-5a4c-441b-97cb-548c992ec12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6817","nonce":1960504083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6819","nonce":1960504083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6823","nonce":1960504083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1960504083},{"type":"v1","addr":"172.21.15.159:6821","nonce":1960504083}]},"public_addr":"172.21.15.159:6817/1960504083","cluster_addr":"172.21.15.159:6819/1960504083","heartbeat_back_addr":"172.21.15.159:6823/1960504083","heartbeat_front_addr":"172.21.15.159:6821/1960504083","state":["exists","up"]},{"osd":9,"uuid":"0abd6b2e-f26c-40c3-ad8e-f65b1296d9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6827","nonce":594522260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6829","nonce":594522260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6833","nonce":594522260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":594522260},{"type":"v1","addr":"172.21.15.93:6831","nonce":594522260}]},"public_addr":"172.21.15.93:6827/594522260","cluster_addr":"172.21.15.93:6829/594522260","heartbeat_back_addr":"172.21.15.93:6833/594522260","heartbeat_front_addr":"172.21.15.93:6831/594522260","state":["exists","up"]},{"osd":10,"uuid":"2d7c0597-8595-4c28-8b1b-12e74fc554eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6825","nonce":1757421100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6827","nonce":1757421100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6831","nonce":1757421100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":1757421100},{"type":"v1","addr":"172.21.15.125:6829","nonce":1757421100}]},"public_addr":"172.21.15.125:6825/1757421100","cluster_addr":"172.21.15.125:6827/1757421100","heartbeat_back_addr":"172.21.15.125:6831/1757421100","heartbeat_front_addr":"172.21.15.125:6829/1757421100","state":["exists","up"]},{"osd":11,"uuid":"77e5e471-2cae-44cc-82e3-13888533f31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6825","nonce":2996534422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6827","nonce":2996534422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6831","nonce":2996534422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2996534422},{"type":"v1","addr":"172.21.15.159:6829","nonce":2996534422}]},"public_addr":"172.21.15.159:6825/2996534422","cluster_addr":"172.21.15.159:6827/2996534422","heartbeat_back_addr":"172.21.15.159:6831/2996534422","heartbeat_front_addr":"172.21.15.159:6829/2996534422","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:26.181217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:41:48.590846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:10.400008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:42:36.952711+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:00.391579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:22.139826+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:43:48.049290+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:11.206390+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:32.760873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:44:59.228520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:22.367928+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:45:44.289289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":1,"to":2}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1465890024":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/3409666551":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6800/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:6801/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6800/772471105":"2023-11-17T16:39:24.569132+0000","172.21.15.93:6801/2198359920":"2023-11-17T16:39:50.432397+0000","172.21.15.93:0/438537209":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/2518663109":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/696516505":"2023-11-17T16:39:24.569132+0000","172.21.15.93:0/3700969994":"2023-11-17T16:39:50.432397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:55:42.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.815+0000 7f1fac7ff700 1 -- 172.21.15.93:0/827840868 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1f900a1150 msgr2=0x7f1f900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:42.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.815+0000 7f1fac7ff700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1f900a1150 0x7f1f900a3600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1fa41012e0 tx=0x7f1fa0009260 comp rx=0 tx=0).stop 2023-11-16T16:55:42.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.815+0000 7f1fac7ff700 1 -- 172.21.15.93:0/827840868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1fa4101480 msgr2=0x7f1fa41987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:42.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.815+0000 7f1fac7ff700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1fa4101480 0x7f1fa41987d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400e9a0 tx=0x7f1f9400ecb0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.815+0000 7f1fac7ff700 1 -- 172.21.15.93:0/827840868 shutdown_connections 2023-11-16T16:55:42.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.815+0000 7f1fac7ff700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f1f900a1150 0x7f1f900a3600 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.815+0000 7f1fac7ff700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa4101e10 0x7f1fa419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.815+0000 7f1fac7ff700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa4100290 0x7f1fa4198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.815+0000 7f1fac7ff700 1 --2- 172.21.15.93:0/827840868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1fa4101480 0x7f1fa41987d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:42.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.815+0000 7f1fac7ff700 1 -- 172.21.15.93:0/827840868 >> 172.21.15.93:0/827840868 conn(0x7f1fa40fb870 msgr2=0x7f1fa40fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:42.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.815+0000 7f1fac7ff700 1 -- 172.21.15.93:0/827840868 shutdown_connections 2023-11-16T16:55:42.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:42.815+0000 7f1fac7ff700 1 -- 172.21.15.93:0/827840868 wait complete. 2023-11-16T16:55:42.835 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-16T16:55:42.984 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:42 smithi159 ceph-mon[131528]: pgmap v559: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:55:42.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:42 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/1233020614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:55:42.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:42 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2779881531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:55:43.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.032+0000 7fb2999ff700 1 -- 172.21.15.93:0/2889678719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb294101430 msgr2=0x7fb294101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:43.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.032+0000 7fb2999ff700 1 --2- 172.21.15.93:0/2889678719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb294101430 0x7fb294101880 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb288009990 tx=0x7fb288009ca0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.032+0000 7fb2999ff700 1 -- 172.21.15.93:0/2889678719 shutdown_connections 2023-11-16T16:55:43.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.032+0000 7fb2999ff700 1 --2- 172.21.15.93:0/2889678719 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb294101dc0 0x7fb294108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.032+0000 7fb2999ff700 1 --2- 172.21.15.93:0/2889678719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb294100240 0x7fb294100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.032+0000 7fb2999ff700 1 --2- 172.21.15.93:0/2889678719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb294101430 0x7fb294101880 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.032+0000 7fb2999ff700 1 -- 172.21.15.93:0/2889678719 >> 172.21.15.93:0/2889678719 conn(0x7fb2940fb820 msgr2=0x7fb2940fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:43.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.032+0000 7fb2999ff700 1 -- 172.21.15.93:0/2889678719 shutdown_connections 2023-11-16T16:55:43.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.032+0000 7fb2999ff700 1 -- 172.21.15.93:0/2889678719 wait complete. 2023-11-16T16:55:43.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.033+0000 7fb2999ff700 1 Processor -- start 2023-11-16T16:55:43.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.033+0000 7fb2999ff700 1 -- start start 2023-11-16T16:55:43.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2999ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb294100240 0x7fb2940748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:43.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2999ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb294101430 0x7fb294072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:43.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2999ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb294101dc0 0x7fb294073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:43.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2999ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb294073bc0 con 0x7fb294100240 2023-11-16T16:55:43.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2999ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb294073d30 con 0x7fb294101dc0 2023-11-16T16:55:43.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2999ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29419ffb0 con 0x7fb294101430 2023-11-16T16:55:43.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb292ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb294100240 0x7fb2940748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:43.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2927fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb294101430 0x7fb294072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:43.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2927fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb294101430 0x7fb294072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:46356/0 (socket says 172.21.15.93:46356) 2023-11-16T16:55:43.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2927fc700 1 -- 172.21.15.93:0/999188299 learned_addr learned my addr 172.21.15.93:0/999188299 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:43.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2937fe700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb294101dc0 0x7fb294073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:43.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2927fc700 1 -- 172.21.15.93:0/999188299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb294101dc0 msgr2=0x7fb294073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2927fc700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb294101dc0 0x7fb294073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2927fc700 1 -- 172.21.15.93:0/999188299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb294100240 msgr2=0x7fb2940748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2927fc700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb294100240 0x7fb2940748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb2927fc700 1 -- 172.21.15.93:0/999188299 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2880095f0 con 0x7fb294101430 2023-11-16T16:55:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.034+0000 7fb292ffd700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb294100240 0x7fb2940748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:55:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.035+0000 7fb2927fc700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb294101430 0x7fb294072ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb288004740 tx=0x7fb288009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.035+0000 7fb2989fd700 1 -- 172.21.15.93:0/999188299 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28801d070 con 0x7fb294101430 2023-11-16T16:55:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.035+0000 7fb2999ff700 1 -- 172.21.15.93:0/999188299 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29410bb90 con 0x7fb294101430 2023-11-16T16:55:43.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.035+0000 7fb2999ff700 1 -- 172.21.15.93:0/999188299 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29410ba00 con 0x7fb294101430 2023-11-16T16:55:43.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.035+0000 7fb2989fd700 1 -- 172.21.15.93:0/999188299 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb288004830 con 0x7fb294101430 2023-11-16T16:55:43.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.035+0000 7fb2989fd700 1 -- 172.21.15.93:0/999188299 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2880053c0 con 0x7fb294101430 2023-11-16T16:55:43.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.036+0000 7fb2999ff700 1 -- 172.21.15.93:0/999188299 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb294107770 con 0x7fb294101430 2023-11-16T16:55:43.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.038+0000 7fb2989fd700 1 -- 172.21.15.93:0/999188299 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb2880049a0 con 0x7fb294101430 2023-11-16T16:55:43.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.040+0000 7fb2989fd700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb27c0a1150 0x7fb27c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:43.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.040+0000 7fb2989fd700 1 -- 172.21.15.93:0/999188299 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9202+0+0 (secure 0 0 0) 0x7fb2880ce2c0 con 0x7fb294101430 2023-11-16T16:55:43.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.040+0000 7fb292ffd700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb27c0a1150 0x7fb27c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:43.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.040+0000 7fb292ffd700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb27c0a1150 0x7fb27c0a3600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb294101290 tx=0x7fb284009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:43.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.042+0000 7fb2989fd700 1 -- 172.21.15.93:0/999188299 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb288097670 con 0x7fb294101430 2023-11-16T16:55:43.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:42 smithi125 ceph-mon[131076]: pgmap v559: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:55:43.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:42 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/1233020614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:55:43.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:42 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2779881531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:55:43.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:42 smithi093 ceph-mon[97863]: pgmap v559: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:55:43.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:42 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/1233020614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:55:43.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:42 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2779881531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:55:43.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.287+0000 7fb2999ff700 1 -- 172.21.15.93:0/999188299 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb2940679f0 con 0x7fb294101430 2023-11-16T16:55:43.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.706+0000 7fb2989fd700 1 -- 172.21.15.93:0/999188299 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb28809ac90 con 0x7fb294101430 2023-11-16T16:55:43.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.709+0000 7fb2999ff700 1 -- 172.21.15.93:0/999188299 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb27c0a1150 msgr2=0x7fb27c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:43.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.709+0000 7fb2999ff700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb27c0a1150 0x7fb27c0a3600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb294101290 tx=0x7fb284009260 comp rx=0 tx=0).stop 2023-11-16T16:55:43.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.709+0000 7fb2999ff700 1 -- 172.21.15.93:0/999188299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb294101430 msgr2=0x7fb294072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:43.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.709+0000 7fb2999ff700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb294101430 0x7fb294072ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb288004740 tx=0x7fb288009580 comp rx=0 tx=0).stop 2023-11-16T16:55:43.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.709+0000 7fb2999ff700 1 -- 172.21.15.93:0/999188299 shutdown_connections 2023-11-16T16:55:43.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.709+0000 7fb2999ff700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fb27c0a1150 0x7fb27c0a3600 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.709+0000 7fb2999ff700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb294100240 0x7fb2940748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.709+0000 7fb2999ff700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb294101dc0 0x7fb294073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.709+0000 7fb2999ff700 1 --2- 172.21.15.93:0/999188299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb294101430 0x7fb294072ee0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.709+0000 7fb2999ff700 1 -- 172.21.15.93:0/999188299 >> 172.21.15.93:0/999188299 conn(0x7fb2940fb820 msgr2=0x7fb2941048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:43.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.709+0000 7fb2999ff700 1 -- 172.21.15.93:0/999188299 shutdown_connections 2023-11-16T16:55:43.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.709+0000 7fb2999ff700 1 -- 172.21.15.93:0/999188299 wait complete. 2023-11-16T16:55:43.712 INFO:teuthology.orchestra.run.smithi093.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-16T16:55:43.731 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-16T16:55:43.827 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/827840868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:55:43.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/999188299' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-16T16:55:43.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-16T16:55:43.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-mon[131076]: osdmap e95: 12 total, 12 up, 12 in 2023-11-16T16:55:43.828 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-b[156540]: ignoring --setuser ceph since I am not root 2023-11-16T16:55:43.828 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-b[156540]: ignoring --setgroup ceph since I am not root 2023-11-16T16:55:43.829 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-b[156540]: starting mds.b at 2023-11-16T16:55:43.829 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-e[155865]: ignoring --setuser ceph since I am not root 2023-11-16T16:55:43.829 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-e[155865]: ignoring --setgroup ceph since I am not root 2023-11-16T16:55:43.829 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-mds[155889]: -- 172.21.15.125:0/151944803 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5573fec05520 con 0x5573febdac00 2023-11-16T16:55:43.829 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-e[155865]: starting mds.e at 2023-11-16T16:55:43.830 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-h[156202]: 2023-11-16T16:55:43.514+0000 7f5d685b1700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-16T16:55:43.830 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-h[156202]: -1> 2023-11-16T16:55:43.514+0000 7f5d685b1700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-16T16:55:43.830 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-h[156202]: ignoring --setuser ceph since I am not root 2023-11-16T16:55:43.830 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-h[156202]: ignoring --setgroup ceph since I am not root 2023-11-16T16:55:43.830 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-mds[156226]: -- 172.21.15.125:0/3791982808 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5613a51bd520 con 0x5613a5192c00 2023-11-16T16:55:43.830 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:55:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-h[156202]: starting mds.h at 2023-11-16T16:55:43.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.932+0000 7fbddf9ff700 1 -- 172.21.15.93:0/1137932405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd8101e40 msgr2=0x7fbdd8108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:43.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.932+0000 7fbddf9ff700 1 --2- 172.21.15.93:0/1137932405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd8101e40 0x7fbdd8108700 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000b0a0 tx=0x7fbdd000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.932+0000 7fbddf9ff700 1 -- 172.21.15.93:0/1137932405 shutdown_connections 2023-11-16T16:55:43.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.932+0000 7fbddf9ff700 1 --2- 172.21.15.93:0/1137932405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd8101e40 0x7fbdd8108700 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.932+0000 7fbddf9ff700 1 --2- 172.21.15.93:0/1137932405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd81002c0 0x7fbdd81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.932+0000 7fbddf9ff700 1 --2- 172.21.15.93:0/1137932405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbdd81014b0 0x7fbdd8101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.932+0000 7fbddf9ff700 1 -- 172.21.15.93:0/1137932405 >> 172.21.15.93:0/1137932405 conn(0x7fbdd80fb8a0 msgr2=0x7fbdd80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:43.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.932+0000 7fbddf9ff700 1 -- 172.21.15.93:0/1137932405 shutdown_connections 2023-11-16T16:55:43.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.932+0000 7fbddf9ff700 1 -- 172.21.15.93:0/1137932405 wait complete. 2023-11-16T16:55:43.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.934+0000 7fbddf9ff700 1 Processor -- start 2023-11-16T16:55:43.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.934+0000 7fbddf9ff700 1 -- start start 2023-11-16T16:55:43.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.934+0000 7fbddf9ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbdd81002c0 0x7fbdd8196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:43.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.934+0000 7fbddf9ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd81014b0 0x7fbdd8196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:43.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.934+0000 7fbddf9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd8101e40 0x7fbdd819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:43.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddf9ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd819dcc0 con 0x7fbdd81014b0 2023-11-16T16:55:43.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddf9ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd819de00 con 0x7fbdd81002c0 2023-11-16T16:55:43.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddf9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd819df90 con 0x7fbdd8101e40 2023-11-16T16:55:43.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddcf9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd81014b0 0x7fbdd8196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:43.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddd79b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbdd81002c0 0x7fbdd8196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:43.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddd79b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbdd81002c0 0x7fbdd8196140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.93:38142/0 (socket says 172.21.15.93:38142) 2023-11-16T16:55:43.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbdddf9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd8101e40 0x7fbdd819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:43.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddd79b700 1 -- 172.21.15.93:0/4209370367 learned_addr learned my addr 172.21.15.93:0/4209370367 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:43.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddd79b700 1 -- 172.21.15.93:0/4209370367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd8101e40 msgr2=0x7fbdd819d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:43.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddd79b700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd8101e40 0x7fbdd819d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddd79b700 1 -- 172.21.15.93:0/4209370367 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd81014b0 msgr2=0x7fbdd8196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:43.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddd79b700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd81014b0 0x7fbdd8196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:43.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddd79b700 1 -- 172.21.15.93:0/4209370367 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd0009d00 con 0x7fbdd81002c0 2023-11-16T16:55:43.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbdddf9c700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd8101e40 0x7fbdd819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:55:43.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddcf9a700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd81014b0 0x7fbdd8196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:55:43.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.935+0000 7fbddd79b700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbdd81002c0 0x7fbdd8196140 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400e970 tx=0x7fbdd400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:43.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.936+0000 7fbdce7fc700 1 -- 172.21.15.93:0/4209370367 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4009790 con 0x7fbdd81002c0 2023-11-16T16:55:43.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.936+0000 7fbdce7fc700 1 -- 172.21.15.93:0/4209370367 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdd400c7e0 con 0x7fbdd81002c0 2023-11-16T16:55:43.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.936+0000 7fbdce7fc700 1 -- 172.21.15.93:0/4209370367 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4003c70 con 0x7fbdd81002c0 2023-11-16T16:55:43.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.936+0000 7fbddf9ff700 1 -- 172.21.15.93:0/4209370367 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd810bc90 con 0x7fbdd81002c0 2023-11-16T16:55:43.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.936+0000 7fbddf9ff700 1 -- 172.21.15.93:0/4209370367 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd810bb00 con 0x7fbdd81002c0 2023-11-16T16:55:43.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.937+0000 7fbddf9ff700 1 -- 172.21.15.93:0/4209370367 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd819e6e0 con 0x7fbdd81002c0 2023-11-16T16:55:43.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.938+0000 7fbdce7fc700 1 -- 172.21.15.93:0/4209370367 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbdd400cc50 con 0x7fbdd81002c0 2023-11-16T16:55:43.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.940+0000 7fbdce7fc700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbdc40a11d0 0x7fbdc40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:43.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.940+0000 7fbdce7fc700 1 -- 172.21.15.93:0/4209370367 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9460+0+0 (secure 0 0 0) 0x7fbdd4014070 con 0x7fbdd81002c0 2023-11-16T16:55:43.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.940+0000 7fbddcf9a700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbdc40a11d0 0x7fbdc40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:43.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.941+0000 7fbddcf9a700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbdc40a11d0 0x7fbdc40a3680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8011490 tx=0x7fbdc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:43.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:43.943+0000 7fbdce7fc700 1 -- 172.21.15.93:0/4209370367 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbdd4096170 con 0x7fbdd81002c0 2023-11-16T16:55:43.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/827840868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:55:43.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/999188299' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-16T16:55:43.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-16T16:55:43.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-mon[131528]: osdmap e95: 12 total, 12 up, 12 in 2023-11-16T16:55:43.986 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c[157316]: ignoring --setuser ceph since I am not root 2023-11-16T16:55:43.986 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c[157316]: ignoring --setgroup ceph since I am not root 2023-11-16T16:55:43.986 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-mds[157340]: -- 172.21.15.159:0/1408715436 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5571f2b41520 con 0x5571f2b17000 2023-11-16T16:55:43.986 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c[157316]: starting mds.c at 2023-11-16T16:55:43.986 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-f[156974]: ignoring --setuser ceph since I am not root 2023-11-16T16:55:43.987 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-f[156974]: ignoring --setgroup ceph since I am not root 2023-11-16T16:55:43.987 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-mds[156998]: -- 172.21.15.159:0/2769771153 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5603d80bd520 con 0x5603d8092c00 2023-11-16T16:55:43.987 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-f[156974]: starting mds.f at 2023-11-16T16:55:43.987 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-i[157654]: ignoring --setuser ceph since I am not root 2023-11-16T16:55:43.987 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-i[157654]: ignoring --setgroup ceph since I am not root 2023-11-16T16:55:43.987 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-mds[157678]: -- 172.21.15.159:0/515430107 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557dea9a1520 con 0x557dea976c00 2023-11-16T16:55:43.988 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:55:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-i[157654]: starting mds.i at 2023-11-16T16:55:44.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:43 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/827840868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:55:44.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:43 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/999188299' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-16T16:55:44.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:43 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-16T16:55:44.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:43 smithi093 ceph-mon[97863]: osdmap e95: 12 total, 12 up, 12 in 2023-11-16T16:55:44.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-a[97839]: 2023-11-16T16:55:43.695+0000 7f37348bd700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-16T16:55:44.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.189+0000 7fbddf9ff700 1 -- 172.21.15.93:0/4209370367 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbdd81070d0 con 0x7fbdd81002c0 2023-11-16T16:55:44.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.708+0000 7fbdce7fc700 1 -- 172.21.15.93:0/4209370367 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbdd4003dd0 con 0x7fbdd81002c0 2023-11-16T16:55:44.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.710+0000 7fbddf9ff700 1 -- 172.21.15.93:0/4209370367 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbdc40a11d0 msgr2=0x7fbdc40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:44.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.710+0000 7fbddf9ff700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbdc40a11d0 0x7fbdc40a3680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8011490 tx=0x7fbdc8009260 comp rx=0 tx=0).stop 2023-11-16T16:55:44.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.710+0000 7fbddf9ff700 1 -- 172.21.15.93:0/4209370367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbdd81002c0 msgr2=0x7fbdd8196140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:44.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.710+0000 7fbddf9ff700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbdd81002c0 0x7fbdd8196140 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400e970 tx=0x7fbdd400ec80 comp rx=0 tx=0).stop 2023-11-16T16:55:44.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.711+0000 7fbddf9ff700 1 -- 172.21.15.93:0/4209370367 shutdown_connections 2023-11-16T16:55:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.711+0000 7fbddf9ff700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fbdc40a11d0 0x7fbdc40a3680 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.711+0000 7fbddf9ff700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd8101e40 0x7fbdd819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.711+0000 7fbddf9ff700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbdd81002c0 0x7fbdd8196140 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.711+0000 7fbddf9ff700 1 --2- 172.21.15.93:0/4209370367 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd81014b0 0x7fbdd8196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.711+0000 7fbddf9ff700 1 -- 172.21.15.93:0/4209370367 >> 172.21.15.93:0/4209370367 conn(0x7fbdd80fb8a0 msgr2=0x7fbdd80fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.711+0000 7fbddf9ff700 1 -- 172.21.15.93:0/4209370367 shutdown_connections 2023-11-16T16:55:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.711+0000 7fbddf9ff700 1 -- 172.21.15.93:0/4209370367 wait complete. 2023-11-16T16:55:44.733 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-16T16:55:44.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.935+0000 7f6aafdff700 1 -- 172.21.15.93:0/1050367901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6aa8101e50 msgr2=0x7f6aa8108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:44.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.935+0000 7f6aafdff700 1 --2- 172.21.15.93:0/1050367901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6aa8101e50 0x7f6aa8108710 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000b0a0 tx=0x7f6aa000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:55:44.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.935+0000 7f6aafdff700 1 -- 172.21.15.93:0/1050367901 shutdown_connections 2023-11-16T16:55:44.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.935+0000 7f6aafdff700 1 --2- 172.21.15.93:0/1050367901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6aa8101e50 0x7f6aa8108710 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:44.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.935+0000 7f6aafdff700 1 --2- 172.21.15.93:0/1050367901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aa81002d0 0x7f6aa81006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:44.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.935+0000 7f6aafdff700 1 --2- 172.21.15.93:0/1050367901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa81014c0 0x7f6aa8101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:44.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.935+0000 7f6aafdff700 1 -- 172.21.15.93:0/1050367901 >> 172.21.15.93:0/1050367901 conn(0x7f6aa80fb8b0 msgr2=0x7f6aa80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:44.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.935+0000 7f6aafdff700 1 -- 172.21.15.93:0/1050367901 shutdown_connections 2023-11-16T16:55:44.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.935+0000 7f6aafdff700 1 -- 172.21.15.93:0/1050367901 wait complete. 2023-11-16T16:55:44.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.936+0000 7f6aafdff700 1 Processor -- start 2023-11-16T16:55:44.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.936+0000 7f6aafdff700 1 -- start start 2023-11-16T16:55:44.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aafdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aa81002d0 0x7f6aa8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:44.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aafdff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa81014c0 0x7f6aa81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:44.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aafdff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6aa8101e50 0x7f6aa819fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:44.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aafdff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa819ffe0 con 0x7f6aa8101e50 2023-11-16T16:55:44.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aafdff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa81a0120 con 0x7f6aa81014c0 2023-11-16T16:55:44.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aafdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa81a02b0 con 0x7f6aa81002d0 2023-11-16T16:55:44.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aadb9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aa81002d0 0x7f6aa8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:44.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aadb9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aa81002d0 0x7f6aa8198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:46384/0 (socket says 172.21.15.93:46384) 2023-11-16T16:55:44.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aad39a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa81014c0 0x7f6aa81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:44.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aadb9b700 1 -- 172.21.15.93:0/2033167135 learned_addr learned my addr 172.21.15.93:0/2033167135 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:44.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aae39c700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6aa8101e50 0x7f6aa819fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:44.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aadb9b700 1 -- 172.21.15.93:0/2033167135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa81014c0 msgr2=0x7f6aa81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:44.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aadb9b700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa81014c0 0x7f6aa81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:44.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aadb9b700 1 -- 172.21.15.93:0/2033167135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6aa8101e50 msgr2=0x7f6aa819fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:44.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aadb9b700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6aa8101e50 0x7f6aa819fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:44.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aadb9b700 1 -- 172.21.15.93:0/2033167135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa0009d00 con 0x7f6aa81002d0 2023-11-16T16:55:44.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.937+0000 7f6aadb9b700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aa81002d0 0x7f6aa8198460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400e970 tx=0x7f6aa400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:44.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.938+0000 7f6a9effd700 1 -- 172.21.15.93:0/2033167135 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4009790 con 0x7f6aa81002d0 2023-11-16T16:55:44.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.938+0000 7f6aafdff700 1 -- 172.21.15.93:0/2033167135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa810bca0 con 0x7f6aa81002d0 2023-11-16T16:55:44.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.938+0000 7f6a9effd700 1 -- 172.21.15.93:0/2033167135 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aa400c7e0 con 0x7f6aa81002d0 2023-11-16T16:55:44.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.938+0000 7f6a9effd700 1 -- 172.21.15.93:0/2033167135 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4003c70 con 0x7f6aa81002d0 2023-11-16T16:55:44.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.938+0000 7f6aafdff700 1 -- 172.21.15.93:0/2033167135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa810bb10 con 0x7f6aa81002d0 2023-11-16T16:55:44.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.939+0000 7f6aafdff700 1 -- 172.21.15.93:0/2033167135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa8192a00 con 0x7f6aa81002d0 2023-11-16T16:55:44.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.941+0000 7f6a9effd700 1 -- 172.21.15.93:0/2033167135 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6aa400cc50 con 0x7f6aa81002d0 2023-11-16T16:55:44.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.942+0000 7f6a9effd700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6a940a1150 0x7f6a940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:44.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.942+0000 7f6a9effd700 1 -- 172.21.15.93:0/2033167135 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9460+0+0 (secure 0 0 0) 0x7f6aa4014070 con 0x7f6aa81002d0 2023-11-16T16:55:44.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.942+0000 7f6aad39a700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6a940a1150 0x7f6a940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:44.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.943+0000 7f6aad39a700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6a940a1150 0x7f6a940a3600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009be0 tx=0x7f6a98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:44.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:44.945+0000 7f6a9effd700 1 -- 172.21.15.93:0/2033167135 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6aa4096350 con 0x7f6aa81002d0 2023-11-16T16:55:44.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:44 smithi159 ceph-mon[131528]: pgmap v560: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-16T16:55:44.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:44 smithi159 ceph-mon[131528]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-16T16:55:44.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:44 smithi159 ceph-mon[131528]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-16T16:55:44.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:44 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-16T16:55:44.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:44 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.125:6836/154963463,v1:172.21.15.125:6837/154963463] up:boot 2023-11-16T16:55:44.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:44 smithi159 ceph-mon[131528]: fsmap cephfs:0/3 7 up:standby, 3 failed 2023-11-16T16:55:44.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:44 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:55:44.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:44 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/4209370367' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:55:44.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:44 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:55:45.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:44 smithi125 ceph-mon[131076]: pgmap v560: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-16T16:55:45.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:44 smithi125 ceph-mon[131076]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-16T16:55:45.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:44 smithi125 ceph-mon[131076]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-16T16:55:45.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:44 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-16T16:55:45.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:44 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.125:6836/154963463,v1:172.21.15.125:6837/154963463] up:boot 2023-11-16T16:55:45.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:44 smithi125 ceph-mon[131076]: fsmap cephfs:0/3 7 up:standby, 3 failed 2023-11-16T16:55:45.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:44 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:55:45.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:44 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/4209370367' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:55:45.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:44 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:55:45.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.144+0000 7f6aafdff700 1 -- 172.21.15.93:0/2033167135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6aa802cfe0 con 0x7f6aa81002d0 2023-11-16T16:55:45.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:44 smithi093 ceph-mon[97863]: pgmap v560: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-16T16:55:45.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:44 smithi093 ceph-mon[97863]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-16T16:55:45.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:44 smithi093 ceph-mon[97863]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-16T16:55:45.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:44 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-16T16:55:45.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:44 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.125:6836/154963463,v1:172.21.15.125:6837/154963463] up:boot 2023-11-16T16:55:45.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:44 smithi093 ceph-mon[97863]: fsmap cephfs:0/3 7 up:standby, 3 failed 2023-11-16T16:55:45.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:44 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:55:45.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:44 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/4209370367' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:55:45.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:44 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:55:45.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.713+0000 7f6a9effd700 1 -- 172.21.15.93:0/2033167135 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6aa4003dd0 con 0x7f6aa81002d0 2023-11-16T16:55:45.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.716+0000 7f6aafdff700 1 -- 172.21.15.93:0/2033167135 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6a940a1150 msgr2=0x7f6a940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:45.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.716+0000 7f6aafdff700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6a940a1150 0x7f6a940a3600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009be0 tx=0x7f6a98009260 comp rx=0 tx=0).stop 2023-11-16T16:55:45.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.716+0000 7f6aafdff700 1 -- 172.21.15.93:0/2033167135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aa81002d0 msgr2=0x7f6aa8198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:45.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.716+0000 7f6aafdff700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aa81002d0 0x7f6aa8198460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400e970 tx=0x7f6aa400ec80 comp rx=0 tx=0).stop 2023-11-16T16:55:45.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.716+0000 7f6aafdff700 1 -- 172.21.15.93:0/2033167135 shutdown_connections 2023-11-16T16:55:45.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.716+0000 7f6aafdff700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f6a940a1150 0x7f6a940a3600 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:45.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.716+0000 7f6aafdff700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6aa8101e50 0x7f6aa819fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:45.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.716+0000 7f6aafdff700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aa81002d0 0x7f6aa8198460 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:45.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.716+0000 7f6aafdff700 1 --2- 172.21.15.93:0/2033167135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aa81014c0 0x7f6aa81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:45.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.716+0000 7f6aafdff700 1 -- 172.21.15.93:0/2033167135 >> 172.21.15.93:0/2033167135 conn(0x7f6aa80fb8b0 msgr2=0x7f6aa8106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:45.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.716+0000 7f6aafdff700 1 -- 172.21.15.93:0/2033167135 shutdown_connections 2023-11-16T16:55:45.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.716+0000 7f6aafdff700 1 -- 172.21.15.93:0/2033167135 wait complete. 2023-11-16T16:55:45.720 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_metadata' removed 2023-11-16T16:55:45.740 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-16T16:55:45.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.941+0000 7fa97d39d700 1 -- 172.21.15.93:0/333571161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa978074a90 msgr2=0x7fa978072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:45.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.941+0000 7fa97d39d700 1 --2- 172.21.15.93:0/333571161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa978074a90 0x7fa978072f80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa968009990 tx=0x7fa968009ca0 comp rx=0 tx=0).stop 2023-11-16T16:55:45.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.941+0000 7fa97d39d700 1 -- 172.21.15.93:0/333571161 shutdown_connections 2023-11-16T16:55:45.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.941+0000 7fa97d39d700 1 --2- 172.21.15.93:0/333571161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa978074a90 0x7fa978072f80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:45.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.941+0000 7fa97d39d700 1 --2- 172.21.15.93:0/333571161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa978107f00 0x7fa97810a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:45.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.941+0000 7fa97d39d700 1 --2- 172.21.15.93:0/333571161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9780734c0 0x7fa978073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:45.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.941+0000 7fa97d39d700 1 -- 172.21.15.93:0/333571161 >> 172.21.15.93:0/333571161 conn(0x7fa9780fb6a0 msgr2=0x7fa9780fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:45.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.941+0000 7fa97d39d700 1 -- 172.21.15.93:0/333571161 shutdown_connections 2023-11-16T16:55:45.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.941+0000 7fa97d39d700 1 -- 172.21.15.93:0/333571161 wait complete. 2023-11-16T16:55:45.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.942+0000 7fa97d39d700 1 Processor -- start 2023-11-16T16:55:45.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.942+0000 7fa97d39d700 1 -- start start 2023-11-16T16:55:45.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa97d39d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9780734c0 0x7fa978198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:45.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa97d39d700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa978074a90 0x7fa978198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:45.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa97d39d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa978107f00 0x7fa97819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:45.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa97d39d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97819fbc0 con 0x7fa978107f00 2023-11-16T16:55:45.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa97d39d700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97819fd00 con 0x7fa978074a90 2023-11-16T16:55:45.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa97d39d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97819fe90 con 0x7fa9780734c0 2023-11-16T16:55:45.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa977fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9780734c0 0x7fa978198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:45.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa97cb9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa978107f00 0x7fa97819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:45.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa977fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9780734c0 0x7fa978198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:46400/0 (socket says 172.21.15.93:46400) 2023-11-16T16:55:45.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa977fff700 1 -- 172.21.15.93:0/2680696268 learned_addr learned my addr 172.21.15.93:0/2680696268 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:45.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa9777fe700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa978074a90 0x7fa978198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:45.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa977fff700 1 -- 172.21.15.93:0/2680696268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa978074a90 msgr2=0x7fa978198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:45.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa977fff700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa978074a90 0x7fa978198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:45.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa977fff700 1 -- 172.21.15.93:0/2680696268 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa978107f00 msgr2=0x7fa97819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:45.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa977fff700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa978107f00 0x7fa97819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:45.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.943+0000 7fa977fff700 1 -- 172.21.15.93:0/2680696268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9680095f0 con 0x7fa9780734c0 2023-11-16T16:55:45.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.944+0000 7fa977fff700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9780734c0 0x7fa978198040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa968004710 tx=0x7fa968009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:45.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.944+0000 7fa9757fa700 1 -- 172.21.15.93:0/2680696268 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96801d070 con 0x7fa9780734c0 2023-11-16T16:55:45.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.944+0000 7fa97d39d700 1 -- 172.21.15.93:0/2680696268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97810dc60 con 0x7fa9780734c0 2023-11-16T16:55:45.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.944+0000 7fa9757fa700 1 -- 172.21.15.93:0/2680696268 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa968021410 con 0x7fa9780734c0 2023-11-16T16:55:45.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.944+0000 7fa9757fa700 1 -- 172.21.15.93:0/2680696268 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968017650 con 0x7fa9780734c0 2023-11-16T16:55:45.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.944+0000 7fa97d39d700 1 -- 172.21.15.93:0/2680696268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97810dad0 con 0x7fa9780734c0 2023-11-16T16:55:45.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.945+0000 7fa97d39d700 1 -- 172.21.15.93:0/2680696268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa978106690 con 0x7fa9780734c0 2023-11-16T16:55:45.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.947+0000 7fa9757fa700 1 -- 172.21.15.93:0/2680696268 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa968021580 con 0x7fa9780734c0 2023-11-16T16:55:45.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.948+0000 7fa9757fa700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa9600a1150 0x7fa9600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:45.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.948+0000 7fa9757fa700 1 -- 172.21.15.93:0/2680696268 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8935+0+0 (secure 0 0 0) 0x7fa9680ce290 con 0x7fa9780734c0 2023-11-16T16:55:45.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.949+0000 7fa9777fe700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa9600a1150 0x7fa9600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:45.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.949+0000 7fa9777fe700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa9600a1150 0x7fa9600a3600 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009c60 tx=0x7fa96c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:45.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:45.950+0000 7fa9757fa700 1 -- 172.21.15.93:0/2680696268 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa968097750 con 0x7fa9780734c0 2023-11-16T16:55:45.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-16T16:55:45.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-16T16:55:45.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: Cluster is now healthy 2023-11-16T16:55:45.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.159:6835/2485530067,v1:172.21.15.159:6837/2485530067] up:boot 2023-11-16T16:55:45.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.125:6838/1114160851,v1:172.21.15.125:6839/1114160851] up:boot 2023-11-16T16:55:45.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.159:6838/1120154525,v1:172.21.15.159:6839/1120154525] up:boot 2023-11-16T16:55:45.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.125:6834/56324103,v1:172.21.15.125:6835/56324103] up:boot 2023-11-16T16:55:45.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: mds.? [v2:172.21.15.159:6834/2883414146,v1:172.21.15.159:6836/2883414146] up:boot 2023-11-16T16:55:45.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-16T16:55:45.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: fsmap 12 up:standby 2023-11-16T16:55:45.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:55:45.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:55:45.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:55:45.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:55:45.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:55:45.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: pgmap v562: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 614 B/s wr, 6 op/s 2023-11-16T16:55:45.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2033167135' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:45.987 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:45 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:46.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-16T16:55:46.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-16T16:55:46.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: Cluster is now healthy 2023-11-16T16:55:46.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.159:6835/2485530067,v1:172.21.15.159:6837/2485530067] up:boot 2023-11-16T16:55:46.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.125:6838/1114160851,v1:172.21.15.125:6839/1114160851] up:boot 2023-11-16T16:55:46.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.159:6838/1120154525,v1:172.21.15.159:6839/1120154525] up:boot 2023-11-16T16:55:46.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.125:6834/56324103,v1:172.21.15.125:6835/56324103] up:boot 2023-11-16T16:55:46.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: mds.? [v2:172.21.15.159:6834/2883414146,v1:172.21.15.159:6836/2883414146] up:boot 2023-11-16T16:55:46.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-16T16:55:46.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: fsmap 12 up:standby 2023-11-16T16:55:46.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:55:46.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:55:46.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:55:46.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:55:46.080 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:55:46.080 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: pgmap v562: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 614 B/s wr, 6 op/s 2023-11-16T16:55:46.080 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2033167135' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:46.080 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:45 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:46.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-16T16:55:46.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-16T16:55:46.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: Cluster is now healthy 2023-11-16T16:55:46.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.159:6835/2485530067,v1:172.21.15.159:6837/2485530067] up:boot 2023-11-16T16:55:46.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.125:6838/1114160851,v1:172.21.15.125:6839/1114160851] up:boot 2023-11-16T16:55:46.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.159:6838/1120154525,v1:172.21.15.159:6839/1120154525] up:boot 2023-11-16T16:55:46.151 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.125:6834/56324103,v1:172.21.15.125:6835/56324103] up:boot 2023-11-16T16:55:46.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: mds.? [v2:172.21.15.159:6834/2883414146,v1:172.21.15.159:6836/2883414146] up:boot 2023-11-16T16:55:46.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-16T16:55:46.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: fsmap 12 up:standby 2023-11-16T16:55:46.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:55:46.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:55:46.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:55:46.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:55:46.152 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: from='mgr.14152 172.21.15.93:0/841033865' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:55:46.153 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: pgmap v562: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 614 B/s wr, 6 op/s 2023-11-16T16:55:46.153 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2033167135' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:46.153 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:45 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:46.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.158+0000 7fa97d39d700 1 -- 172.21.15.93:0/2680696268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa97802cfe0 con 0x7fa9780734c0 2023-11-16T16:55:46.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.716+0000 7fa9757fa700 1 -- 172.21.15.93:0/2680696268 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa96809ad70 con 0x7fa9780734c0 2023-11-16T16:55:46.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.721+0000 7fa97d39d700 1 -- 172.21.15.93:0/2680696268 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa9600a1150 msgr2=0x7fa9600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:46.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.721+0000 7fa97d39d700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa9600a1150 0x7fa9600a3600 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009c60 tx=0x7fa96c009260 comp rx=0 tx=0).stop 2023-11-16T16:55:46.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.721+0000 7fa97d39d700 1 -- 172.21.15.93:0/2680696268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9780734c0 msgr2=0x7fa978198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:46.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.721+0000 7fa97d39d700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9780734c0 0x7fa978198040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa968004710 tx=0x7fa968009580 comp rx=0 tx=0).stop 2023-11-16T16:55:46.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.723+0000 7fa97d39d700 1 -- 172.21.15.93:0/2680696268 shutdown_connections 2023-11-16T16:55:46.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.723+0000 7fa97d39d700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7fa9600a1150 0x7fa9600a3600 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:46.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.723+0000 7fa97d39d700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa978107f00 0x7fa97819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:46.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.723+0000 7fa97d39d700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9780734c0 0x7fa978198040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:46.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.723+0000 7fa97d39d700 1 --2- 172.21.15.93:0/2680696268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa978074a90 0x7fa978198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:46.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.723+0000 7fa97d39d700 1 -- 172.21.15.93:0/2680696268 >> 172.21.15.93:0/2680696268 conn(0x7fa9780fb6a0 msgr2=0x7fa9780fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:46.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.723+0000 7fa97d39d700 1 -- 172.21.15.93:0/2680696268 shutdown_connections 2023-11-16T16:55:46.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.723+0000 7fa97d39d700 1 -- 172.21.15.93:0/2680696268 wait complete. 2023-11-16T16:55:46.727 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_data' removed 2023-11-16T16:55:46.755 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-16T16:55:46.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.957+0000 7f786c3ff700 1 -- 172.21.15.93:0/678855512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7864101e40 msgr2=0x7f7864108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:46.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.957+0000 7f786c3ff700 1 --2- 172.21.15.93:0/678855512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7864101e40 0x7f7864108700 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f785c00b0a0 tx=0x7f785c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:55:46.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.957+0000 7f786c3ff700 1 -- 172.21.15.93:0/678855512 shutdown_connections 2023-11-16T16:55:46.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.957+0000 7f786c3ff700 1 --2- 172.21.15.93:0/678855512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7864101e40 0x7f7864108700 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:46.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.957+0000 7f786c3ff700 1 --2- 172.21.15.93:0/678855512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78641002c0 0x7f78641006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:46.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.957+0000 7f786c3ff700 1 --2- 172.21.15.93:0/678855512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78641014b0 0x7f7864101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:46.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.957+0000 7f786c3ff700 1 -- 172.21.15.93:0/678855512 >> 172.21.15.93:0/678855512 conn(0x7f78640fb8a0 msgr2=0x7f78640fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:46.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.957+0000 7f786c3ff700 1 -- 172.21.15.93:0/678855512 shutdown_connections 2023-11-16T16:55:46.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.957+0000 7f786c3ff700 1 -- 172.21.15.93:0/678855512 wait complete. 2023-11-16T16:55:46.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.958+0000 7f786c3ff700 1 Processor -- start 2023-11-16T16:55:46.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.958+0000 7f786c3ff700 1 -- start start 2023-11-16T16:55:46.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786c3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78641002c0 0x7f7864198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:46.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786c3ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78641014b0 0x7f7864198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:46.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786c3ff700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7864101e40 0x7f786419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:46.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786c3ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786419fec0 con 0x7f78641014b0 2023-11-16T16:55:46.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786c3ff700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78641a0000 con 0x7f7864101e40 2023-11-16T16:55:46.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786c3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78641a0190 con 0x7f78641002c0 2023-11-16T16:55:46.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786a19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78641002c0 0x7f7864198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:46.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786999a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78641014b0 0x7f7864198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:46.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786a19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78641002c0 0x7f7864198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.93:44398/0 (socket says 172.21.15.93:44398) 2023-11-16T16:55:46.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786a19b700 1 -- 172.21.15.93:0/2770596946 learned_addr learned my addr 172.21.15.93:0/2770596946 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-16T16:55:46.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786a99c700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7864101e40 0x7f786419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:46.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786999a700 1 -- 172.21.15.93:0/2770596946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78641002c0 msgr2=0x7f7864198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:46.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786999a700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78641002c0 0x7f7864198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:46.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786999a700 1 -- 172.21.15.93:0/2770596946 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7864101e40 msgr2=0x7f786419f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:46.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786999a700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7864101e40 0x7f786419f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:46.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.959+0000 7f786999a700 1 -- 172.21.15.93:0/2770596946 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c009d00 con 0x7f78641014b0 2023-11-16T16:55:46.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.960+0000 7f786999a700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78641014b0 0x7f7864198880 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f785400e9f0 tx=0x7f785400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:46.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.960+0000 7f785b7fe700 1 -- 172.21.15.93:0/2770596946 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785400c9c0 con 0x7f78641014b0 2023-11-16T16:55:46.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.960+0000 7f785b7fe700 1 -- 172.21.15.93:0/2770596946 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7854010430 con 0x7f78641014b0 2023-11-16T16:55:46.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.960+0000 7f786c3ff700 1 -- 172.21.15.93:0/2770596946 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786410bc90 con 0x7f78641014b0 2023-11-16T16:55:46.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.960+0000 7f785b7fe700 1 -- 172.21.15.93:0/2770596946 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854018680 con 0x7f78641014b0 2023-11-16T16:55:46.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.960+0000 7f786c3ff700 1 -- 172.21.15.93:0/2770596946 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786410bb00 con 0x7f78641014b0 2023-11-16T16:55:46.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.961+0000 7f786c3ff700 1 -- 172.21.15.93:0/2770596946 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78641a0880 con 0x7f78641014b0 2023-11-16T16:55:46.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.963+0000 7f785b7fe700 1 -- 172.21.15.93:0/2770596946 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7854010860 con 0x7f78641014b0 2023-11-16T16:55:46.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.964+0000 7f785b7fe700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f78500a1150 0x7f78500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:55:46.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.964+0000 7f785b7fe700 1 -- 172.21.15.93:0/2770596946 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8252+0+0 (secure 0 0 0) 0x7f7854014070 con 0x7f78641014b0 2023-11-16T16:55:46.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.964+0000 7f786a19b700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f78500a1150 0x7f78500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:55:46.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.965+0000 7f786a19b700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f78500a1150 0x7f78500a3600 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f786000bd70 tx=0x7f786000b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:55:46.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:46.968+0000 7f785b7fe700 1 -- 172.21.15.93:0/2770596946 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7854051730 con 0x7f78641014b0 2023-11-16T16:55:46.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:46 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-16T16:55:46.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:46 smithi159 ceph-mon[131528]: osdmap e96: 12 total, 12 up, 12 in 2023-11-16T16:55:46.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:46 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2680696268' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:46.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:46 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:47.077 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:46 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-16T16:55:47.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:46 smithi125 ceph-mon[131076]: osdmap e96: 12 total, 12 up, 12 in 2023-11-16T16:55:47.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:46 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2680696268' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:47.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:46 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:47.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:46 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-16T16:55:47.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:46 smithi093 ceph-mon[97863]: osdmap e96: 12 total, 12 up, 12 in 2023-11-16T16:55:47.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:46 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2680696268' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:47.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:46 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:47.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.168+0000 7f786c3ff700 1 -- 172.21.15.93:0/2770596946 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f78641a0f20 con 0x7f78641014b0 2023-11-16T16:55:47.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.728+0000 7f785b7fe700 1 -- 172.21.15.93:0/2770596946 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) v1 ==== 178+0+0 (secure 0 0 0) 0x7f7854099c10 con 0x7f78641014b0 2023-11-16T16:55:47.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.732+0000 7f786c3ff700 1 -- 172.21.15.93:0/2770596946 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f78500a1150 msgr2=0x7f78500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:47.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.732+0000 7f786c3ff700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f78500a1150 0x7f78500a3600 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f786000bd70 tx=0x7f786000b4a0 comp rx=0 tx=0).stop 2023-11-16T16:55:47.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.732+0000 7f786c3ff700 1 -- 172.21.15.93:0/2770596946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78641014b0 msgr2=0x7f7864198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:55:47.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.732+0000 7f786c3ff700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78641014b0 0x7f7864198880 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f785400e9f0 tx=0x7f785400ed00 comp rx=0 tx=0).stop 2023-11-16T16:55:47.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.732+0000 7f786c3ff700 1 -- 172.21.15.93:0/2770596946 shutdown_connections 2023-11-16T16:55:47.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.732+0000 7f786c3ff700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.93:6800/3283196047,v1:172.21.15.93:6801/3283196047] conn(0x7f78500a1150 0x7f78500a3600 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:47.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.732+0000 7f786c3ff700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7864101e40 0x7f786419f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:47.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.732+0000 7f786c3ff700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78641002c0 0x7f7864198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:47.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.732+0000 7f786c3ff700 1 --2- 172.21.15.93:0/2770596946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78641014b0 0x7f7864198880 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:55:47.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.732+0000 7f786c3ff700 1 -- 172.21.15.93:0/2770596946 >> 172.21.15.93:0/2770596946 conn(0x7f78640fb8a0 msgr2=0x7f7864106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:55:47.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.733+0000 7f786c3ff700 1 -- 172.21.15.93:0/2770596946 shutdown_connections 2023-11-16T16:55:47.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-16T16:55:47.734+0000 7f786c3ff700 1 -- 172.21.15.93:0/2770596946 wait complete. 2023-11-16T16:55:47.736 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_data_ec' removed 2023-11-16T16:55:47.762 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:55:47.832 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:55:47.875 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:55:47.911 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-11-16T16:55:47.911 DEBUG:teuthology.orchestra.run.smithi093:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-16T16:55:47.985 DEBUG:teuthology.orchestra.run.smithi125:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-16T16:55:47.985 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:47 smithi159 ceph-mon[131528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-11-16T16:55:47.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:47 smithi159 ceph-mon[131528]: osdmap e97: 12 total, 12 up, 12 in 2023-11-16T16:55:47.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:47 smithi159 ceph-mon[131528]: pgmap v565: 33 pgs: 33 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:55:47.986 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:47 smithi159 ceph-mon[131528]: from='client.? 172.21.15.93:0/2770596946' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:48.003 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-16T16:55:48.021 INFO:tasks.cephadm:Stopping all daemons... 2023-11-16T16:55:48.022 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-11-16T16:55:48.022 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.a 2023-11-16T16:55:48.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:47 smithi125 ceph-mon[131076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-11-16T16:55:48.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:47 smithi125 ceph-mon[131076]: osdmap e97: 12 total, 12 up, 12 in 2023-11-16T16:55:48.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:47 smithi125 ceph-mon[131076]: pgmap v565: 33 pgs: 33 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:55:48.078 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:47 smithi125 ceph-mon[131076]: from='client.? 172.21.15.93:0/2770596946' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:48.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:47 smithi093 ceph-mon[97863]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-11-16T16:55:48.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:47 smithi093 ceph-mon[97863]: osdmap e97: 12 total, 12 up, 12 in 2023-11-16T16:55:48.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:47 smithi093 ceph-mon[97863]: pgmap v565: 33 pgs: 33 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:55:48.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:47 smithi093 ceph-mon[97863]: from='client.? 172.21.15.93:0/2770596946' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:55:48.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:48 smithi093 systemd[1]: Stopping Ceph mon.a for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:55:49.106 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-a[97839]: 2023-11-16T16:55:48.656+0000 7f37378c3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:55:49.107 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-a[97839]: 2023-11-16T16:55:48.656+0000 7f37378c3700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-11-16T16:55:49.359 INFO:journalctl@ceph.mon.a.smithi093.stdout:Nov 16 16:55:49 smithi093 bash[254153]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-a 2023-11-16T16:55:49.661 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.a.service' 2023-11-16T16:55:49.767 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:55:49.768 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-11-16T16:55:49.768 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-11-16T16:55:49.769 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.b 2023-11-16T16:55:50.079 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:49 smithi125 systemd[1]: Stopping Ceph mon.b for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:55:50.575 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:50 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-b[131052]: 2023-11-16T16:55:50.124+0000 7fbd76945700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:55:50.575 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:50 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-b[131052]: 2023-11-16T16:55:50.124+0000 7fbd76945700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-11-16T16:55:50.773 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.b.service' 2023-11-16T16:55:50.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:50 smithi125 bash[160422]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-b 2023-11-16T16:55:50.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:50 smithi125 systemd[1]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.b.service: Succeeded. 2023-11-16T16:55:50.828 INFO:journalctl@ceph.mon.b.smithi125.stdout:Nov 16 16:55:50 smithi125 systemd[1]: Stopped Ceph mon.b for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:55:50.845 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:55:50.846 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-11-16T16:55:50.846 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-11-16T16:55:50.847 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.c 2023-11-16T16:55:51.226 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:50 smithi159 systemd[1]: Stopping Ceph mon.c for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:55:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:51 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-c[131487]: 2023-11-16T16:55:51.223+0000 7f6aff4e9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:55:51.485 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:51 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-c[131487]: 2023-11-16T16:55:51.223+0000 7f6aff4e9700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-11-16T16:55:51.859 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.c.service' 2023-11-16T16:55:51.885 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:51 smithi159 bash[161802]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mon-c 2023-11-16T16:55:51.885 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:51 smithi159 systemd[1]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mon.c.service: Succeeded. 2023-11-16T16:55:51.885 INFO:journalctl@ceph.mon.c.smithi159.stdout:Nov 16 16:55:51 smithi159 systemd[1]: Stopped Ceph mon.c for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:55:51.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:55:51.905 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-11-16T16:55:51.905 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-11-16T16:55:51.905 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.x 2023-11-16T16:55:52.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:55:52 smithi093 systemd[1]: Stopping Ceph mgr.x for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:55:52.681 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:55:52 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:55:52.422+0000 7f1905c2a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:55:52.681 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:55:52 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x[98111]: 2023-11-16T16:55:52.422+0000 7f1905c2a700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-11-16T16:55:53.137 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Nov 16 16:55:52 smithi093 bash[254302]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-x 2023-11-16T16:55:53.170 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.x.service' 2023-11-16T16:55:53.258 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:55:53.258 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-11-16T16:55:53.258 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-11-16T16:55:53.259 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.y 2023-11-16T16:55:53.578 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:55:53 smithi125 systemd[1]: Stopping Ceph mgr.y for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:55:53.968 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:55:53 smithi125 bash[160573]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-y 2023-11-16T16:55:53.968 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Nov 16 16:55:53 smithi125 systemd[1]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-11-16T16:55:54.149 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.y.service' 2023-11-16T16:55:54.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:55:54.185 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-11-16T16:55:54.185 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-11-16T16:55:54.185 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.z 2023-11-16T16:55:54.485 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:55:54 smithi159 systemd[1]: Stopping Ceph mgr.z for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:55:54.896 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:55:54 smithi159 bash[161953]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mgr-z 2023-11-16T16:55:55.077 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Nov 16 16:55:54 smithi159 systemd[1]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-11-16T16:55:55.082 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mgr.z.service' 2023-11-16T16:55:55.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:55:55.120 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-11-16T16:55:55.121 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-11-16T16:55:55.121 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.0 2023-11-16T16:55:55.648 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:55:55 smithi093 systemd[1]: Stopping Ceph osd.0 for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:55:55.900 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:55:55 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-0[110160]: 2023-11-16T16:55:55.646+0000 7ff27d68c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:55:55.900 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:55:55 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-0[110160]: 2023-11-16T16:55:55.646+0000 7ff27d68c700 -1 osd.0 98 *** Got signal Terminated *** 2023-11-16T16:55:55.900 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:55:55 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-0[110160]: 2023-11-16T16:55:55.646+0000 7ff27d68c700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:56:01.900 INFO:journalctl@ceph.osd.0.smithi093.stdout:Nov 16 16:56:01 smithi093 bash[254448]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-0 2023-11-16T16:56:04.950 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.0.service' 2023-11-16T16:56:05.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:56:05.041 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-11-16T16:56:05.041 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-11-16T16:56:05.041 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.1 2023-11-16T16:56:05.328 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:56:05 smithi125 systemd[1]: Stopping Ceph osd.1 for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:56:05.828 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:56:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-1[136185]: 2023-11-16T16:56:05.406+0000 7f3e559de700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:56:05.828 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:56:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-1[136185]: 2023-11-16T16:56:05.406+0000 7f3e559de700 -1 osd.1 98 *** Got signal Terminated *** 2023-11-16T16:56:05.828 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:56:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-1[136185]: 2023-11-16T16:56:05.406+0000 7f3e559de700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:56:11.179 INFO:journalctl@ceph.osd.1.smithi125.stdout:Nov 16 16:56:11 smithi125 bash[160719]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-1 2023-11-16T16:56:12.447 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.1.service' 2023-11-16T16:56:12.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:56:12.487 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-11-16T16:56:12.488 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-11-16T16:56:12.488 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.2 2023-11-16T16:56:12.855 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:56:12 smithi159 systemd[1]: Stopping Ceph osd.2 for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:56:13.234 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:56:12 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-2[137311]: 2023-11-16T16:56:12.853+0000 7fdad3468700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:56:13.235 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:56:12 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-2[137311]: 2023-11-16T16:56:12.853+0000 7fdad3468700 -1 osd.2 98 *** Got signal Terminated *** 2023-11-16T16:56:13.235 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:56:12 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-2[137311]: 2023-11-16T16:56:12.853+0000 7fdad3468700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:56:18.735 INFO:journalctl@ceph.osd.2.smithi159.stdout:Nov 16 16:56:18 smithi159 bash[162098]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-2 2023-11-16T16:56:19.893 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.2.service' 2023-11-16T16:56:19.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:56:19.934 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-11-16T16:56:19.935 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-11-16T16:56:19.935 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.3 2023-11-16T16:56:20.400 INFO:journalctl@ceph.osd.3.smithi093.stdout:Nov 16 16:56:20 smithi093 systemd[1]: Stopping Ceph osd.3 for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:56:20.900 INFO:journalctl@ceph.osd.3.smithi093.stdout:Nov 16 16:56:20 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-3[115738]: 2023-11-16T16:56:20.468+0000 7fb70b144700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:56:20.900 INFO:journalctl@ceph.osd.3.smithi093.stdout:Nov 16 16:56:20 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-3[115738]: 2023-11-16T16:56:20.468+0000 7fb70b144700 -1 osd.3 98 *** Got signal Terminated *** 2023-11-16T16:56:20.900 INFO:journalctl@ceph.osd.3.smithi093.stdout:Nov 16 16:56:20 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-3[115738]: 2023-11-16T16:56:20.468+0000 7fb70b144700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:56:23.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:22 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:22.987+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:23.735 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:23 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:23.396+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:24.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:23 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:23.805+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:24.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:23 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:23.977+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:24.735 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:24 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:24.362+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:25.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:24 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:24.787+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:25.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:24 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:24.949+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:25.650 INFO:journalctl@ceph.osd.3.smithi093.stdout:Nov 16 16:56:25 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-3[115738]: 2023-11-16T16:56:25.287+0000 7fb706f39700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.549556+0000 front 2023-11-16T16:55:59.549446+0000 (oldest deadline 2023-11-16T16:56:24.849616+0000) 2023-11-16T16:56:25.734 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:25 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:25.405+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:26.133 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:25 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:25.781+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:26.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:25 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:25.920+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:26.400 INFO:journalctl@ceph.osd.3.smithi093.stdout:Nov 16 16:56:26 smithi093 bash[254729]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-3 2023-11-16T16:56:26.735 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:26 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:26.444+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:27.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:26 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:26.763+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:27.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:26 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:26.815+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:27.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:26 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:26.885+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:27.734 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:27 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:27.470+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:28.114 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:27 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:27.722+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:28.142 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.3.service' 2023-11-16T16:56:28.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:56:28.232 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-11-16T16:56:28.232 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-11-16T16:56:28.232 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.4 2023-11-16T16:56:28.234 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:27 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:27.847+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:28.262 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:27 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:27.900+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:28.577 INFO:journalctl@ceph.osd.4.smithi125.stdout:Nov 16 16:56:28 smithi125 systemd[1]: Stopping Ceph osd.4 for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:56:28.735 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:28 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:28.433+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:28.909 INFO:journalctl@ceph.osd.4.smithi125.stdout:Nov 16 16:56:28 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-4[141625]: 2023-11-16T16:56:28.594+0000 7f2eee6ec700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:56:28.910 INFO:journalctl@ceph.osd.4.smithi125.stdout:Nov 16 16:56:28 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-4[141625]: 2023-11-16T16:56:28.594+0000 7f2eee6ec700 -1 osd.4 98 *** Got signal Terminated *** 2023-11-16T16:56:28.910 INFO:journalctl@ceph.osd.4.smithi125.stdout:Nov 16 16:56:28 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-4[141625]: 2023-11-16T16:56:28.594+0000 7f2eee6ec700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:56:29.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:28 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:28.720+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:29.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:28 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:28.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:29.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:28 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:28.908+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:29.735 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:29 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:29.472+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:30.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:29 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:29.769+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:30.234 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:29 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:29.883+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:30.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:29 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:29.911+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:30.735 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:30 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:30.444+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:31.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:30 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:30.791+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:31.234 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:30 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:30.919+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:31.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:30 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:30.902+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:31.735 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:31 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:31.423+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:31.827 INFO:journalctl@ceph.osd.4.smithi125.stdout:Nov 16 16:56:31 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-4[141625]: 2023-11-16T16:56:31.462+0000 7f2ee94df700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.460022+0000 front 2023-11-16T16:56:08.460044+0000 (oldest deadline 2023-11-16T16:56:31.360119+0000) 2023-11-16T16:56:32.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:31 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:31.799+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:32.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:31 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:31.871+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:32.328 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:32 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:32.055+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:32.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:31 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:31.911+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:32.735 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:32 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:32.460+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:32.828 INFO:journalctl@ceph.osd.4.smithi125.stdout:Nov 16 16:56:32 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-4[141625]: 2023-11-16T16:56:32.508+0000 7f2ee94df700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.460022+0000 front 2023-11-16T16:56:08.460044+0000 (oldest deadline 2023-11-16T16:56:31.360119+0000) 2023-11-16T16:56:33.125 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:32 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:32.814+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:33.234 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:32 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:32.832+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:33.328 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:33 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:33.082+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:33.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:32 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:32.938+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:33.400 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:33 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:33.124+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:33.784 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:33 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:33.507+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:33.827 INFO:journalctl@ceph.osd.4.smithi125.stdout:Nov 16 16:56:33 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-4[141625]: 2023-11-16T16:56:33.489+0000 7f2ee94df700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.460022+0000 front 2023-11-16T16:56:08.460044+0000 (oldest deadline 2023-11-16T16:56:31.360119+0000) 2023-11-16T16:56:34.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:33 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:33.849+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:34.234 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:33 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:33.783+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:34.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:33 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:33.783+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:34.268 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:34 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:34.076+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:34.268 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:33 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:33.897+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:34.268 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:33 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:33.897+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:34.578 INFO:journalctl@ceph.osd.4.smithi125.stdout:Nov 16 16:56:34 smithi125 bash[160979]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-4 2023-11-16T16:56:34.650 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:34 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:34.158+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:34.985 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:34 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:34.535+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:34.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:34 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:34.641+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:34.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:34 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:34.754+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:34.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:34 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:34.754+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:35.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:34 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:34.826+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:35.307 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:35 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:35.062+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:35.307 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:34 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:34.893+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:35.307 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:34 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:34.893+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:35.628 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.4.service' 2023-11-16T16:56:35.650 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:35 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:35.197+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:35.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:56:35.708 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-11-16T16:56:35.708 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-11-16T16:56:35.709 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.5 2023-11-16T16:56:35.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:35 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:35.671+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:35.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:35 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:35.756+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:35.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:35 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:35.756+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:35.986 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:35 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:35.560+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:35.986 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:35 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:35.560+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.157718+0000 front 2023-11-16T16:56:10.157674+0000 (oldest deadline 2023-11-16T16:56:35.457704+0000) 2023-11-16T16:56:35.986 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:35 smithi159 systemd[1]: Stopping Ceph osd.5 for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:56:36.070 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:35 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:35.872+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:36.070 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:35 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:35.872+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:36.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:35 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:35.871+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:36.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:35 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:35.871+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:36.327 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:36 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:36.069+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:36.485 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:36 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:36.067+0000 7f9190ac1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:56:36.485 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:36 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:36.067+0000 7f9190ac1700 -1 osd.5 98 *** Got signal Terminated *** 2023-11-16T16:56:36.485 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:36 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:36.067+0000 7f9190ac1700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:56:36.650 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:36 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:36.151+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:36.785 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:36 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:36.665+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:36.786 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:36 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:36.524+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:36.786 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:36 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:36.524+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.157718+0000 front 2023-11-16T16:56:10.157674+0000 (oldest deadline 2023-11-16T16:56:35.457704+0000) 2023-11-16T16:56:37.106 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:36 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:36.827+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:37.106 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:36 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:36.827+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:37.144 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:36 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:36.859+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:37.144 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:36 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:36.859+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:37.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:36 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:36.784+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:37.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:36 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:36.784+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:37.400 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:37 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:37.142+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:37.577 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:37 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:37.104+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:37.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:37 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:37.681+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:37.985 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:37 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:37.516+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:37.985 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:37 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:37.516+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.157718+0000 front 2023-11-16T16:56:10.157674+0000 (oldest deadline 2023-11-16T16:56:35.457704+0000) 2023-11-16T16:56:37.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:37 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:37.755+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:37.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:37 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:37.755+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:38.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:37 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:37.816+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:38.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:37 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:37.816+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:38.327 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:38 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:38.068+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:38.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:37 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:37.844+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:38.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:37 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:37.844+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:38.650 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:38 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:38.155+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:38.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:38 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:38.632+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:38.986 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:38 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:38.549+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:38.986 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:38 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:38.549+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.157718+0000 front 2023-11-16T16:56:10.157674+0000 (oldest deadline 2023-11-16T16:56:35.457704+0000) 2023-11-16T16:56:38.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:38 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:38.793+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:38.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:38 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:38.793+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:39.101 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:38 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:38.848+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:39.101 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:38 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:38.848+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:39.124 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:38 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:38.821+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:39.124 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:38 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:38.821+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:39.400 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:39 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:39.122+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:39.577 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:39 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:39.099+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:39.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:39 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:39.099+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:39.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:39 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:39.643+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:39.985 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:39 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:39.509+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:39.985 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:39 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:39.509+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.157718+0000 front 2023-11-16T16:56:10.157674+0000 (oldest deadline 2023-11-16T16:56:35.457704+0000) 2023-11-16T16:56:39.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:39 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:39.755+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:39.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:39 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:39.755+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:40.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:39 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:39.857+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:40.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:39 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:39.857+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:40.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:39 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:39.832+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:40.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:39 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:39.832+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:40.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:39 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:39.832+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:40.328 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:40 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:40.064+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:40.328 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:40 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:40.064+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:40.650 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:40 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:40.159+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:40.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:40 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:40.635+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:40.985 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:40 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:40.541+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:59.055774+0000 front 2023-11-16T16:55:59.055593+0000 (oldest deadline 2023-11-16T16:56:22.555675+0000) 2023-11-16T16:56:40.985 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:40 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5[142801]: 2023-11-16T16:56:40.541+0000 7f918b0b3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.157718+0000 front 2023-11-16T16:56:10.157674+0000 (oldest deadline 2023-11-16T16:56:35.457704+0000) 2023-11-16T16:56:40.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:40 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:40.769+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:40.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:40 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:40.769+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:40.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:40 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:40.769+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:41.077 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:40 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:40.795+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:41.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:40 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:40.795+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:41.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:40 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:40.795+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:41.137 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:40 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:40.837+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:41.137 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:40 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:40.837+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:41.400 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:41 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:41.136+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:41.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:41 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:41.086+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:41.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:41 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:41.087+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:41.985 INFO:journalctl@ceph.osd.5.smithi159.stdout:Nov 16 16:56:41 smithi159 bash[162363]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-5 2023-11-16T16:56:41.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:41 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:41.661+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:41.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:41 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:41.661+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:41.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:41 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:41.740+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:41.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:41 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:41.740+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:41.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:41 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:41.740+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:42.077 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:41 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:41.777+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:42.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:41 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:41.777+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:42.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:41 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:41.777+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:42.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:41 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:41.808+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:42.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:41 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:41.808+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:42.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:41 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:41.808+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:42.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:42 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:42.114+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:42.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:42 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:42.114+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:42.650 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:42 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:42.185+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:42.783 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:42 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:42.661+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:42.783 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:42 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:42.661+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:43.077 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:42 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:42.826+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:43.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:42 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:42.826+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:43.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:42 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:42.826+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:43.078 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:43.067+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:43.078 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:43.067+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:43.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:42 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:42.854+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:43.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:42 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:42.854+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:43.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:42 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:42.854+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:43.211 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:42 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:42.781+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:43.211 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:42 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:42.781+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:43.211 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:42 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:42.781+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:43.218 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.5.service' 2023-11-16T16:56:43.258 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:56:43.258 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-11-16T16:56:43.259 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-11-16T16:56:43.259 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.6 2023-11-16T16:56:43.650 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:43.186+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:43.650 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:43.186+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:16.431367+0000 front 2023-11-16T16:56:16.431252+0000 (oldest deadline 2023-11-16T16:56:42.832013+0000) 2023-11-16T16:56:43.650 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:43 smithi093 systemd[1]: Stopping Ceph osd.6 for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:56:43.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:43.711+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:43.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:43.711+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:43.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:43.801+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:43.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:43.801+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:43.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:43 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:43.801+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:44.066 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:43.797+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:44.066 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:43.797+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:44.066 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:43 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:43.797+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:44.150 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:43.772+0000 7f12abb66700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:56:44.150 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:43.772+0000 7f12abb66700 -1 osd.6 98 *** Got signal Terminated *** 2023-11-16T16:56:44.150 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:43.772+0000 7f12abb66700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:56:44.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:43.892+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:44.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:43.892+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:44.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:43 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:43.892+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:44.328 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:44 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:44.065+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:44.328 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:44 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:44.065+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:44.650 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:44 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:44.160+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:44.650 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:44 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:44.160+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:16.431367+0000 front 2023-11-16T16:56:16.431252+0000 (oldest deadline 2023-11-16T16:56:42.832013+0000) 2023-11-16T16:56:44.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:44 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:44.679+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:44.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:44 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:44.679+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:44.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:44 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:44.842+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:44.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:44 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:44.842+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:44.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:44 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:44.842+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:45.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:44 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:44.749+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:45.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:44 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:44.749+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:45.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:44 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:44.749+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:45.131 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:44 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:44.852+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:45.131 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:44 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:44.852+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:45.131 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:44 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:44.852+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:45.400 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:45 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:45.130+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:45.400 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:45 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:45.130+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:16.431367+0000 front 2023-11-16T16:56:16.431252+0000 (oldest deadline 2023-11-16T16:56:42.832013+0000) 2023-11-16T16:56:45.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:45 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:45.109+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:45.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:45 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:45.109+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:45.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:45 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:45.713+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:45.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:45 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:45.713+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:45.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:45 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:45.859+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:45.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:45 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:45.859+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:45.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:45 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:45.859+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:46.065 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:45 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:45.772+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:46.066 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:45 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:45.772+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:46.066 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:45 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:45.772+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:46.150 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:46 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:46.129+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:46.150 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:46 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:46.129+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:16.431367+0000 front 2023-11-16T16:56:16.431252+0000 (oldest deadline 2023-11-16T16:56:42.832013+0000) 2023-11-16T16:56:46.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:45 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:45.891+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:46.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:45 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:45.891+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:46.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:45 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:45.891+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:46.327 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:46 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:46.065+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:46.327 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:46 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:46.065+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:47.077 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:46 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:46.808+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:47.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:46 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:46.808+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:47.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:46 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:46.808+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:47.078 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:47 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:47.025+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:47.078 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:47 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:47.025+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:47.131 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:46 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:46.864+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:47.131 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:46 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:46.864+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:47.131 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:46 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:46.864+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:47.131 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:46 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:46.864+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:47.235 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:46 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:46.738+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:47.235 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:46 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:46.738+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:47.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:46 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:46.825+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:47.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:46 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:46.825+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:47.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:46 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:46.825+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:47.400 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:47 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:47.129+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:47.400 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:47 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:47.130+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:16.431367+0000 front 2023-11-16T16:56:16.431252+0000 (oldest deadline 2023-11-16T16:56:42.832013+0000) 2023-11-16T16:56:47.400 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:47 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:47.130+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:22.832388+0000 front 2023-11-16T16:56:22.832470+0000 (oldest deadline 2023-11-16T16:56:46.932452+0000) 2023-11-16T16:56:47.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:47 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:47.695+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:47.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:47 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:47.695+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:47.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:47 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:47.779+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:47.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:47 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:47.779+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:47.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:47 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:47.779+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:48.054 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:47 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:47.793+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:48.055 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:47 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:47.793+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:48.055 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:47 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:47.793+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:48.328 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:48 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:48.054+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:48.328 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:48 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:48.054+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:48.400 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:48.136+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:09.429780+0000 front 2023-11-16T16:56:09.429669+0000 (oldest deadline 2023-11-16T16:56:32.929694+0000) 2023-11-16T16:56:48.400 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:48.136+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:16.431367+0000 front 2023-11-16T16:56:16.431252+0000 (oldest deadline 2023-11-16T16:56:42.832013+0000) 2023-11-16T16:56:48.400 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6[121266]: 2023-11-16T16:56:48.136+0000 7f12a6158700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:22.832388+0000 front 2023-11-16T16:56:22.832470+0000 (oldest deadline 2023-11-16T16:56:46.932452+0000) 2023-11-16T16:56:48.401 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:47 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:47.900+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:48.401 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:47 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:47.900+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:48.401 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:47 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:47.900+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:48.401 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:47 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:47.900+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:48.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:48 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:48.663+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:48.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:48 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:48.663+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:48.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:48 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:48.785+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:48.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:48 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:48.785+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:48.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:48 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:48.785+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:49.039 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:48 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:48.777+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:49.040 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:48 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:48.777+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:49.040 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:48 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:48.777+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:49.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:48.860+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:49.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:48.860+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:49.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:48.860+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:49.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:48 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:48.860+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:49.328 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:49 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:49.038+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:49.328 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:49 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:49.038+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:49.650 INFO:journalctl@ceph.osd.6.smithi093.stdout:Nov 16 16:56:49 smithi093 bash[255005]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-6 2023-11-16T16:56:49.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:49 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:49.677+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:49.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:49 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:49.677+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:49.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:49 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:49.793+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:49.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:49 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:49.793+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:49.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:49 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:49.793+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:50.078 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:50 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:50.036+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:50.078 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:50 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:50.036+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:50.078 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:50 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:50.036+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:25.347748+0000 front 2023-11-16T16:56:25.347606+0000 (oldest deadline 2023-11-16T16:56:49.447795+0000) 2023-11-16T16:56:50.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:49 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:49.817+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:50.079 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:49 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:49.817+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:50.079 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:49 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:49.817+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:50.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:49 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:49.856+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:50.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:49 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:49.856+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:50.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:49 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:49.856+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:50.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:49 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:49.856+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:50.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:50 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:50.649+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:50.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:50 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:50.649+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:50.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:50 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:50.649+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:56:50.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:50 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:50.831+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:50.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:50 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:50.831+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:50.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:50 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:50.831+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:51.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:50 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:50.870+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:51.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:50 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:50.870+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:51.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:50 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:50.870+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:51.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:50 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:50.870+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:51.317 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.6.service' 2023-11-16T16:56:51.327 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:51 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:51.087+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:51.328 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:51 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:51.087+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:51.328 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:51 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:51.087+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:25.347748+0000 front 2023-11-16T16:56:25.347606+0000 (oldest deadline 2023-11-16T16:56:49.447795+0000) 2023-11-16T16:56:51.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:50 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:50.855+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:51.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:50 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:50.855+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:51.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:50 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:50.855+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:51.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:50 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:50.855+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:56:51.406 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:56:51.407 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-11-16T16:56:51.407 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-11-16T16:56:51.407 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.7 2023-11-16T16:56:51.767 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:51 smithi125 systemd[1]: Stopping Ceph osd.7 for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:56:51.767 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:51 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:51.764+0000 7f2ccd65a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:56:51.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:51 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:51.674+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:51.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:51 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:51.674+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:51.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:51 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:51.674+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:56:51.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:51 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:51.824+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:51.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:51 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:51.824+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:51.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:51 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:51.824+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:52.077 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:51 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:51.764+0000 7f2ccd65a700 -1 osd.7 98 *** Got signal Terminated *** 2023-11-16T16:56:52.078 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:51 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:51.764+0000 7f2ccd65a700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:56:52.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:51 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:51.853+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:52.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:51 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:51.853+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:52.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:51 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:51.853+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:52.079 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:51 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:51.853+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:56:52.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:51 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:51.909+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:52.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:51 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:51.909+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:52.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:51 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:51.909+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:52.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:51 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:51.909+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:52.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:52 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:52.104+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:52.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:52 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:52.104+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:52.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:52 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:52.104+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:25.347748+0000 front 2023-11-16T16:56:25.347606+0000 (oldest deadline 2023-11-16T16:56:49.447795+0000) 2023-11-16T16:56:52.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:52 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:52.634+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:52.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:52 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:52.634+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:52.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:52 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:52.634+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:56:52.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:52 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:52.873+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:52.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:52 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:52.873+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:52.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:52 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:52.873+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:53.077 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:52 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:52.813+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:53.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:52 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:52.814+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:53.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:52 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:52.814+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:53.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:52 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:52.814+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:56:53.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:52 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:52.874+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:53.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:52 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:52.874+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:53.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:52 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:52.874+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:53.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:52 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:52.874+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:53.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:53 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:53.102+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:53.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:53 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:53.102+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:53.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:53 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:53.102+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:25.347748+0000 front 2023-11-16T16:56:25.347606+0000 (oldest deadline 2023-11-16T16:56:49.447795+0000) 2023-11-16T16:56:53.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:53 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:53.865+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:53.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:53 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:53.865+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:53.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:53 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:53.865+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:53.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:53 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:53.620+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:53.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:53 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:53.620+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:53.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:53 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:53.620+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:56:54.077 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:53 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:53.791+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:54.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:53 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:53.791+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:54.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:53 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:53.791+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:54.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:53 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:53.791+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:56:54.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:53 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:53.897+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:54.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:53 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:53.897+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:54.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:53 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:53.897+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:54.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:53 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:53.897+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:54.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:54 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:54.116+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:54.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:54 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:54.116+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:54.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:54 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:54.116+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:25.347748+0000 front 2023-11-16T16:56:25.347606+0000 (oldest deadline 2023-11-16T16:56:49.447795+0000) 2023-11-16T16:56:54.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:54 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:54.654+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:54.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:54 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:54.654+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:54.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:54 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:54.654+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:56:54.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:54 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:54.817+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:54.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:54 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:54.817+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:54.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:54 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:54.817+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:55.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:54 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:54.746+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:55.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:54 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:54.746+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:55.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:54 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:54.746+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:55.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:54 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:54.746+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:56:55.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:54 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:54.909+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:55.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:54 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:54.909+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:55.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:54 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:54.909+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:55.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:54 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:54.909+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:55.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:55 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:55.144+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:55.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:55 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:55.144+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:55.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:55 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:55.144+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:25.347748+0000 front 2023-11-16T16:56:25.347606+0000 (oldest deadline 2023-11-16T16:56:49.447795+0000) 2023-11-16T16:56:55.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:55 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:55.650+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:55.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:55 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:55.650+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:55.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:55 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:55.650+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:56:55.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:55 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:55.793+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:55.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:55 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:55.793+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:55.987 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:55 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:55.793+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:55.987 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:55 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:55.793+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:56:56.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:55 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:55.700+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:56.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:55 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:55.700+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:56.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:55 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:55.700+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:56.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:55 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:55.700+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:56:56.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:55 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:55.865+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:56.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:55 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:55.865+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:56.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:55 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:55.865+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:56.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:55 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:55.865+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:56.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:55 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:55.865+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:56:56.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:56 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:56.171+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.643295+0000 front 2023-11-16T16:56:08.643317+0000 (oldest deadline 2023-11-16T16:56:31.543499+0000) 2023-11-16T16:56:56.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:56 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:56.171+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:13.844655+0000 front 2023-11-16T16:56:13.844768+0000 (oldest deadline 2023-11-16T16:56:38.544769+0000) 2023-11-16T16:56:56.578 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:56 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:56.171+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:25.347748+0000 front 2023-11-16T16:56:25.347606+0000 (oldest deadline 2023-11-16T16:56:49.447795+0000) 2023-11-16T16:56:56.579 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:56 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7[147172]: 2023-11-16T16:56:56.171+0000 7f2cc6449700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:31.748668+0000 front 2023-11-16T16:56:31.748688+0000 (oldest deadline 2023-11-16T16:56:55.248848+0000) 2023-11-16T16:56:56.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:56 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:56.689+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:56.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:56 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:56.689+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:56.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:56 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:56.689+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:56:56.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:56 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:56.831+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:56.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:56 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:56.831+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:56.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:56 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:56.831+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:56.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:56 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:56.831+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:56:57.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:56 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:56.680+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:57.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:56 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:56.680+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:57.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:56 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:56.680+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:57.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:56 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:56.680+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:56:57.079 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:56 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:56.680+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:56:57.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:56 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:56.888+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:57.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:56 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:56.888+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:57.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:56 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:56.888+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:57.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:56 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:56.888+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:57.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:56 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:56.888+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:56:57.446 INFO:journalctl@ceph.osd.7.smithi125.stdout:Nov 16 16:56:57 smithi125 bash[161251]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-7 2023-11-16T16:56:57.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:57 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:57.698+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:57.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:57 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:57.698+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:57.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:57 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:57.698+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:56:57.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:57 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:57.698+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.442001+0000 front 2023-11-16T16:56:30.442033+0000 (oldest deadline 2023-11-16T16:56:57.442722+0000) 2023-11-16T16:56:57.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:57 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:57.816+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:57.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:57 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:57.816+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:57.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:57 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:57.816+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:57.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:57 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:57.816+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:56:57.986 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:57 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:57.728+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:57.987 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:57 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:57.729+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:57.987 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:57 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:57.729+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:57.987 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:57 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:57.729+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:56:57.987 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:57 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:57.729+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:56:58.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:57 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:57.878+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:58.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:57 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:57.878+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:58.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:57 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:57.878+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:58.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:57 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:57.878+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:58.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:57 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:57.878+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:56:58.712 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.7.service' 2023-11-16T16:56:58.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:56:58.752 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-11-16T16:56:58.752 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-11-16T16:56:58.752 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.8 2023-11-16T16:56:58.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:58 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:58.849+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:58.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:58 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:58.849+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:58.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:58 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:58.849+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:58.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:58 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:58.849+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:56:58.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:58 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:58.654+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:58.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:58 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:58.654+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:58.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:58 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:58.654+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:56:58.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:58 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:58.654+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.442001+0000 front 2023-11-16T16:56:30.442033+0000 (oldest deadline 2023-11-16T16:56:57.442722+0000) 2023-11-16T16:56:58.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:58 smithi159 systemd[1]: Stopping Ceph osd.8 for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:56:59.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:58 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:58.774+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:56:59.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:58 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:58.774+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:56:59.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:58 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:58.774+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:56:59.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:58 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:58.774+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:56:59.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:58 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:58.774+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:56:59.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:58 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:58.911+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:56:59.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:58 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:58.911+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:56:59.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:58 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:58.911+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:56:59.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:58 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:58.911+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:56:59.400 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:58 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:58.911+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:56:59.485 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:59.121+0000 7fc094b94700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:56:59.485 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:59.121+0000 7fc094b94700 -1 osd.8 98 *** Got signal Terminated *** 2023-11-16T16:56:59.485 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:59.121+0000 7fc094b94700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:56:59.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:59.685+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:56:59.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:59.685+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:56:59.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:59.685+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:56:59.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:56:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:56:59.685+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.442001+0000 front 2023-11-16T16:56:30.442033+0000 (oldest deadline 2023-11-16T16:56:57.442722+0000) 2023-11-16T16:56:59.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:59.883+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:56:59.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:59.883+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:56:59.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:59.883+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:56:59.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:56:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:56:59.883+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:00.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:59 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:59.795+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:00.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:59 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:59.795+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:00.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:59 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:59.795+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:00.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:59 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:59.795+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:00.078 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:56:59 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:56:59.795+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:00.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:59 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:59.864+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:00.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:59 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:59.864+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:00.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:59 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:59.864+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:00.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:59 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:59.864+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:00.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:56:59 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:56:59.864+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:00.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:00.697+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:57:00.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:00.697+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:57:00.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:00.697+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:57:00.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:00.697+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.442001+0000 front 2023-11-16T16:56:30.442033+0000 (oldest deadline 2023-11-16T16:56:57.442722+0000) 2023-11-16T16:57:00.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:00.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:00.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:00.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:00.987 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:00.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:00.987 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:00 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:00.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:01.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:00 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:00.879+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:01.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:00 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:00.879+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:01.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:00 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:00.879+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:01.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:00 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:00.879+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:01.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:00 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:00.879+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:01.336 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:00 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:00.832+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:01.336 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:00 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:00.832+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:01.337 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:00 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:00.832+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:01.337 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:00 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:00.832+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:01.337 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:00 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:00.832+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:01.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:01 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:01.713+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:57:01.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:01 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:01.713+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:57:01.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:01 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:01.713+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:57:01.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:01 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:01.713+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.442001+0000 front 2023-11-16T16:56:30.442033+0000 (oldest deadline 2023-11-16T16:56:57.442722+0000) 2023-11-16T16:57:01.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:01 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:01.859+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:01.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:01 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:01.859+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:01.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:01 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:01.859+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:01.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:01 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:01.859+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:02.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:01 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:01.841+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:02.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:01 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:01.841+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:02.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:01 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:01.841+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:02.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:01 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:01.841+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:02.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:01 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:01.841+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:02.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:01 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:01.850+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:02.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:01 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:01.850+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:02.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:01 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:01.850+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:02.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:01 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:01.850+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:02.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:01 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:01.850+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:02.329 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:01 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:01.850+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:02.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:02.718+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:57:02.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:02.718+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:57:02.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:02.718+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:57:02.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:02.718+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.442001+0000 front 2023-11-16T16:56:30.442033+0000 (oldest deadline 2023-11-16T16:56:57.442722+0000) 2023-11-16T16:57:02.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:02.718+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:39.043946+0000 front 2023-11-16T16:56:39.043967+0000 (oldest deadline 2023-11-16T16:57:01.844529+0000) 2023-11-16T16:57:02.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:02.830+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:02.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:02.830+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:02.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:02.830+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:02.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:02.830+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:02.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:02 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:02.830+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:03.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:02 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:02.831+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:03.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:02 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:02.831+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:03.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:02 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:02.831+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:03.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:02 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:02.831+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:03.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:02 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:02.831+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:03.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:02.840+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:03.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:02.840+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:03.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:02.840+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:03.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:02.840+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:03.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:02.840+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:03.329 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:02 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:02.840+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:03.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:03.705+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.138989+0000 front 2023-11-16T16:56:08.139103+0000 (oldest deadline 2023-11-16T16:56:34.039180+0000) 2023-11-16T16:57:03.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:03.705+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:14.039602+0000 front 2023-11-16T16:56:14.039582+0000 (oldest deadline 2023-11-16T16:56:41.040378+0000) 2023-11-16T16:57:03.985 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:03.705+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.041051+0000 front 2023-11-16T16:56:21.041085+0000 (oldest deadline 2023-11-16T16:56:50.441471+0000) 2023-11-16T16:57:03.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:03.705+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.442001+0000 front 2023-11-16T16:56:30.442033+0000 (oldest deadline 2023-11-16T16:56:57.442722+0000) 2023-11-16T16:57:03.986 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8[148243]: 2023-11-16T16:57:03.705+0000 7fc08d983700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:39.043946+0000 front 2023-11-16T16:56:39.043967+0000 (oldest deadline 2023-11-16T16:57:01.844529+0000) 2023-11-16T16:57:03.986 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:03.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:03.987 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:03.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:03.987 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:03.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:03.987 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:03.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:03.988 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:03 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:03.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:04.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:03 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:03.874+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:04.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:03 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:03.874+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:04.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:03 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:03.874+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:04.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:03 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:03.874+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:04.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:03 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:03.874+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:04.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:03 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:03.874+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.459685+0000 front 2023-11-16T16:56:38.459704+0000 (oldest deadline 2023-11-16T16:57:03.759762+0000) 2023-11-16T16:57:04.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:03 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:03.871+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:04.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:03 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:03.871+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:04.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:03 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:03.871+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:04.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:03 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:03.871+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:04.329 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:03 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:03.871+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:04.329 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:03 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:03.871+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:05.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:04 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:04.841+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:05.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:04 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:04.841+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:05.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:04 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:04.841+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:05.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:04 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:04.841+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:05.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:04 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:04.841+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:05.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:04 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:04.841+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.459685+0000 front 2023-11-16T16:56:38.459704+0000 (oldest deadline 2023-11-16T16:57:03.759762+0000) 2023-11-16T16:57:05.235 INFO:journalctl@ceph.osd.8.smithi159.stdout:Nov 16 16:57:04 smithi159 bash[162650]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-8 2023-11-16T16:57:05.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:04 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:04.905+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:05.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:04 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:04.905+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:05.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:04 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:04.905+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:05.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:04 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:04.905+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:05.237 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:04 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:04.905+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:05.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:04 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:04.913+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:05.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:04 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:04.913+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:05.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:04 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:04.913+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:05.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:04 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:04.913+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:05.329 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:04 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:04.913+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:05.329 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:04 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:04.913+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:06.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:05 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:05.886+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:06.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:05 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:05.886+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:06.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:05 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:05.886+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:06.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:05 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:05.886+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:06.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:05 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:05.886+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:06.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:05 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:05.886+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.459685+0000 front 2023-11-16T16:56:38.459704+0000 (oldest deadline 2023-11-16T16:57:03.759762+0000) 2023-11-16T16:57:06.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:05 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:05.903+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:06.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:05 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:05.903+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:06.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:05 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:05.903+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:06.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:05 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:05.903+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:06.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:05 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:05.903+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:06.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:05.931+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:06.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:05.931+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:06.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:05.931+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:06.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:05.931+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:06.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:05.931+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:06.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:05 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:05.931+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:06.351 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.8.service' 2023-11-16T16:57:06.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:57:06.391 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-11-16T16:57:06.391 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-11-16T16:57:06.392 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.9 2023-11-16T16:57:06.899 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:06 smithi093 systemd[1]: Stopping Ceph osd.9 for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:57:07.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:06 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:06.898+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:07.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:06 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:06.898+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:07.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:06 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:06.898+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:07.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:06 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:06.898+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:07.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:06 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:06.898+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:07.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:06 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:06.898+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.459685+0000 front 2023-11-16T16:56:38.459704+0000 (oldest deadline 2023-11-16T16:57:03.759762+0000) 2023-11-16T16:57:07.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:06 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:06.960+0000 7f9831cb8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:57:07.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:06 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:06.960+0000 7f9831cb8700 -1 osd.9 98 *** Got signal Terminated *** 2023-11-16T16:57:07.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:06 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:06.960+0000 7f9831cb8700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:57:07.234 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:06 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:06.894+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:07.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:06 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:06.894+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:07.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:06 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:06.894+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:07.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:06 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:06.894+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:07.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:06 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:06.894+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:07.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:06 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:06.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:07.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:06 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:06.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:07.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:06 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:06.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:07.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:06 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:06.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:07.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:06 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:06.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:07.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:06 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:06.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:08.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:07 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:07.873+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:08.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:07 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:07.873+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:08.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:07 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:07.873+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:08.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:07 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:07.873+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:08.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:07 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:07.873+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:08.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:07 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:07.873+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.459685+0000 front 2023-11-16T16:56:38.459704+0000 (oldest deadline 2023-11-16T16:57:03.759762+0000) 2023-11-16T16:57:08.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:07 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:07.870+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:08.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:07 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:07.870+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:08.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:07 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:07.870+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:08.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:07 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:07.870+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:08.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:07 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:07.870+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:08.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:07 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:07.942+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:08.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:07 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:07.942+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:08.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:07 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:07.942+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:08.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:07 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:07.942+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:08.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:07 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:07.942+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:08.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:07 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:07.942+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:09.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:08 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:08.840+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:09.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:08 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:08.840+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:09.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:08 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:08.840+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:09.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:08 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:08.840+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:09.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:08 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:08.840+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:09.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:08 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:08.840+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.459685+0000 front 2023-11-16T16:56:38.459704+0000 (oldest deadline 2023-11-16T16:57:03.759762+0000) 2023-11-16T16:57:09.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:08 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:08.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:09.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:08 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:08.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:09.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:08 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:08.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:09.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:08 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:08.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:09.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:08 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:08.874+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:09.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:08 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:08.935+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:09.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:08 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:08.935+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:09.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:08 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:08.935+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:09.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:08 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:08.935+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:09.329 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:08 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:08.935+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:09.329 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:08 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:08.935+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:10.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:09 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:09.872+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:10.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:09 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:09.872+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:10.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:09 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:09.872+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:10.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:09 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:09.872+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:10.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:09 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:09.872+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:10.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:09 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:09.872+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.459685+0000 front 2023-11-16T16:56:38.459704+0000 (oldest deadline 2023-11-16T16:57:03.759762+0000) 2023-11-16T16:57:10.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:09 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:09.853+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:10.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:09 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:09.853+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:10.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:09 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:09.853+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:10.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:09 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:09.853+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:10.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:09 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:09.853+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:10.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:09 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:09.908+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:10.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:09 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:09.908+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:10.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:09 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:09.908+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:10.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:09 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:09.908+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:10.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:09 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:09.908+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:10.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:09 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:09.908+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:11.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:10 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:10.821+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:11.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:10 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:10.821+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:11.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:10 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:10.821+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:11.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:10 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:10.821+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:11.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:10 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:10.821+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:11.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:10 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:10.821+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.459685+0000 front 2023-11-16T16:56:38.459704+0000 (oldest deadline 2023-11-16T16:57:03.759762+0000) 2023-11-16T16:57:11.234 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:10 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:10.862+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:11.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:10 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:10.862+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:11.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:10 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:10.862+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:11.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:10 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:10.862+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:11.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:10 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:10.862+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:11.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:10 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:10.923+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:11.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:10 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:10.923+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:11.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:10 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:10.923+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:11.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:10 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:10.923+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:11.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:10 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:10.923+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:11.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:10 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:10.923+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:11.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:10 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:10.923+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:45.061464+0000 front 2023-11-16T16:56:45.061554+0000 (oldest deadline 2023-11-16T16:57:10.261767+0000) 2023-11-16T16:57:12.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:11 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:11.795+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:56:00.151130+0000 front 2023-11-16T16:56:00.151293+0000 (oldest deadline 2023-11-16T16:56:23.651336+0000) 2023-11-16T16:57:12.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:11 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:11.795+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:10.053685+0000 front 2023-11-16T16:56:10.053548+0000 (oldest deadline 2023-11-16T16:56:35.854230+0000) 2023-11-16T16:57:12.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:11 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:11.795+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.555530+0000 front 2023-11-16T16:56:17.555655+0000 (oldest deadline 2023-11-16T16:56:41.655480+0000) 2023-11-16T16:57:12.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:11 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:11.795+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:21.655934+0000 front 2023-11-16T16:56:21.655964+0000 (oldest deadline 2023-11-16T16:56:46.356026+0000) 2023-11-16T16:57:12.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:11 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:11.795+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.158038+0000 front 2023-11-16T16:56:32.157851+0000 (oldest deadline 2023-11-16T16:56:55.657831+0000) 2023-11-16T16:57:12.151 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:11 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9[126847]: 2023-11-16T16:57:11.795+0000 7f982caab700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.459685+0000 front 2023-11-16T16:56:38.459704+0000 (oldest deadline 2023-11-16T16:57:03.759762+0000) 2023-11-16T16:57:12.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:11 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:11.846+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:12.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:11 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:11.846+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:12.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:11 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:11.846+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:12.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:11 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:11.846+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:12.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:11 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:11.846+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:12.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:11 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:11.921+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:12.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:11 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:11.921+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:12.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:11 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:11.921+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:12.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:11 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:11.921+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:12.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:11 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:11.921+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:12.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:11 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:11.921+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:12.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:11 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:11.921+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:45.061464+0000 front 2023-11-16T16:56:45.061554+0000 (oldest deadline 2023-11-16T16:57:10.261767+0000) 2023-11-16T16:57:13.150 INFO:journalctl@ceph.osd.9.smithi093.stdout:Nov 16 16:57:12 smithi093 bash[255277]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-9 2023-11-16T16:57:13.234 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:12 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:12.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:13.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:12 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:12.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:13.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:12 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:12.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:13.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:12 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:12.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:13.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:12 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:12.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:13.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:12 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:12.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:13.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:12 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:12.964+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:13.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:12 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:12.964+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:13.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:12 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:12.964+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:13.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:12 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:12.964+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:13.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:12 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:12.964+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:13.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:12 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:12.964+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:13.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:12 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:12.964+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:45.061464+0000 front 2023-11-16T16:56:45.061554+0000 (oldest deadline 2023-11-16T16:57:10.261767+0000) 2023-11-16T16:57:14.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:13 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:13.923+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:14.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:13 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:13.923+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:14.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:13 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:13.923+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:14.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:13 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:13.923+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:14.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:13 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:13.923+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:14.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:13 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:13.923+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:14.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.006+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:14.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.006+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:14.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.006+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:14.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.006+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:14.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.006+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:14.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.006+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:14.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.006+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:45.061464+0000 front 2023-11-16T16:56:45.061554+0000 (oldest deadline 2023-11-16T16:57:10.261767+0000) 2023-11-16T16:57:14.718 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.9.service' 2023-11-16T16:57:14.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:57:14.806 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-11-16T16:57:14.806 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-11-16T16:57:14.806 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.10 2023-11-16T16:57:15.162 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 systemd[1]: Stopping Ceph osd.10 for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:57:15.162 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.979+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:15.162 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.979+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:15.163 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.979+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:15.163 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.979+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:15.163 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.979+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:15.163 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.979+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:15.163 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:14 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:14.979+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:45.061464+0000 front 2023-11-16T16:56:45.061554+0000 (oldest deadline 2023-11-16T16:57:10.261767+0000) 2023-11-16T16:57:15.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:14 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:14.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:15.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:14 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:14.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:15.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:14 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:14.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:15.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:14 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:14.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:15.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:14 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:14.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:15.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:14 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:14.880+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:15.578 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:15 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:15.160+0000 7fd2db9e3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:57:15.578 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:15 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:15.160+0000 7fd2db9e3700 -1 osd.10 98 *** Got signal Terminated *** 2023-11-16T16:57:15.578 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:15 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:15.160+0000 7fd2db9e3700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:57:16.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:15 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:15.833+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:16.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:15 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:15.833+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:16.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:15 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:15.833+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:16.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:15 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:15.833+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:16.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:15 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:15.833+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:16.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:15 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:15.833+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:16.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:15 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:15.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:16.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:15 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:15.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:16.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:15 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:15.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:16.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:15 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:15.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:16.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:15 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:15.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:16.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:15 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:15.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:16.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:15 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:15.958+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:45.061464+0000 front 2023-11-16T16:56:45.061554+0000 (oldest deadline 2023-11-16T16:57:10.261767+0000) 2023-11-16T16:57:17.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:16 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:16.816+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:17.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:16 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:16.816+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:17.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:16 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:16.816+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:17.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:16 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:16.816+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:17.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:16 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:16.816+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:17.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:16 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:16.816+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:17.327 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:16 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:16.915+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:17.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:16 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:16.915+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:17.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:16 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:16.915+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:17.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:16 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:16.915+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:17.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:16 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:16.915+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:17.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:16 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:16.915+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:17.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:16 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:16.915+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:45.061464+0000 front 2023-11-16T16:56:45.061554+0000 (oldest deadline 2023-11-16T16:57:10.261767+0000) 2023-11-16T16:57:18.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:17 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:17.780+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:18.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:17 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:17.780+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:18.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:17 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:17.780+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:18.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:17 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:17.780+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:18.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:17 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:17.780+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:18.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:17 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:17.780+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:18.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:17 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:17.957+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:18.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:17 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:17.957+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:18.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:17 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:17.957+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:18.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:17 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:17.957+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:18.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:17 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:17.957+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:18.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:17 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:17.957+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:18.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:17 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:17.957+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:45.061464+0000 front 2023-11-16T16:56:45.061554+0000 (oldest deadline 2023-11-16T16:57:10.261767+0000) 2023-11-16T16:57:19.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:18 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:18.776+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:19.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:18 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:18.776+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:19.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:18 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:18.776+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:19.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:18 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:18.776+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:19.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:18 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:18.776+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:19.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:18 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:18.776+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:19.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:18 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:18.967+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:19.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:18 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:18.967+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:19.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:18 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:18.967+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:19.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:18 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:18.967+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:19.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:18 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:18.967+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:19.329 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:18 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:18.967+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:19.329 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:18 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:18.967+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:45.061464+0000 front 2023-11-16T16:56:45.061554+0000 (oldest deadline 2023-11-16T16:57:10.261767+0000) 2023-11-16T16:57:20.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:19 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:19.784+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:20.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:19 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:19.784+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:20.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:19 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:19.784+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:20.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:19 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:19.784+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:20.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:19 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:19.784+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:20.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:19 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:19.784+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:20.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:19 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:19.920+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:56.852646+0000 front 2023-11-16T16:55:56.852686+0000 (oldest deadline 2023-11-16T16:56:22.152641+0000) 2023-11-16T16:57:20.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:19 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:19.920+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.454979+0000 front 2023-11-16T16:56:08.455000+0000 (oldest deadline 2023-11-16T16:56:33.056059+0000) 2023-11-16T16:57:20.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:19 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:19.920+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.757485+0000 front 2023-11-16T16:56:17.757301+0000 (oldest deadline 2023-11-16T16:56:38.857290+0000) 2023-11-16T16:57:20.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:19 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:19.920+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-11-16T16:56:24.758320+0000 front 2023-11-16T16:56:24.758443+0000 (oldest deadline 2023-11-16T16:56:50.658165+0000) 2023-11-16T16:57:20.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:19 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:19.920+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:30.658476+0000 front 2023-11-16T16:56:30.658494+0000 (oldest deadline 2023-11-16T16:56:55.958736+0000) 2023-11-16T16:57:20.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:19 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:19.920+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:38.760085+0000 front 2023-11-16T16:56:38.760008+0000 (oldest deadline 2023-11-16T16:57:01.659998+0000) 2023-11-16T16:57:20.328 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:19 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10[152754]: 2023-11-16T16:57:19.920+0000 7fd2d67d6700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:45.061464+0000 front 2023-11-16T16:56:45.061554+0000 (oldest deadline 2023-11-16T16:57:10.261767+0000) 2023-11-16T16:57:20.847 INFO:journalctl@ceph.osd.10.smithi125.stdout:Nov 16 16:57:20 smithi125 bash[161513]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-10 2023-11-16T16:57:21.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:20 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:20.736+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:21.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:20 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:20.736+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:21.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:20 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:20.736+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:21.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:20 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:20.736+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:21.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:20 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:20.736+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:21.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:20 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:20.736+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:21.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:21 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:21.727+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:21.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:21 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:21.727+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:21.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:21 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:21.727+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:21.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:21 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:21.727+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:21.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:21 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:21.727+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:21.985 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:21 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:21.727+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:22.112 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.10.service' 2023-11-16T16:57:22.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:57:22.151 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-11-16T16:57:22.152 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-11-16T16:57:22.152 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.11 2023-11-16T16:57:22.485 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:22 smithi159 systemd[1]: Stopping Ceph osd.11 for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:57:22.764 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:22 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:22.510+0000 7f79a8f2f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:57:22.764 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:22 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:22.510+0000 7f79a8f2f700 -1 osd.11 98 *** Got signal Terminated *** 2023-11-16T16:57:22.764 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:22 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:22.510+0000 7f79a8f2f700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:57:23.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:22 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:22.762+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:23.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:22 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:22.762+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:23.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:22 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:22.762+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:23.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:22 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:22.762+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:23.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:22 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:22.762+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:23.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:22 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:22.762+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:23.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:22 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:22.762+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2023-11-16T16:56:55.948831+0000 front 2023-11-16T16:56:55.948808+0000 (oldest deadline 2023-11-16T16:57:21.848736+0000) 2023-11-16T16:57:24.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:23 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:23.812+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:24.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:23 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:23.812+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:24.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:23 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:23.812+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:24.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:23 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:23.812+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:24.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:23 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:23.812+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:24.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:23 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:23.812+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:24.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:23 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:23.812+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2023-11-16T16:56:55.948831+0000 front 2023-11-16T16:56:55.948808+0000 (oldest deadline 2023-11-16T16:57:21.848736+0000) 2023-11-16T16:57:25.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:24 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:24.778+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:25.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:24 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:24.778+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:25.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:24 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:24.778+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:25.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:24 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:24.778+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:25.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:24 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:24.778+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:25.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:24 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:24.778+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:25.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:24 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:24.778+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2023-11-16T16:56:55.948831+0000 front 2023-11-16T16:56:55.948808+0000 (oldest deadline 2023-11-16T16:57:21.848736+0000) 2023-11-16T16:57:26.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:25 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:25.738+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:26.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:25 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:25.738+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:26.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:25 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:25.738+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:26.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:25 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:25.738+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:26.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:25 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:25.738+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:26.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:25 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:25.738+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:26.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:25 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:25.738+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2023-11-16T16:56:55.948831+0000 front 2023-11-16T16:56:55.948808+0000 (oldest deadline 2023-11-16T16:57:21.848736+0000) 2023-11-16T16:57:27.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:26 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:26.755+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-11-16T16:55:57.238223+0000 front 2023-11-16T16:55:57.238275+0000 (oldest deadline 2023-11-16T16:56:26.638946+0000) 2023-11-16T16:57:27.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:26 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:26.755+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2023-11-16T16:56:08.940009+0000 front 2023-11-16T16:56:08.939948+0000 (oldest deadline 2023-11-16T16:56:33.540469+0000) 2023-11-16T16:57:27.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:26 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:26.755+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-11-16T16:56:17.041704+0000 front 2023-11-16T16:56:17.041726+0000 (oldest deadline 2023-11-16T16:56:39.941963+0000) 2023-11-16T16:57:27.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:26 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:26.755+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2023-11-16T16:56:32.744727+0000 front 2023-11-16T16:56:32.744674+0000 (oldest deadline 2023-11-16T16:56:55.644616+0000) 2023-11-16T16:57:27.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:26 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:26.755+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-11-16T16:56:40.945442+0000 front 2023-11-16T16:56:40.945479+0000 (oldest deadline 2023-11-16T16:57:02.645533+0000) 2023-11-16T16:57:27.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:26 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:26.755+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-11-16T16:56:46.746750+0000 front 2023-11-16T16:56:46.746825+0000 (oldest deadline 2023-11-16T16:57:12.046652+0000) 2023-11-16T16:57:27.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:26 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11[153698]: 2023-11-16T16:57:26.755+0000 7f79a151d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2023-11-16T16:56:55.948831+0000 front 2023-11-16T16:56:55.948808+0000 (oldest deadline 2023-11-16T16:57:21.848736+0000) 2023-11-16T16:57:28.485 INFO:journalctl@ceph.osd.11.smithi159.stdout:Nov 16 16:57:28 smithi159 bash[162918]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-osd-11 2023-11-16T16:57:29.490 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@osd.11.service' 2023-11-16T16:57:29.531 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:57:29.531 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-11-16T16:57:29.532 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-11-16T16:57:29.532 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.a 2023-11-16T16:57:29.900 INFO:journalctl@ceph.mds.a.smithi093.stdout:Nov 16 16:57:29 smithi093 systemd[1]: Stopping Ceph mds.a for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:57:30.400 INFO:journalctl@ceph.mds.a.smithi093.stdout:Nov 16 16:57:30 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-a[132232]: 2023-11-16T16:57:30.066+0000 7f79e4920700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:57:30.400 INFO:journalctl@ceph.mds.a.smithi093.stdout:Nov 16 16:57:30 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-a[132232]: 2023-11-16T16:57:30.066+0000 7f79e4920700 -1 mds.a *** got signal Terminated *** 2023-11-16T16:57:31.684 INFO:journalctl@ceph.mds.a.smithi093.stdout:Nov 16 16:57:31 smithi093 bash[255551]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-a 2023-11-16T16:57:31.988 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.a.service' 2023-11-16T16:57:32.135 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:57:32.135 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-11-16T16:57:32.135 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-11-16T16:57:32.135 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.d 2023-11-16T16:57:32.400 INFO:journalctl@ceph.mds.d.smithi093.stdout:Nov 16 16:57:32 smithi093 systemd[1]: Stopping Ceph mds.d for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:57:32.900 INFO:journalctl@ceph.mds.d.smithi093.stdout:Nov 16 16:57:32 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-d[131274]: 2023-11-16T16:57:32.622+0000 7f8426e3e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:57:32.900 INFO:journalctl@ceph.mds.d.smithi093.stdout:Nov 16 16:57:32 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-d[131274]: 2023-11-16T16:57:32.622+0000 7f8426e3e700 -1 mds.d *** got signal Terminated *** 2023-11-16T16:57:37.150 INFO:journalctl@ceph.mds.d.smithi093.stdout:Nov 16 16:57:36 smithi093 bash[255697]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-d 2023-11-16T16:57:37.410 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.d.service' 2023-11-16T16:57:37.493 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:57:37.494 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-11-16T16:57:37.494 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-11-16T16:57:37.494 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.g 2023-11-16T16:57:37.900 INFO:journalctl@ceph.mds.g.smithi093.stdout:Nov 16 16:57:37 smithi093 systemd[1]: Stopping Ceph mds.g for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:57:38.400 INFO:journalctl@ceph.mds.g.smithi093.stdout:Nov 16 16:57:37 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-g[131887]: 2023-11-16T16:57:37.980+0000 7f263cbd3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:57:38.400 INFO:journalctl@ceph.mds.g.smithi093.stdout:Nov 16 16:57:37 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-g[131887]: 2023-11-16T16:57:37.980+0000 7f263cbd3700 -1 mds.g *** got signal Terminated *** 2023-11-16T16:57:40.650 INFO:journalctl@ceph.mds.g.smithi093.stdout:Nov 16 16:57:40 smithi093 bash[255842]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-g 2023-11-16T16:57:40.721 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.g.service' 2023-11-16T16:57:40.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:57:40.803 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-11-16T16:57:40.803 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-11-16T16:57:40.803 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.j 2023-11-16T16:57:41.150 INFO:journalctl@ceph.mds.j.smithi093.stdout:Nov 16 16:57:40 smithi093 systemd[1]: Stopping Ceph mds.j for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:57:41.650 INFO:journalctl@ceph.mds.j.smithi093.stdout:Nov 16 16:57:41 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-j[130888]: 2023-11-16T16:57:41.297+0000 7fe13df6f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:57:41.650 INFO:journalctl@ceph.mds.j.smithi093.stdout:Nov 16 16:57:41 smithi093 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-j[130888]: 2023-11-16T16:57:41.297+0000 7fe13df6f700 -1 mds.j *** got signal Terminated *** 2023-11-16T16:57:45.778 INFO:journalctl@ceph.mds.j.smithi093.stdout:Nov 16 16:57:45 smithi093 bash[255984]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-j 2023-11-16T16:57:45.988 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.j.service' 2023-11-16T16:57:46.049 INFO:journalctl@ceph.mds.j.smithi093.stdout:Nov 16 16:57:45 smithi093 systemd[1]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.j.service: Succeeded. 2023-11-16T16:57:46.049 INFO:journalctl@ceph.mds.j.smithi093.stdout:Nov 16 16:57:45 smithi093 systemd[1]: Stopped Ceph mds.j for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7. 2023-11-16T16:57:46.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:57:46.071 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-11-16T16:57:46.072 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-11-16T16:57:46.072 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.b 2023-11-16T16:57:46.507 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:57:46 smithi125 systemd[1]: Stopping Ceph mds.b for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:57:46.828 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:57:46 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-b[156540]: 2023-11-16T16:57:46.505+0000 7fbdafcb0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:57:46.828 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:57:46 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-b[156540]: 2023-11-16T16:57:46.505+0000 7fbdafcb0700 -1 mds.b *** got signal Terminated *** 2023-11-16T16:57:48.327 INFO:journalctl@ceph.mds.b.smithi125.stdout:Nov 16 16:57:48 smithi125 bash[161771]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-b 2023-11-16T16:57:48.405 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.b.service' 2023-11-16T16:57:48.439 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:57:48.440 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-11-16T16:57:48.440 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-11-16T16:57:48.440 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.e 2023-11-16T16:57:48.814 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:57:48 smithi125 systemd[1]: Stopping Ceph mds.e for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:57:49.078 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:57:48 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-e[155865]: 2023-11-16T16:57:48.812+0000 7f0e0cf9e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:57:49.078 INFO:journalctl@ceph.mds.e.smithi125.stdout:Nov 16 16:57:48 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-e[155865]: 2023-11-16T16:57:48.812+0000 7f0e0cf9e700 -1 mds.e *** got signal Terminated *** 2023-11-16T16:57:52.276 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.e.service' 2023-11-16T16:57:52.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:57:52.315 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-11-16T16:57:52.315 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-11-16T16:57:52.315 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.h 2023-11-16T16:57:52.689 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:57:52 smithi125 systemd[1]: Stopping Ceph mds.h for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:57:53.078 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:57:52 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-h[156202]: 2023-11-16T16:57:52.687+0000 7fdaf5c2d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:57:53.078 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:57:52 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-h[156202]: 2023-11-16T16:57:52.687+0000 7fdaf5c2d700 -1 mds.h *** got signal Terminated *** 2023-11-16T16:57:56.078 INFO:journalctl@ceph.mds.h.smithi125.stdout:Nov 16 16:57:55 smithi125 bash[162065]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-h 2023-11-16T16:57:56.196 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.h.service' 2023-11-16T16:57:56.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:57:56.231 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-11-16T16:57:56.231 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-11-16T16:57:56.232 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.k 2023-11-16T16:57:56.578 INFO:journalctl@ceph.mds.k.smithi125.stdout:Nov 16 16:57:56 smithi125 systemd[1]: Stopping Ceph mds.k for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:57:57.078 INFO:journalctl@ceph.mds.k.smithi125.stdout:Nov 16 16:57:56 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-k[155524]: 2023-11-16T16:57:56.611+0000 7f7072b16700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:57:57.078 INFO:journalctl@ceph.mds.k.smithi125.stdout:Nov 16 16:57:56 smithi125 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-k[155524]: 2023-11-16T16:57:56.611+0000 7f7072b16700 -1 mds.k *** got signal Terminated *** 2023-11-16T16:57:59.304 INFO:journalctl@ceph.mds.k.smithi125.stdout:Nov 16 16:57:59 smithi125 bash[162212]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-k 2023-11-16T16:57:59.428 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.k.service' 2023-11-16T16:57:59.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:57:59.463 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-11-16T16:57:59.463 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-11-16T16:57:59.463 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.c 2023-11-16T16:57:59.866 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:57:59 smithi159 systemd[1]: Stopping Ceph mds.c for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:58:00.234 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:57:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c[157316]: 2023-11-16T16:57:59.864+0000 7f2ae8ebb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:58:00.235 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:57:59 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c[157316]: 2023-11-16T16:57:59.864+0000 7f2ae8ebb700 -1 mds.c *** got signal Terminated *** 2023-11-16T16:58:04.235 INFO:journalctl@ceph.mds.c.smithi159.stdout:Nov 16 16:58:03 smithi159 bash[163198]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-c 2023-11-16T16:58:04.269 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.c.service' 2023-11-16T16:58:04.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:58:04.307 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-11-16T16:58:04.307 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-11-16T16:58:04.308 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.f 2023-11-16T16:58:04.687 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:58:04 smithi159 systemd[1]: Stopping Ceph mds.f for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:58:04.984 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:58:04 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-f[156974]: 2023-11-16T16:58:04.685+0000 7f7d3f655700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:58:04.985 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:58:04 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-f[156974]: 2023-11-16T16:58:04.685+0000 7f7d3f655700 -1 mds.f *** got signal Terminated *** 2023-11-16T16:58:08.371 INFO:journalctl@ceph.mds.f.smithi159.stdout:Nov 16 16:58:08 smithi159 bash[163355]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-f 2023-11-16T16:58:08.376 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.f.service' 2023-11-16T16:58:08.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:58:08.411 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-11-16T16:58:08.411 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-11-16T16:58:08.411 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.i 2023-11-16T16:58:08.734 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:58:08 smithi159 systemd[1]: Stopping Ceph mds.i for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:58:09.235 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:58:08 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-i[157654]: 2023-11-16T16:58:08.793+0000 7f5569553700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:58:09.235 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:58:08 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-i[157654]: 2023-11-16T16:58:08.793+0000 7f5569553700 -1 mds.i *** got signal Terminated *** 2023-11-16T16:58:12.235 INFO:journalctl@ceph.mds.i.smithi159.stdout:Nov 16 16:58:11 smithi159 bash[163501]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-i 2023-11-16T16:58:12.320 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.i.service' 2023-11-16T16:58:12.354 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:58:12.354 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-11-16T16:58:12.355 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-11-16T16:58:12.355 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.l 2023-11-16T16:58:12.734 INFO:journalctl@ceph.mds.l.smithi159.stdout:Nov 16 16:58:12 smithi159 systemd[1]: Stopping Ceph mds.l for 6a59c1a8-849e-11ee-8dbe-212e2dc638e7... 2023-11-16T16:58:13.234 INFO:journalctl@ceph.mds.l.smithi159.stdout:Nov 16 16:58:12 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-l[156635]: 2023-11-16T16:58:12.735+0000 7f0391b3a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:58:13.235 INFO:journalctl@ceph.mds.l.smithi159.stdout:Nov 16 16:58:12 smithi159 ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-l[156635]: 2023-11-16T16:58:12.735+0000 7f0391b3a700 -1 mds.l *** got signal Terminated *** 2023-11-16T16:58:16.703 INFO:journalctl@ceph.mds.l.smithi159.stdout:Nov 16 16:58:16 smithi159 bash[163654]: ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7-mds-l 2023-11-16T16:58:16.901 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6a59c1a8-849e-11ee-8dbe-212e2dc638e7@mds.l.service' 2023-11-16T16:58:16.937 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:58:16.937 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-11-16T16:58:16.937 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 --force --keep-logs 2023-11-16T16:58:21.281 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 --force --keep-logs 2023-11-16T16:58:23.251 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 --force --keep-logs 2023-11-16T16:58:25.201 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:58:25.279 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:58:25.316 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:58:25.345 INFO:tasks.cephadm:Archiving crash dumps... 2023-11-16T16:58:25.347 DEBUG:teuthology.misc:Transferring archived files from smithi093:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/crash to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918/remote/smithi093/crash 2023-11-16T16:58:25.348 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/crash -- . 2023-11-16T16:58:25.434 INFO:teuthology.orchestra.run.smithi093.stderr:tar: /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/crash: Cannot open: No such file or directory 2023-11-16T16:58:25.435 INFO:teuthology.orchestra.run.smithi093.stderr:tar: Error is not recoverable: exiting now 2023-11-16T16:58:25.438 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/crash to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918/remote/smithi125/crash 2023-11-16T16:58:25.439 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/crash -- . 2023-11-16T16:58:25.526 INFO:teuthology.orchestra.run.smithi125.stderr:tar: /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/crash: Cannot open: No such file or directory 2023-11-16T16:58:25.526 INFO:teuthology.orchestra.run.smithi125.stderr:tar: Error is not recoverable: exiting now 2023-11-16T16:58:25.528 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/crash to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918/remote/smithi159/crash 2023-11-16T16:58:25.530 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/crash -- . 2023-11-16T16:58:25.565 INFO:teuthology.orchestra.run.smithi159.stderr:tar: /var/lib/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/crash: Cannot open: No such file or directory 2023-11-16T16:58:25.566 INFO:teuthology.orchestra.run.smithi159.stderr:tar: Error is not recoverable: exiting now 2023-11-16T16:58:25.567 INFO:tasks.cephadm:Checking cluster log for badness... 2023-11-16T16:58:25.567 DEBUG:teuthology.orchestra.run.smithi093:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-11-16T16:58:25.650 INFO:teuthology.orchestra.run.smithi093.stderr:grep: /var/log/ceph/6a59c1a8-849e-11ee-8dbe-212e2dc638e7/ceph.log: No such file or directory 2023-11-16T16:58:25.655 INFO:tasks.cephadm:Compressing logs... 2023-11-16T16:58:25.655 DEBUG:teuthology.orchestra.run.smithi093:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-16T16:58:25.660 DEBUG:teuthology.orchestra.run.smithi125:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-16T16:58:25.663 DEBUG:teuthology.orchestra.run.smithi159:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-16T16:58:25.690 INFO:teuthology.orchestra.run.smithi159.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-16T16:58:25.693 INFO:teuthology.orchestra.run.smithi125.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-16T16:58:25.744 INFO:teuthology.orchestra.run.smithi093.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-16T17:01:24.815 INFO:tasks.cephadm:Archiving logs... 2023-11-16T17:01:24.816 DEBUG:teuthology.misc:Transferring archived files from smithi093:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918/remote/smithi093/log 2023-11-16T17:01:24.817 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-16T17:01:32.326 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918/remote/smithi125/log 2023-11-16T17:01:32.327 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-16T17:01:53.801 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918/remote/smithi159/log 2023-11-16T17:01:53.803 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-16T17:02:10.112 INFO:tasks.cephadm:Removing cluster... 2023-11-16T17:02:10.112 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 --force 2023-11-16T17:02:11.504 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 --force 2023-11-16T17:02:12.531 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6a59c1a8-849e-11ee-8dbe-212e2dc638e7 --force 2023-11-16T17:02:13.525 INFO:tasks.cephadm:Removing cephadm ... 2023-11-16T17:02:13.526 DEBUG:teuthology.orchestra.run.smithi093:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-16T17:02:13.578 DEBUG:teuthology.orchestra.run.smithi125:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-16T17:02:13.600 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-16T17:02:13.617 INFO:tasks.cephadm:Teardown complete 2023-11-16T17:02:13.618 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-16T17:02:13.627 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-16T17:02:13.628 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-16T17:02:13.632 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-16T17:02:13.642 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-16T17:02:13.778 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-16T17:02:13.787 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-16T17:02:13.787 DEBUG:teuthology.orchestra.run.smithi093:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T17:02:13.792 DEBUG:teuthology.orchestra.run.smithi125:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T17:02:13.795 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T17:02:13.810 INFO:teuthology.orchestra.run.smithi159.stderr:bash: ntpq: command not found 2023-11-16T17:02:13.811 INFO:teuthology.orchestra.run.smithi125.stderr:bash: ntpq: command not found 2023-11-16T17:02:13.814 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T17:02:13.814 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2023-11-16T17:02:13.814 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 8 +1251us[+1251us] +/- 69ms 2023-11-16T17:02:13.814 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv02.front.sepia.ceph.com 3 6 377 10 -383us[ -378us] +/- 70ms 2023-11-16T17:02:13.815 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -496us[ -496us] +/- 66ms 2023-11-16T17:02:13.815 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T17:02:13.816 INFO:teuthology.orchestra.run.smithi125.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T17:02:13.816 INFO:teuthology.orchestra.run.smithi125.stdout:=============================================================================== 2023-11-16T17:02:13.816 INFO:teuthology.orchestra.run.smithi125.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 11 +1239us[+1239us] +/- 69ms 2023-11-16T17:02:13.817 INFO:teuthology.orchestra.run.smithi125.stdout:^* hv02.front.sepia.ceph.com 3 6 377 13 -410us[ -409us] +/- 70ms 2023-11-16T17:02:13.817 INFO:teuthology.orchestra.run.smithi125.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 -517us[ -517us] +/- 66ms 2023-11-16T17:02:13.817 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T17:02:13.831 INFO:teuthology.orchestra.run.smithi093.stderr:bash: ntpq: command not found 2023-11-16T17:02:13.882 INFO:teuthology.orchestra.run.smithi093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T17:02:13.882 INFO:teuthology.orchestra.run.smithi093.stdout:=============================================================================== 2023-11-16T17:02:13.883 INFO:teuthology.orchestra.run.smithi093.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 12 +1268us[+1268us] +/- 69ms 2023-11-16T17:02:13.883 INFO:teuthology.orchestra.run.smithi093.stdout:^* hv02.front.sepia.ceph.com 3 7 377 14 -320us[ -333us] +/- 70ms 2023-11-16T17:02:13.883 INFO:teuthology.orchestra.run.smithi093.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -497us[ -497us] +/- 66ms 2023-11-16T17:02:13.883 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T17:02:13.884 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-16T17:02:13.898 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-16T17:02:13.899 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-16T17:02:13.977 DEBUG:teuthology.orchestra.run.smithi093:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-16T17:02:14.236 DEBUG:teuthology.orchestra.run.smithi125:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-16T17:02:14.339 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-16T17:02:14.435 DEBUG:teuthology.orchestra.run.smithi093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T17:02:14.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T17:02:14.511 DEBUG:teuthology.orchestra.run.smithi125:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T17:02:14.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T17:02:14.545 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T17:02:14.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T17:02:14.577 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-16T17:02:14.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-16T17:02:14.636 INFO:teuthology.task.internal:Duration was 2532.787746 seconds 2023-11-16T17:02:14.636 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-16T17:02:14.696 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-16T17:02:14.696 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-16T17:02:14.701 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-16T17:02:14.703 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-16T17:02:14.750 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T17:02:14.751 INFO:teuthology.orchestra.run.smithi125.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T17:02:14.893 INFO:teuthology.orchestra.run.smithi093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T17:02:15.220 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-16T17:02:15.220 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi093.front.sepia.ceph.com 2023-11-16T17:02:15.220 DEBUG:teuthology.orchestra.run.smithi093:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-16T17:02:15.302 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi125.front.sepia.ceph.com 2023-11-16T17:02:15.303 DEBUG:teuthology.orchestra.run.smithi125:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-16T17:02:15.327 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2023-11-16T17:02:15.327 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-16T17:02:15.350 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-16T17:02:15.350 DEBUG:teuthology.orchestra.run.smithi093:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-16T17:02:15.355 DEBUG:teuthology.orchestra.run.smithi125:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-16T17:02:15.370 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-16T17:02:15.488 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-16T17:02:15.489 DEBUG:teuthology.orchestra.run.smithi093:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-16T17:02:15.493 DEBUG:teuthology.orchestra.run.smithi125:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-16T17:02:15.496 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-16T17:02:15.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-16T17:02:15.673 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-16T17:02:15.673 DEBUG:teuthology.orchestra.run.smithi093:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-16T17:02:15.753 DEBUG:teuthology.orchestra.run.smithi125:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-16T17:02:15.780 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-16T17:02:15.808 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-16T17:02:15.818 DEBUG:teuthology.orchestra.run.smithi093:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-16T17:02:15.823 DEBUG:teuthology.orchestra.run.smithi125:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-16T17:02:15.825 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-16T17:02:15.848 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern = core 2023-11-16T17:02:15.875 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2023-11-16T17:02:15.904 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern = core 2023-11-16T17:02:15.953 DEBUG:teuthology.orchestra.run.smithi093:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-16T17:02:15.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T17:02:15.995 DEBUG:teuthology.orchestra.run.smithi125:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-16T17:02:16.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T17:02:16.011 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-16T17:02:16.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T17:02:16.028 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-16T17:02:16.039 INFO:teuthology.task.internal:Transferring archived files... 2023-11-16T17:02:16.040 DEBUG:teuthology.misc:Transferring archived files from smithi093:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918/remote/smithi093 2023-11-16T17:02:16.040 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-16T17:02:16.123 DEBUG:teuthology.misc:Transferring archived files from smithi125:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918/remote/smithi125 2023-11-16T17:02:16.123 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-16T17:02:16.162 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918/remote/smithi159 2023-11-16T17:02:16.163 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-16T17:02:16.202 INFO:teuthology.task.internal:Removing archive directory... 2023-11-16T17:02:16.202 DEBUG:teuthology.orchestra.run.smithi093:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-16T17:02:16.207 DEBUG:teuthology.orchestra.run.smithi125:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-16T17:02:16.210 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-16T17:02:16.253 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-16T17:02:16.266 INFO:teuthology.task.internal:Not uploading archives. 2023-11-16T17:02:16.266 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-16T17:02:16.303 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-16T17:02:16.303 DEBUG:teuthology.orchestra.run.smithi093:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-16T17:02:16.308 DEBUG:teuthology.orchestra.run.smithi125:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-16T17:02:16.310 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-16T17:02:16.323 INFO:teuthology.orchestra.run.smithi125.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 16 17:02 /home/ubuntu/cephtest 2023-11-16T17:02:16.326 INFO:teuthology.orchestra.run.smithi159.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 16 17:02 /home/ubuntu/cephtest 2023-11-16T17:02:16.346 INFO:teuthology.orchestra.run.smithi093.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 16 17:02 /home/ubuntu/cephtest 2023-11-16T17:02:16.350 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-16T17:02:16.370 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-16T17:02:16.504 INFO:teuthology.nuke:Checking targets against current locks 2023-11-16T17:02:16.530 DEBUG:teuthology.nuke:shortname: smithi093 2023-11-16T17:02:16.531 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-16T17:02:16.555 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 16:10:25.786979', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T17:02:16.559 DEBUG:teuthology.nuke:shortname: smithi125 2023-11-16T17:02:16.559 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-16T17:02:16.582 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 16:10:25.788387', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T17:02:16.588 DEBUG:teuthology.nuke:shortname: smithi159 2023-11-16T17:02:16.589 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-16T17:02:16.615 INFO:teuthology.orchestra.console.smithi093:Power off 2023-11-16T17:02:16.615 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-16T17:02:16.625 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460918', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 16:10:25.789645', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T17:02:16.638 DEBUG:teuthology.orchestra.console.smithi093:power off output: Chassis Power Control: Down/Off 2023-11-16T17:02:16.638 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T17:02:16.651 INFO:teuthology.orchestra.console.smithi125:Power off 2023-11-16T17:02:16.651 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-16T17:02:16.663 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on 2023-11-16T17:02:16.672 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off 2023-11-16T17:02:16.672 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T17:02:16.693 INFO:teuthology.orchestra.console.smithi159:Power off 2023-11-16T17:02:16.693 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-16T17:02:16.703 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2023-11-16T17:02:16.715 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2023-11-16T17:02:16.715 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T17:02:16.738 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2023-11-16T17:02:20.664 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T17:02:20.704 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T17:02:20.739 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T17:02:20.774 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on 2023-11-16T17:02:20.814 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2023-11-16T17:02:20.848 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2023-11-16T17:02:24.778 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T17:02:24.815 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T17:02:24.849 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T17:02:24.889 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is off 2023-11-16T17:02:24.924 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2023-11-16T17:02:24.959 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2023-11-16T17:02:24.990 INFO:teuthology.orchestra.console.smithi093:Power off completed 2023-11-16T17:02:25.024 INFO:teuthology.orchestra.console.smithi125:Power off completed 2023-11-16T17:02:25.060 INFO:teuthology.orchestra.console.smithi159:Power off completed 2023-11-16T17:02:25.166 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 description: fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/misc}} duration: 2532.787746191025 failure_reason: 'Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi093 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh''' flavor: default owner: scheduled_choffman@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=bd6ad00590474c61b9881987bce57f35 status: fail success: false 2023-11-16T17:02:25.166 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T17:02:25.311 INFO:teuthology.run:FAIL